Image processing method, device and storage medium

Abstract

This application provides an image processing method, apparatus, and storage medium, including acquiring a first image frame, determining a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is a luminance channel image of the first image frame, the second resolution image is a color channel image of the first image frame, and the first resolution is higher than the second resolution; generating a first target image based on the first resolution image and the second resolution image, thereby processing only the high-resolution luminance channel and the low-resolution color channel of the first image frame, simplifying the processing flow, shortening the processing time, and thus improving image processing efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus and storage medium. Background Technology

[0002] As image processing technology plays an increasingly important role in fields such as intelligent recognition, fingerprint detection, and autonomous driving, the importance of optimizing image processing speed while ensuring image quality is self-evident.

[0003] For example, during the image reconstruction process, the color space processing of the image captured by the camera involves processing a large number of three channels simultaneously, which greatly increases the image processing time. For users, the impact on both power consumption and performance is unacceptable. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and storage medium to improve image processing efficiency.

[0005] In a first aspect, embodiments of this application provide an image processing method, including:

[0006] A first image frame is acquired, and a first resolution image and a second resolution image of the first image frame are determined, wherein the first resolution image is the luminance channel image of the first image frame, and the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution.

[0007] A first target image is generated based on the first resolution image and the second resolution image.

[0008] In some embodiments, generating a first target image based on the first resolution image and the second resolution image includes:

[0009] A target second resolution image is generated based on the first resolution image, corresponding to the second resolution image, wherein the target second resolution image is a color channel image, and the target second resolution is equal to the first resolution;

[0010] A first target image is generated based on the target second resolution image and the first resolution image.

[0011] In some embodiments, generating a target second resolution image corresponding to the second resolution image based on the first resolution image includes:

[0012] Determine the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image;

[0013] Based on the brightness distribution of the first pixel and the brightness distribution of the second pixel, the second resolution image is mapped onto the color channel that matches the first resolution image to obtain the target second resolution image.

[0014] In some embodiments, the brightness distribution of the first pixel is the neighborhood distribution and value range distribution of the brightness of each first pixel, and the brightness distribution of the second pixel is the neighborhood distribution and value range distribution of the brightness of each second pixel.

[0015] In some embodiments, mapping the second resolution image onto a color channel matching the first resolution image based on the first pixel brightness distribution and the second pixel brightness distribution to obtain a target second resolution image includes:

[0016] Based on the brightness distribution of the first pixel and the brightness distribution of the second pixel, several pairs of first pixel points are determined, wherein each pair of first pixel points includes a first pixel point and a second pixel point with the smallest difference in brightness distribution.

[0017] The second pixel in each of the first pixel pairs is mapped onto the color channel that matches the first pixel to obtain the target second resolution image.

[0018] In some embodiments, before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes:

[0019] The second resolution image is subjected to a preset image smoothing process, wherein the preset image smoothing process includes at least one of the following processing methods: mean filtering, median filtering, Gaussian filtering, and edge filtering.

[0020] In some embodiments, before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes:

[0021] The second-resolution image is transformed from the spatial domain to the spectral domain based on discrete cosine transform.

[0022] Gaussian filtering is applied to the second-resolution image in the spectral domain, and then inverse discrete cosine transform is performed on the Gaussian-filtered second-resolution image to obtain the second-resolution image converted back to the spatial domain.

[0023] In some embodiments, determining the first resolution image and the second resolution image of the first image frame includes:

[0024] The first image frame is downsampled to obtain the second image frame;

[0025] Determine the first resolution image of the first image frame and the second resolution image of the second image frame.

[0026] In some embodiments, before generating the first target image based on the first resolution image and the second resolution image, the method further includes:

[0027] The image at the first resolution is subjected to image noise reduction and sharpening processing.

[0028] In some embodiments, before determining the first resolution image and the second resolution image of the first image frame, the method further includes:

[0029] Determine the image acquisition spectral information of the first image frame;

[0030] The color space mapping relationship of the first image frame is determined based on the image acquisition spectral information, wherein the color space mapping relationship is used to determine the first resolution image and the second resolution image of the first image frame.

[0031] Secondly, embodiments of this application also provide an image processing method, including:

[0032] Acquire a third-resolution image of a third image frame and a fourth-resolution image of a fourth image frame, wherein the third-resolution image is a luminance channel image and the fourth-resolution image is a color channel image, and the third resolution is higher than the fourth resolution;

[0033] A second target image is generated based on the fourth resolution image and the third resolution image.

[0034] In some embodiments, generating a second target image based on the fourth resolution image and the third resolution image includes:

[0035] A target fourth resolution image corresponding to the fourth resolution image is generated based on the third resolution image, wherein the target fourth resolution image is a color channel image and the target fourth resolution is equal to the third resolution;

[0036] A second target image is generated based on the target fourth-resolution image and the target third-resolution image.

[0037] In some embodiments, generating a target fourth resolution image corresponding to the fourth resolution image based on the third resolution image includes:

[0038] Based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, several pairs of second pixel points are determined, wherein each pair of second pixel points includes the third pixel point and the fourth pixel point with the smallest difference in brightness distribution.

[0039] The fourth pixel in each of the second pixel pairs is mapped onto the color channel matching the third pixel to obtain the target fourth resolution image.

[0040] Thirdly, embodiments of this application also provide an image processing apparatus, comprising:

[0041] The first acquisition unit is configured to acquire a first image frame and determine a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is the luminance channel image of the first image frame, the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution.

[0042] The first generation unit is used to generate a first target image based on the first resolution image and the second resolution image.

[0043] Fourthly, embodiments of this application also provide an image processing apparatus, comprising:

[0044] The second acquisition unit is used to acquire the third resolution image of the third image frame and the fourth resolution image of the fourth image frame, wherein the third resolution image is a luminance channel image, the fourth resolution image is a color channel image, and the third resolution is higher than the fourth resolution.

[0045] The second generation unit is used to generate a second target image based on the target fourth resolution image and the target third resolution image.

[0046] Fifthly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the image processing methods described above.

[0047] Sixthly, embodiments of this application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described above.

[0048] In a seventh aspect, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the image processing method described above. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is one of the schematic flowcharts of an image processing method provided in an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of an image processing scene provided in one embodiment of this application;

[0052] Figure 3 This is one of the schematic flowcharts of an image processing method provided in an embodiment of this application;

[0053] Figure 4 This is one of the structural schematic diagrams of an image processing apparatus provided in one embodiment of this application;

[0054] Figure 5 This is a second schematic diagram of the structure of an image processing apparatus provided in one embodiment of this application;

[0055] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0056] To make the technical solutions and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] In practical applications, most image processing focuses on processing the three channels of a specific color space simultaneously. For example, when a mobile phone camera captures a high-resolution image, it uses multi-frame original image synthesis for processing. In the process of processing the original image, all three channels of the original image need to be processed on the same frame size. Although this process can effectively improve clarity, it increases the complexity many times over, resulting in a significant increase in processing time. For users, the impact on both power consumption and performance is unacceptable, especially for low-end chips, which further affects the overall smoothness of the system.

[0058] Therefore, for the purpose of this application to propose an image processing method, the executing entity can be any terminal or functional module within a terminal capable of implementing an image processing method. The image processing method provided in this application will be described below using a mobile terminal as an example of the executing entity. It should be noted that the aforementioned executing entity does not constitute a limitation on this application.

[0059] In this document, "terminal" (terminal device) includes various handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless communication capabilities, such as mobile phones, tablets, desktop laptops, and smart devices capable of running applications, including the central control console of a smart car. Specifically, it can refer to user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device. Terminal devices can also be satellite phones, cellular phones, smartphones, wireless data cards, wireless modems, machine-type communication devices, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices or wearable devices, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical care, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, terminal devices in 5G networks, or terminal devices in future communication networks. Terminals can be battery-powered or attached to and powered by the power system of a vehicle or vessel. The power system of a vehicle or ship can also charge the terminal's battery to extend the terminal's communication time.

[0060] The following is combined Figure 1 This application describes an image processing method provided in one embodiment. Figure 1 This is one of the schematic flowcharts of an image processing method provided in an embodiment of this application, such as... Figure 1 As shown, the image processing methods include:

[0061] Step 101: Obtain the first image frame and determine the first resolution image and the second resolution image of the first image frame;

[0062] For example, the first image frame can be input by the user, or it can be generated by other hardware devices or transmitted by software.

[0063] The user input referred to can be input by the user through touch input, voice input, gesture input, visual input, brain-computer input, etc.

[0064] The term "hardware device generation" can refer to information generated by a terminal under certain conditions, such as automatic image acquisition by the terminal's image acquisition device when a preset image acquisition time point is reached. For example, the first image frame can be an image frame with the original resolution output by the terminal's image acquisition device. The terminal can be a smartphone, tablet, smart wearable device, camera, or camcorder, etc. Different terminals acquiring the same object will obtain image data with different or the same colors.

[0065] The so-called software transmission can be transmitted from local third-party software or cloud applications to image processing software on the terminal.

[0066] The first image frame can be a separately acquired image frame or an image frame extracted from a video stream.

[0067] Wherein, the first resolution image is the luminance channel image of the first image frame, the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution.

[0068] For example, the first image frame can be an image frame with an orthogonal color space, such as an image frame in the Lαβ color space. Alternatively, the first image frame can be an image frame in an orthogonal color space obtained by converting an image frame in a three-channel color space such as RGB or YUV. For example, an image frame in the RGB color space can be converted into an image frame in the Lαβ color space; or an image frame in the YUV color space can be converted into an image frame in the Lαβ color space. This application will not elaborate further on these variations.

[0069] The Lαβ color space is divided into a luminance component L and two chromaticity components: α (yellow-blue channel) and β (red-green channel).

[0070] The Lαβ color space not only largely eliminates the strong correlation between color components, but also effectively separates the grayscale and color information of an image. Therefore, independent operations can be performed on the three channels of the image without modifying the information of the other two channels, thus not affecting the natural effect of the original image.

[0071] For example, when the first image frame is an image frame in the Lαβ color space, the first resolution image can be the luminance channel image corresponding to the luminance component L at high resolution, and the second resolution image can be the color channel image corresponding to the chrominance component αβ at low resolution.

[0072] The second resolution image can be extracted from a low-resolution image frame obtained by downsampling the first image frame, or the second resolution image can be obtained by high-to-low resolution mapping of the first resolution color channel image extracted from the first image frame. This application does not limit this.

[0073] Step 102: Generate a first target image based on the first resolution image and the second resolution image.

[0074] Understandably, the first target image is a high-resolution color image with the same resolution as the first resolution, and the first target image has both the luminance component L and the chrominance component αβ.

[0075] For example, the explanation will be based on the luminance channel image of an image frame with a first resolution image in the Lαβ color space and the color channel image of an image frame with a second resolution image in the Lαβ color space:

[0076] In this embodiment, a low-resolution (i.e., second-resolution) color channel image and a high-resolution (i.e., first-resolution) luminance channel image are fused. During the fusion process, pixel-related data processing operations need to be performed on the luminance component L of the luminance channel image and the chrominance component αβ of the color channel image, respectively. Then, the processed color channel image and luminance channel image are fused to obtain the final first target image.

[0077] For example, please refer to Figure 2 , Figure 2 This illustration shows a schematic diagram of the fusion process of a color channel image and a luminance channel image according to an embodiment of this application. From left to right, the images are a low-resolution color channel image, a high-resolution luminance channel image, a color image after the fusion of the color channel image and the luminance channel image, wherein the image resolution of the fused color image is equal to the first resolution.

[0078] When performing image fusion, the low-resolution color channel image is first selected for processing, while the high-resolution luminance channel image is retained for fusion.

[0079] The reason for the above processing is that the detailed information of the image is concentrated in the luminance channel image. Processing the pixels contained in the low-resolution color channel image does not affect the detailed information in the original image, and it takes less time than processing the high-resolution color channel image in the original image. Therefore, the overall image processing efficiency is also faster.

[0080] In this embodiment of the application, in order to improve image processing efficiency, a low-resolution color channel image is used to replace the original high-resolution color channel image and the luminance channel image for image fusion to obtain the desired first target image. Since the detailed information of the image is concentrated in the luminance channel, using a low-resolution color channel image and the original high-resolution luminance channel image can shorten the processing time and obtain the desired first target image.

[0081] In some implementations, generating the first target image based on the first resolution image and the second resolution image includes:

[0082] Generate a target second-resolution image corresponding to the second-resolution image based on the first-resolution image;

[0083] A first target image is generated based on the second-resolution target image and the first-resolution target image.

[0084] The target second resolution image is a color channel image, and the target second resolution is equal to the first resolution.

[0085] Understandably, the resolution of the first resolution image is higher than that of the second resolution image, and the details of the image are basically concentrated in the first resolution image (i.e., the luminance channel image). Therefore, while ensuring minimal loss of imaging quality to improve imaging speed, after obtaining the second resolution image with a lower resolution, in order to effectively preserve color authenticity, the second resolution image is restored to the target second resolution image with the same resolution as the first resolution image, and then image fusion is performed to generate the first target image.

[0086] In one example, a second-resolution image can be restored to a target second-resolution image consistent with the first resolution image by using a mapping matrix between the first-resolution image and the second-resolution image.

[0087] In another example, the second resolution image can be restored to a target second resolution image consistent with the first resolution image by determining the first total pixel information corresponding to the first resolution image and the second total pixel information corresponding to the second resolution image by using the pixel differences between the first total pixel information and the second total pixel information.

[0088] In this embodiment, since the detailed features of the image are concentrated in the first resolution image (i.e., the luminance channel image) of the first image frame, the image details of the first resolution image (i.e., the luminance channel image) are preserved during the processing. The first resolution image (i.e., the luminance channel image) is used as a reference image to process the second resolution image (i.e., the color channel image). This simplifies the image processing steps while effectively preserving the color authenticity, thereby reducing the image processing time.

[0089] In some implementations, generating a target second resolution image corresponding to the second resolution image based on the first resolution image includes:

[0090] Determine the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image;

[0091] Based on the brightness distribution of the first pixel and the brightness distribution of the second pixel, the second resolution image is mapped onto the color channel that matches the first resolution image to obtain the target second resolution image.

[0092] For example, the pixel brightness distribution can be the pixel value distribution of the surrounding pixels centered on the pixel; or it can be the pixel value difference distribution between the surrounding pixels and the central pixel centered on the pixel.

[0093] For example, the pixel brightness distribution can also be the neighborhood distribution and value range distribution of pixel brightness. For instance, a first resolution image includes 768×576 pixels. The neighborhood distribution of the first pixel is the brightness value of the 100 surrounding pixels centered on the first pixel and the brightness value of the center point. The value range distribution of the first pixel is the difference in brightness value between the 100 surrounding pixels and the center point.

[0094] It should be understood that the above pixel brightness distribution can refer to the brightness distribution of the first pixel or the brightness distribution of the second pixel, which will not be elaborated further here.

[0095] In some implementations, to simplify calculations, every n*n second pixels in the second resolution image can be treated as a single pixel block, and every m*m first pixels in the first resolution image can be treated as a single pixel block. Here, n and m are in a multiple relationship, and the multiple is usually a natural number greater than 1.

[0096] For example, in the second resolution image, each 3*3 first pixel point is treated as a unit pixel block, and the brightness value of each second pixel point in each pixel block and its difference with the brightness values ​​of the other eight pixels in the pixel block are calculated. In the first resolution image, each 9*9 first pixel point is treated as a unit pixel block, and the brightness value of each 3*3 unit sub-pixel block in each 9*9 pixel block and its difference with the brightness values ​​of the other eight 3*3 unit sub-pixel blocks are calculated.

[0097] In this embodiment, the pixel brightness distribution can be used to accurately map each pixel in the second resolution image to the matching color channel of the first resolution image, thereby ensuring the color authenticity of the color channel image.

[0098] In some implementations, mapping the second resolution image onto a color channel matching the first resolution image based on the first pixel brightness distribution and the second pixel brightness distribution to obtain the target second resolution image includes:

[0099] Several pairs of first pixels are determined based on the brightness distribution of the first pixel and the brightness distribution of the second pixel;

[0100] The second pixel in each of the first pixel pairs is mapped onto the color channel that matches the first pixel to obtain the target second resolution image.

[0101] Among them, the pixel pair includes the first pixel and the second pixel with the smallest difference in brightness distribution. The smallest difference in brightness distribution means that the difference in brightness distribution between the first pixel and the second pixel is the smallest.

[0102] In one example, a pixel pair can be the pair of pixels whose pixel value distributions have the smallest difference among the surrounding pixels of the central pixel. For instance, if the resolution ratio of the images corresponding to pixel a and pixel b is A, then pixel a and pixel b are a pixel pair if the pixel value distributions of the B pixels surrounding pixel a are the smallest compared to the pixel value distributions of the B / A pixels surrounding pixel b. Typically, A and B are natural numbers greater than 1.

[0103] In one example, a pixel pair can also be the pixel pair where the difference between the pixel values ​​of the surrounding pixels and the central pixel is the smallest. For example, if the resolution ratio of the images corresponding to pixel a and pixel b is A, then pixel a and pixel b are a pixel pair if the difference between the pixel values ​​of the B surrounding pixels centered on pixel a and pixel a is the smallest, and the difference between the pixel values ​​of the B / A surrounding pixels centered on pixel b and pixel b is the smallest.

[0104] In one example, a pixel pair can also be a pair where the differences in pixel value distribution among the surrounding pixels of the central pixel are minimal, and the differences in pixel value differences between the surrounding pixels and the central pixel are minimal. For example, if the resolution ratio of the images corresponding to pixel a and pixel b is A, then pixel a and pixel b are a pixel pair if the differences in pixel values ​​among the B surrounding pixels of pixel a are minimal, and the differences in pixel value differences between the B surrounding pixels of pixel a and pixel a are minimal, and the differences in pixel value differences between the B surrounding pixels of pixel a and pixel a are minimal, and the differences in pixel value differences between the B surrounding pixels of pixel a and pixel b are minimal, and the differences in pixel value differences between the B surrounding pixels of pixel b and pixel b are minimal.

[0105] Since the second resolution image is a low-resolution image corresponding to the first resolution image, that is, the value of each second pixel in the second resolution image corresponds to the average value of all first pixels in the matching S*S pixel block in the first resolution image, the color channel data of the second pixel can be evenly distributed to the corresponding color channel according to the 1-to-S mapping relationship during the mapping process, thus achieving the effect of color transfer.

[0106] For example, after obtaining the pixel brightness distribution, in order to accurately match several first pixel pairs, the brightness distribution of the first pixel and the brightness distribution of the second pixel can be normalized and compared. This can identify the second pixel with the smallest brightness difference and the target sub-pixel block. Then, the pixel value of each first pixel in the target sub-pixel block and its difference from the brightness values ​​of the other eight first pixels are calculated and normalized and compared again to obtain the first pixel with the smallest brightness difference in the target sub-pixel block for each second pixel, forming several pixel pairs. This makes the first color channel image after color transfer more accurate.

[0107] In some embodiments, before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes performing a preset image smoothing process on the second resolution image.

[0108] Understandably, the purpose of pre-setting image smoothing is to enable more accurate calculation of the brightness value of each pixel in the subsequent process.

[0109] The preset image smoothing process includes at least one of the following processing methods: mean filtering, median filtering, Gaussian filtering, and edge filtering. This application does not limit this process.

[0110] The filtering processes of mean filtering, median filtering, Gaussian filtering, and edge filtering can be the same as or different from the filtering methods in the prior art, and this application does not impose any restrictions on this.

[0111] In some embodiments, before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes:

[0112] The second-resolution image is transformed from the spatial domain to the spectral domain based on discrete cosine transform.

[0113] Gaussian filtering is applied to the second-resolution image in the spectral domain, and then inverse discrete cosine transform is performed on the Gaussian-filtered second-resolution image to obtain the second-resolution image converted back to the spatial domain.

[0114] Here, the original second-resolution image is denoted as O(r,θ), and the second-resolution image after discrete cosine transform, Gaussian filtering, and inverse discrete cosine transform is denoted as O'(r,θ).

[0115] The functional relationship between O(r,θ) and O'(r,θ) is as follows:

[0116] O'(r,θ)=D -1 [D[O(r,θ)]*Gauss(r-r0,θ)]

[0117] Among them, D -1 [] represents the inverse discrete cosine transform, D[] represents the discrete cosine transform, and Gauss(r-r0,θ) represents a function whose value varies with the radius of the circle centered at the top left corner of the image.

[0118] In this embodiment, after converting the second resolution image to the spectral domain using discrete cosine transform, it is subjected to Gaussian filtering with a relatively low smoothness, and then subjected to inverse discrete cosine transform back to the spatial domain. This makes the pixel transitions of the processed second resolution image smoother, thereby enabling more accurate calculation of the brightness value of each pixel.

[0119] In some embodiments, determining the first resolution image and the second resolution image of the first image frame includes:

[0120] The first image frame is downsampled to obtain the second image frame;

[0121] Determine the first resolution image of the first image frame and the second resolution image of the second image frame.

[0122] For example, when the first image frame is an image frame in the RGB color space, it can be understood that an image frame in the RGB color space is essentially an M*N*3 array of colored pixels, and each colored pixel is a three-value group. These three values ​​correspond to the red, green and blue components of an image frame located at a specific spatial position in the RGB color space.

[0123] Specifically, downsampling refers to downsampling the first image frame of size M*N by a factor of S to obtain a second image frame with a resolution of (M / S)*(N / S), which is to reduce the size of the image and thus filter out pixels with little effect and redundant information while retaining key information.

[0124] Wherein, with a downsampling factor of S times, the value of each second pixel in the second image frame corresponds to the average value of all pixels in a pixel block of size S*S that matches the first pixel in the first image frame.

[0125] In some embodiments, the value of the downsampling factor S mainly depends on the time parameters and image quality parameters in the image processing process. For example, the downsampling factor is usually the common divisor of M and N. Thus, when downsampling is performed, the first image frame can be divided into an integer number of S*S pixel blocks, which facilitates subsequent image processing.

[0126] Because pixels with little effect and redundant information are filtered out during the downsampling process of the first image frame, the quality of processing using the downsampled image frame is higher than that of processing directly using two image frames of different resolutions.

[0127] In some embodiments, such as when the first image frame and the second image frame are image frames in RGB color space or YUV color space, in order to extract the luminance channel image and the color channel image, the first image frame can be converted to a color space to obtain an image frame in a first Lαβ color space, and the second image frame can be converted to a color space to obtain an image frame in a second Lαβ color space; then the luminance channel image can be extracted from the image frame in the first Lαβ color space, and the color channel image can be extracted from the image frame in the second Lαβ color space.

[0128] Since the three channels of the RGB color space have strong correlation, while the channels of the Lαβ color space have minimal correlation, it becomes very convenient to convert the image frame in the RGB color space to an image frame in the Lαβ color space in this embodiment, and then process the image frame in the Lαβ color space.

[0129] In some embodiments, converting an image frame in RGB color space to an image frame in Lαβ color space can be done in two steps. For example, first convert the image frame in RGB color space to an image frame in LMS color space, and then perform decorrelation to convert it to an image frame in Lαβ color space. Of course, converting an image frame in YUV color space to an image frame in Lαβ color space can also be done in two steps.

[0130] For example, the following formula is first used to convert an image frame in the RGB color space into an image frame in the LMS color space;

[0131]

[0132] Where L, M, and S are values ​​in the LMS color space, and R, G, and B are values ​​in the RGB color space.

[0133] Next, the image frame in the LMS color space is converted into an image frame in the Lαβ color space using the following formula.

[0134]

[0135] Here, L, α, and β are the values ​​in the Lαβ color space. Taking the logarithm of each of the three LMS channels is to converge the data and make the color distribution more in line with human visual perception.

[0136] In some embodiments, converting an image frame in the RGB color space to an image frame in the Lαβ color space can be performed in three steps. For example, first convert the image frame in the RGB color space to an image frame in the CIEXYZ color space, then convert the image frame in the CIEXYZ color space to an image frame in the LMS color space, and finally perform decorrelation to convert it to an image frame in the Lαβ color space.

[0137] The reason for the above processing is that the RGB color space is a color space that is independent of the device and the lighting. However, in actual applications, the imaging effect is different under different lighting conditions. Therefore, in this embodiment, the image frame in the RGB color space is linearly transformed to obtain an image frame in the CIEXYZ color space that is independent of the device and has a luminance channel added.

[0138] Since the detailed features of the image are concentrated in the first resolution image (i.e., the luminance channel image) of the first image frame, this embodiment downsamples the first image frame and extracts the second resolution image (i.e., the color channel image) from the downsampled second image frame. This allows subsequent image fusion to be performed using the second resolution image with less data. As the amount of data to be processed is reduced, the processing time will also be shortened.

[0139] In some embodiments, before generating the first target image based on the first resolution image and the second resolution image, image denoising and sharpening processing can be performed on the first resolution image, thereby enhancing image details and reducing the impact of image noise.

[0140] Since images may contain some noise during the acquisition process due to factors such as equipment, in order to ensure that the synthesized image is more accurate, in this embodiment, image noise reduction is performed on the first resolution image before generating the first target image.

[0141] In one example, image denoising can be performed by using a mean filter, a median filter, a morphological noise filter, etc. In another example, image denoising can be performed using the device's built-in image signal processor (ISP).

[0142] In another example, to enhance image clarity, this step also involves sharpening the image, thereby highlighting details in the image or enhancing blurred details, improving the visual effect of the image, and making the edges of objects in the image more defined.

[0143] In some embodiments, before determining the first resolution image and the second resolution image of the first image frame, the method further includes:

[0144] Determine the image acquisition spectral information of the first image frame;

[0145] The color space mapping relationship of the first image frame is determined based on the spectral information acquired from the image.

[0146] Among them, the image acquisition spectral information refers to the spectral information of the image acquisition device that acquires the first image frame during the acquisition process, such as the parameter information of the devices that affect the spectral imaging results, such as the filter, beam splitter, and photosensitive film installed on the terminal.

[0147] The color space mapping relationship refers to the color space transformation weights corresponding to the image after imaging under the fixed spectral information of the lens. The color space mapping relationship is used to determine the first resolution image of the first image frame and the second resolution image of the first image frame.

[0148] The reason for the above processing is that the first image frame is a color space image related to the device. The same object acquired under different device parameters will obtain image data with different colors. Therefore, in order to preserve the real image data, the spectral information of the image acquisition device of the first image frame can be calculated, so as to obtain the first resolution image and the second resolution image that meet the color fidelity according to the color space mapping relationship obtained after the calculation.

[0149] For example, when the first image frame is an image frame in the RGB color space, in order to extract the luminance channel image and the color channel image, the RGB color space image frame can first be converted into an LMS color space image frame according to the color space mapping relationship of the first image frame, and then the luminance channel image and the color channel image can be extracted based on the LMS color space image frame.

[0150] For example, the following is the conversion formula for an image frame in LMS color space obtained by converting an image frame in RGB color space using color space mapping:

[0151]

[0152] Here, matrix x is the color space mapping relationship obtained after mapping and converting the spectral information of the image frame in the RGB color space.

[0153] Where L, M, and S are values ​​in the LMS color space, and R, G, and B are values ​​in the RGB color space.

[0154] In some embodiments, when the color space of the image frame generated by fusing the first resolution image and the second resolution image is inconsistent with the color space of the first target image frame, further color space conversion processing is required to obtain the final desired first target image frame.

[0155] For example, an image frame with the first image frame as the RGB color space and an image frame generated by fusing the first resolution image and the second resolution image as the Lαβ color space are described as follows:

[0156] Step 1: Convert the merged Lαβ color space image frames into LMS color space image frames using the following spatial mapping relationship:

[0157]

[0158] Step 2: Use the following spatial mapping relationship to convert the merged LMS color space image frame into an RGB color space image frame:

[0159]

[0160] The image processing method provided in this embodiment improves the color fidelity of the image by using color space mapping to convert and adjust the first image frame. This enables the generated first target image to effectively retain color authenticity relative to the first image frame after subsequent mapping processing of the first resolution image of the first image frame.

[0161] Figure 3 This is a second schematic flowchart of an image processing method provided in one embodiment of this application, as shown below. Figure 3 As shown, the image processing methods include:

[0162] Step 301: Obtain the third resolution image of the third image frame and the fourth resolution image of the fourth image frame;

[0163] For example, the third and fourth image frames can be user input, generated by other hardware devices, or transmitted via software.

[0164] The user input referred to can be input by the user through touch input, voice input, gesture input, visual input, brain-computer input, etc.

[0165] The term "hardware device generation" can refer to information generated by a terminal under certain conditions, such as automatic image acquisition by the terminal's image acquisition device when a preset image acquisition time point is reached. For example, the third image frame can be an image frame with the original resolution output by the terminal's image acquisition device. The aforementioned terminal can be a smartphone, tablet, smart wearable device, camera, or camcorder, etc. Different terminals acquiring the same object will obtain image data with different or the same colors.

[0166] The so-called software transmission can be transmitted from local third-party software or cloud applications to image processing software on the terminal.

[0167] The third resolution image is a luminance channel image, the fourth resolution image is a color channel image, and the third resolution is higher than the fourth resolution.

[0168] The third and fourth image frames can be separately acquired image frames or image frames extracted from a video stream.

[0169] For example, the third and fourth image frames may originate from the same image acquisition device. For instance, after acquiring a video stream from an image acquisition device, several image frames are extracted from the video stream, and the two image frames with the highest image feature similarity are selected. One of these image frames is then used as the third image frame, and the other image frame is downsampled to a low-resolution image frame and used as the fourth image frame.

[0170] For example, the third and fourth image frames can also originate from two image acquisition modules with different resolutions. For instance, currently, terminals are typically equipped with three lenses of different pixel counts. One lens with a high pixel count and one lens with a low pixel count can be used to acquire images simultaneously, resulting in high-resolution and low-resolution image frames. After image processing techniques such as cropping, translation, and alignment, third and fourth image frames with different resolutions but the same image imaging angle and image imaging time can be obtained.

[0171] In one example, in order to eliminate some interference noise in the image frame, after obtaining the third and fourth image frames, one of the image frames can be used as a reference image frame to eliminate interference noise and supplement missing features in the other image frame. Then, the third resolution image can be extracted from the third image frame and the fourth resolution image can be extracted from the fourth image frame, thereby enhancing the detail information in the image frame.

[0172] Step 302: Generate a second target image based on the fourth resolution image and the third resolution image.

[0173] The second target image is a high-resolution color image with the same resolution as the third target image, and the second target image has both the luminance component L and the chrominance component αβ.

[0174] For example, the explanation will be based on the luminance channel image of an image frame with a third resolution and an Lαβ color space, and the color channel image of an image frame with a fourth resolution and an Lαβ color space.

[0175] In this embodiment, a low-resolution (i.e., fourth-resolution) color channel image and a high-resolution (i.e., third-resolution) luminance channel image are fused. During the fusion process, pixel-related data processing operations are performed on the chromaticity components αβ of the color channel image. Then, the processed color channel image and luminance channel image are fused to obtain the final desired second target image.

[0176] The reason for the above processing is that the detailed information of the image is concentrated in the third resolution image of the third image frame. Data processing operations are performed on the pixels contained in the fourth resolution image (i.e., the color channel image) of the fourth image frame. This does not affect the detailed information of the third image frame, and it takes less time than processing the color channel image of the third image frame. Therefore, the overall image processing efficiency is also faster.

[0177] In this embodiment of the application, compared with using only one image frame, using two image frames can compensate for the missing features in the acquisition process of each image frame, expand the image details, and reduce the impact of image noise.

[0178] In this embodiment of the application, in order to improve image processing efficiency, the color channel image of the third image frame is replaced by the fourth resolution image (i.e., color channel image) of the fourth image frame and the third resolution image (i.e., luminance channel image) of the third image frame are fused to obtain the desired second target image. Since the detailed information of the image is concentrated on the third resolution image, the desired second target image can also be obtained by using the fourth resolution image and the third resolution image for image fusion.

[0179] In some implementations, generating the second target image based on the fourth resolution image and the third resolution image includes:

[0180] Generate a target fourth-resolution image corresponding to the fourth-resolution image based on the third-resolution image;

[0181] A second target image is generated based on the target fourth-resolution image and the target third-resolution image.

[0182] The target fourth resolution image is a color channel image, and the target fourth resolution is equal to the third resolution.

[0183] Understandably, the resolution of the third-resolution image is higher than that of the fourth-resolution image, and the details of the image are mainly concentrated in the brightness channel. Therefore, while ensuring minimal loss of imaging quality to improve imaging speed, after obtaining the lower-resolution fourth-resolution image, the fourth-resolution image is restored to the target fourth-resolution image that is consistent with the third resolution. Then, the target fourth-resolution image and the third-resolution image are fused to generate the second target image.

[0184] In one example, a fourth-resolution image can be restored to a target fourth-resolution image that is consistent with the third-resolution image by using a mapping matrix between the third-resolution image and the fourth-resolution image.

[0185] Since the detailed features of an image are concentrated in the luminance channel, the luminance channel image details are preserved during processing. The luminance channel image is used as a reference image to process the color channel image. This simplifies the image processing steps while effectively preserving color fidelity, thereby reducing image processing time.

[0186] In some implementations, generating a target fourth resolution image corresponding to the fourth resolution image based on the third resolution image includes:

[0187] Based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, several pairs of second pixel points are determined, wherein each pair of second pixel points includes the third pixel point and the fourth pixel point with the smallest difference in brightness distribution.

[0188] The fourth pixel in each of the second pixel pairs is mapped onto the color channel matching the third pixel to obtain the target fourth resolution image.

[0189] The pixel brightness distribution can be the pixel value distribution of surrounding pixels centered on the pixel; it can also be the pixel value difference distribution between surrounding pixels and the central pixel; or it can be the neighborhood distribution and value range distribution of the pixel brightness.

[0190] It should be understood that the above pixel brightness distribution can refer to the brightness distribution of the third pixel or the brightness distribution of the fourth pixel, which will not be elaborated here.

[0191] In this embodiment, the process of generating the fourth resolution image of the target can be the same as the process of generating the second resolution image of the target described above, and will not be repeated here.

[0192] In this embodiment, the pixel brightness distribution can be used to accurately map the fourth resolution image onto the color channel that matches the third resolution image, thus ensuring both image detail and color fidelity.

[0193] In some embodiments, before determining a plurality of second pixel pairs based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, the method further includes: performing a preset image smoothing process on the fourth resolution image, wherein the preset image smoothing process includes at least one of mean filtering, median filtering, Gaussian filtering and edge filtering.

[0194] The purpose of the preset image smoothing process is to enable more accurate calculation of the brightness value of each pixel in the subsequent process.

[0195] In some implementations, before determining several pairs of second pixels based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, the method further includes: converting the fourth resolution image from the spatial domain to the spectral domain based on discrete cosine transform; performing Gaussian filtering on the fourth resolution image in the spectral domain, and performing inverse discrete cosine transform on the Gaussian-filtered fourth resolution image to obtain the fourth resolution image converted back to the spatial domain.

[0196] The discrete cosine transform, Gaussian filtering, and inverse discrete cosine transform processes for the fourth resolution image in this embodiment are the same as those for the second resolution image in the previous embodiments, and will not be repeated here.

[0197] The reason for the above processing is that after converting the fourth resolution image to the frequency domain using discrete cosine transform, it is subjected to Gaussian filtering with a relatively small smoothing effect, and then subjected to inverse discrete cosine transform back to the spatial domain. This makes the pixel transition of the processed fourth resolution image smoother, so that the brightness value of each pixel can be calculated more accurately in the future.

[0198] In some embodiments, before acquiring the third resolution image of the third image frame and the fourth resolution image of the fourth image frame, the method further includes: determining the image acquisition spectral information of the third image frame and the image acquisition spectral information of the fourth image frame; determining the color space mapping relationship of the third image frame based on the image acquisition spectral information of the third image frame; and determining the color space mapping relationship of the fourth image frame based on the image acquisition spectral information of the fourth image frame.

[0199] The color space mapping relationship of the third image frame is used to determine the third resolution image of the third image frame, and the color space mapping relationship of the fourth image frame is used to determine the fourth resolution image of the fourth image frame.

[0200] The process of determining the color space mapping relationship of the third image frame and the color space mapping relationship of the fourth image frame in this embodiment is the same as the process of determining the color space mapping relationship of the first image frame in the above embodiments, and will not be repeated here.

[0201] The image processing method provided in this embodiment improves the color fidelity of the image by converting and adjusting the third and fourth image frames using color space mapping relationships. This enables the second target image to effectively retain color authenticity relative to the fourth image frame when the fourth resolution image of the fourth image frame is subsequently mapped.

[0202] The image processing apparatus provided in one embodiment of this application is described below. The image processing apparatus described below can be referred to in correspondence with the image processing method described above.

[0203] Reference Figure 4 , Figure 4 This is one of the structural schematic diagrams of an image processing apparatus provided in an embodiment of this application, such as... Figure 4As shown, the image processing device includes: a first acquisition unit 410, configured to acquire a first image frame and determine a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is the luminance channel image of the first image frame, the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution; and a first generation unit 420, configured to generate a first target image based on the first resolution image and the second resolution image.

[0204] In some embodiments, the first generation unit 420 is further configured to generate a target second resolution image corresponding to the second resolution image based on the first resolution image, wherein the target second resolution image is a color channel image and the target second resolution is equal to the first resolution; and generate a first target image based on the target second resolution image and the first resolution image.

[0205] In some embodiments, the first generation unit 420 is further configured to determine the first pixel brightness distribution of the first resolution image and the second pixel brightness distribution of the second resolution image; and to map the second resolution image onto a color channel matching the first resolution image based on the first pixel brightness distribution and the second pixel brightness distribution to obtain a target second resolution image.

[0206] In some embodiments, the brightness distribution of the first pixel is the neighborhood distribution and value range distribution of the brightness of each first pixel, and the brightness distribution of the second pixel is the neighborhood distribution and value range distribution of the brightness of each second pixel.

[0207] In some embodiments, the first generation unit 420 is further configured to determine a plurality of first pixel pairs based on the brightness distribution of the first pixel and the brightness distribution of the second pixel, wherein each first pixel pair includes a first pixel and a second pixel with the smallest difference in brightness distribution; and to map the second pixel in each first pixel pair onto the color channel matching the first pixel to obtain a target second resolution image.

[0208] In some embodiments, the first generation unit 420 is further configured to perform a preset image smoothing process on the second resolution image, wherein the preset image smoothing process includes at least one of mean filtering, median filtering, Gaussian filtering and edge filtering.

[0209] In some embodiments, the first generation unit 420 is further configured to convert the second resolution image from the spatial domain to the spectral domain based on discrete cosine transform; perform Gaussian filtering on the second resolution image in the spectral domain, and perform inverse discrete cosine transform on the Gaussian-filtered second resolution image to obtain the second resolution image converted back to the spatial domain.

[0210] In some embodiments, the first acquisition unit 410 is further configured to downsample the first image frame to obtain a second image frame; and determine a first resolution image of the first image frame and a second resolution image of the second image frame.

[0211] In some embodiments, the first acquisition unit 410 is further configured to perform image noise reduction and sharpening processing on the first resolution image.

[0212] In some embodiments, the first acquisition unit 410 is further configured to determine the image acquisition spectral information of the first image frame; and to determine the color space mapping relationship of the first image frame based on the image acquisition spectral information, wherein the color space mapping relationship is used to determine the first resolution image of the first image frame and the second resolution image of the first image frame.

[0213] It should be noted that the image processing apparatus provided in this application embodiment can implement all the method steps implemented in the above image processing method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0214] Reference Figure 5 , Figure 5 This is a second schematic diagram of the structure of an image processing apparatus provided in one embodiment of this application, as shown below. Figure 5 As shown, the image processing device includes: a second acquisition unit 510, configured to acquire a third resolution image of a third image frame and a fourth resolution image of a fourth image frame, wherein the third resolution image is a luminance channel image and the fourth resolution image is a color channel image, and the third resolution is higher than the fourth resolution; and a second generation unit 520, configured to generate a second target image based on the target fourth resolution image and the third resolution image.

[0215] In some embodiments, the second generation unit 520 is further configured to generate a target fourth resolution image corresponding to the fourth resolution image based on the third resolution image, wherein the target fourth resolution image is a color channel image and the target fourth resolution is equal to the third resolution; and to generate a second target image based on the target fourth resolution image and the third resolution image.

[0216] In some embodiments, the second generation unit 520 is further configured to determine a plurality of second pixel pairs based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, wherein each second pixel pair includes a third pixel and a fourth pixel with the smallest difference in brightness distribution; and to map the fourth pixel in each second pixel pair onto the color channel that matches the third pixel to obtain a target fourth resolution image.

[0217] In some embodiments, the second generation unit 520 is further configured to perform a preset image smoothing process on the fourth resolution image, wherein the preset image smoothing process includes at least one of mean filtering, median filtering, Gaussian filtering and edge filtering.

[0218] In some embodiments, the second generation unit 520 is further configured to convert the fourth resolution image from the spatial domain to the spectral domain based on discrete cosine transform; perform Gaussian filtering on the fourth resolution image in the spectral domain, and perform inverse discrete cosine transform on the Gaussian-filtered fourth resolution image to obtain the fourth resolution image converted back to the spatial domain.

[0219] In some embodiments, the second generation unit 520 is further configured to determine the image acquisition spectral information of the third image frame and the image acquisition spectral information of the fourth image frame; determine the color space mapping relationship of the third image frame based on the image acquisition spectral information of the third image frame; and determine the color space mapping relationship of the fourth image frame based on the image acquisition spectral information of the fourth image frame; wherein the color space mapping relationship of the third image frame is used to determine the third resolution image of the third image frame, and the color space mapping relationship of the fourth image frame is used to determine the fourth resolution image of the fourth image frame.

[0220] It should be noted that the image processing apparatus provided in this application embodiment can implement all the method steps implemented in the above image processing method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0221] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an image processing method, the method including: acquiring a first image frame; determining a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is the luminance channel image of the first image frame, and the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution; generating a first target image based on the first resolution image and the second resolution image.

[0222] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0223] On the other hand, one embodiment of this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing method provided by the above methods. The method includes: acquiring a first image frame; determining a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is a luminance channel image of the first image frame, the second resolution image is a color channel image of the first image frame, and the first resolution is higher than the second resolution; and generating a first target image based on the first resolution image and the second resolution image.

[0224] In another aspect, one embodiment of this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the image processing methods provided by the above methods. The method includes: acquiring a first image frame; determining a first resolution image and a second resolution image of the first image frame, wherein the first resolution image is a luminance channel image of the first image frame, the second resolution image is a color channel image of the first image frame, and the first resolution is higher than the second resolution; and generating a first target image based on the first resolution image and the second resolution image.

[0225] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, wherein, include: A first image frame is acquired, and a first resolution image and a second resolution image of the first image frame are determined, wherein the first resolution image is the luminance channel image of the first image frame, and the second resolution image is the color channel image of the first image frame, and the first resolution is higher than the second resolution. A first target image is generated based on the first resolution image and the second resolution image; the first target image is a high-resolution color image with the same resolution as the first resolution, and the first target image has both luminance and chrominance components; The step of generating a first target image based on the first resolution image and the second resolution image includes: Determine the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image; Based on the brightness distribution of the first pixel and the brightness distribution of the second pixel, several pairs of first pixel points are determined, wherein each pair of first pixel points includes a first pixel point and a second pixel point with the smallest difference in brightness distribution. Map the second pixel in each of the first pixel pairs to the color channel that matches the first pixel to obtain the target second resolution image; A first target image is generated based on the target second resolution image and the first resolution image.

2. The image processing method of claim 1, wherein, The brightness distribution of the first pixel is the neighborhood distribution and value range distribution of the brightness of each first pixel, and the brightness distribution of the second pixel is the neighborhood distribution and value range distribution of the brightness of each second pixel.

3. The image processing method of claim 1, wherein, Before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes: The second resolution image is subjected to a preset image smoothing process, wherein the preset image smoothing process includes at least one of the following processing methods: mean filtering, median filtering, Gaussian filtering, and edge filtering.

4. The image processing method of claim 1, wherein, Before determining the brightness distribution of the first pixel in the first resolution image and the brightness distribution of the second pixel in the second resolution image, the method further includes: The second-resolution image is transformed from the spatial domain to the spectral domain based on discrete cosine transform. Gaussian filtering is applied to the second-resolution image in the spectral domain, and then inverse discrete cosine transform is performed on the Gaussian-filtered second-resolution image to obtain the second-resolution image converted back to the spatial domain.

5. The image processing method according to any one of claims 1 to 4, wherein, Determining the first resolution image and the second resolution image of the first image frame includes: The first image frame is downsampled to obtain the second image frame; Determine the first resolution image of the first image frame and the second resolution image of the second image frame.

6. The image processing method according to any one of claims 1 to 4, wherein Before generating the first target image based on the first resolution image and the second resolution image, the method further includes: The image at the first resolution is subjected to image noise reduction and sharpening processing.

7. The image processing method according to any one of claims 1 to 4, wherein Before determining the first resolution image and the second resolution image of the first image frame, the method further includes: Determine the image acquisition spectral information of the first image frame; The color space mapping relationship of the first image frame is determined based on the image acquisition spectral information, wherein the color space mapping relationship is used to determine the first resolution image and the second resolution image of the first image frame.

8. An image processing method, wherein, include: Acquire a third-resolution image of a third image frame and a fourth-resolution image of a fourth image frame, wherein the third-resolution image is a luminance channel image and the fourth-resolution image is a color channel image, and the third resolution is higher than the fourth resolution; A second target image is generated based on the fourth resolution image and the third resolution image; the second target image is a high-resolution color image with the same resolution as the third resolution, and the second target image has both luminance and chrominance components; The generation of the second target image based on the fourth resolution image and the third resolution image includes: Based on the brightness distribution of the third pixel in the third resolution image and the brightness distribution of the fourth pixel in the fourth resolution image, several pairs of second pixel points are determined, wherein each pair of second pixel points includes the third pixel point and the fourth pixel point with the smallest difference in brightness distribution. Map the fourth pixel in each of the second pixel pairs to the color channel that matches the third pixel to obtain the target fourth resolution image; A second target image is generated based on the target fourth-resolution image and the target third-resolution image.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the program, it implements the image processing method as described in any one of claims 1 to 7 or the image processing method as described in any one of claims 8.

10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1 to 7 or the image processing method as described in any one of claims 8.

11. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1 to 7 or the image processing method as described in any one of claims 8.

Images