Image reconstruction method, device, module, equipment, medium and program product

Abstract

The application provides an image reconstruction method, device, module, equipment, medium and program product. The image reconstruction method comprises the following steps: traversing a native domain image based on a preset window to obtain each image block; for each image block, detecting a white channel overexposed pixel point in the image block; correcting the white channel overexposed pixel point of the native domain image according to the pixel value of the pixel point of other channels in the same image block as the white channel overexposed pixel point; performing image reconstruction according to the pixel value of the corrected white channel pixel point; wherein the white channel overexposed pixel point is a white channel pixel point with a pixel value greater than a preset threshold. By correcting the white channel overexposed pixel based on the pixels of other channels, the problem of texture detail loss caused by white channel overexposed pixels during image reconstruction is avoided, the image reconstruction quality is improved, and the sampling image reconstruction method is used to correct the overexposure problem, which is low in cost.

Description

Technical Field

[0001] This application relates to the field of image reconstruction technology, and in particular to an image reconstruction method, apparatus, module, device, medium and program product. Background Technology

[0002] RGBW (Red Green Blue White) technology adds a white channel pixel to the original RGB three-primary-color technology, forming a four-color pixel design. Compared with RGB technology, it has a larger light intake and is more suitable for image shooting in low-light scenes.

[0003] Because the white channel receives more light than other channels, it will be overexposed under the same exposure time, resulting in the loss of texture details during image reconstruction. To solve the problem of white channel overexposure, the exposure time is usually shortened, or different exposure times are set for different channels.

[0004] Shortening the exposure time will result in an overall darker image, thus reducing the signal-to-noise ratio; using different exposure times will increase the cost of image capture.

[0005] Therefore, there is an urgent need for an image reconstruction method to correct the white channel overexposure problem. Summary of the Invention

[0006] This application provides an image reconstruction method, apparatus, module, device, medium, and program product, which corrects overexposed pixels in the white channel, thereby avoiding the loss of texture details around the overexposed points during image reconstruction due to overexposure of the white channel, and improving the image reconstruction quality.

[0007] In a first aspect, this application provides an image reconstruction method, comprising:

[0008] Detect overexposed white channel pixels in the native domain image; correct the overexposed white channel pixels in the native domain image based on the pixel values ​​of pixels in other channels that are in the same image block as the overexposed white channel pixels; reconstruct the image based on the corrected pixel values ​​of the white channel pixels; wherein, the overexposed white channel pixels are white channel pixels with pixel values ​​greater than a preset threshold.

[0009] In one possible implementation, the overexposed white channel pixels of the native domain image are corrected based on the pixel values ​​of pixels in other channels within the same image block as the overexposed white channel pixels, including:

[0010] Based on the first preset interpolation algorithm, the first interpolation value of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixel in the white channel is determined; and the overexposed pixel in the white channel is corrected according to the first interpolation value of the green channel.

[0011] In one possible implementation, the native domain image is the merged readout image, including white channel information and a sampled image; the overexposed pixels in the white channel are the overexposed pixels in the white channel information.

[0012] Accordingly, based on the pixel values ​​of pixels in other channels within the same image block as the overexposed white channel pixel, the overexposed white channel pixel of the native domain image is corrected, including:

[0013] For each overexposed pixel in the white channel information, the overexposed pixel in the white channel is corrected according to the pixel value of each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel.

[0014] In one possible implementation, the overexposed pixels in the white channel are corrected based on the pixel values ​​of each pixel in the image block of the sampled image corresponding to the overexposed white channel pixels, including:

[0015] The gradient direction is determined based on the pixel values ​​of each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; the filter kernel is determined based on the type of the image block in the sampled image corresponding to the overexposed pixel in the white channel and the gradient direction; the correction value of the overexposed pixel in the white channel is determined based on the product of the filter kernel and each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; if the correction value of the overexposed pixel in the white channel is greater than the pixel value of the overexposed pixel in the white channel, the pixel value of the overexposed pixel in the white channel is corrected to the correction value.

[0016] In one possible implementation, the method further includes:

[0017] Determine the scene type of the native domain image;

[0018] If the scene type of the native domain image is a dark scene, then the native domain image is merged and read out to obtain white channel information and a sampled image.

[0019] In one possible implementation, the method further includes:

[0020] If the scene of the native domain image is a bright scene, then based on the first preset interpolation algorithm, determine the first interpolation value of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixel in the white channel; and correct the overexposed pixel in the white channel according to the first interpolation value of the green channel.

[0021] In one possible implementation, the overexposed pixels in the white channel are corrected based on the first interpolation of the green channel, including:

[0022] The pixel values ​​of the corresponding red or blue channel pixels are replaced by the first interpolation of each green channel to obtain a first interpolated image block; based on a second preset interpolation algorithm, the second interpolation of the green channel corresponding to the overexposed white channel pixels in the first interpolated image block is determined; based on the difference between the first interpolation of the green channel and the pixel values ​​of the corresponding red and blue channel pixels, the mean red-green difference of the red channel and the mean blue-green difference of the blue channel are calculated; based on the second interpolation of the green channel, the mean red-green difference, and the mean blue-green difference, the overexposed white channel pixels are corrected.

[0023] In one possible implementation, the corrected pixel value W of the overexposed white channel pixel is:

[0024]

[0025] Where W0 is the pixel value of the overexposed pixel in the white channel; G W0 This is the second interpolation of the green channel corresponding to the overexposed pixels in the white channel; The average blue-green difference corresponding to overexposed pixels in the white channel; cd R This represents the average red-green difference corresponding to the overexposed pixels in the white channel; the max() function is used to retrieve the maximum value of the parameter.

[0026] In one possible implementation, detecting overexposed white channel pixels in the native domain image includes:

[0027] The native domain image is traversed based on a preset window to obtain each image block; for each image block, overexposed pixels in the white channel are detected.

[0028] In one possible implementation, image reconstruction is performed based on the pixel values ​​of the corrected white channel pixels, including:

[0029] Based on the pixel values ​​of each pixel in the corrected pixel block, interpolation is performed on the white, red, green, and blue channels; based on the interpolation results, image reconstruction is performed.

[0030] In one possible implementation, the method further includes:

[0031] The native domain image is acquired based on an RGBW (Red Green Blue White) sensor.

[0032] Secondly, this application provides an image reconstruction apparatus, comprising:

[0033] An overexposure point detection module is used to detect overexposed pixels in the white channel of the native domain image; an overexposure point correction module is used to correct the overexposed pixels in the white channel of the native domain image based on the pixel values ​​of pixels in other channels that are in the same image block as the overexposed pixels in the white channel; and an image reconstruction module is used to reconstruct the image based on the corrected pixel values ​​of the white channel pixels; wherein, the overexposed pixels in the white channel are white channel pixels whose pixel values ​​are greater than a preset threshold.

[0034] In one possible implementation, the overexposure point correction module is specifically used for:

[0035] Based on the first preset interpolation algorithm, the first interpolation value of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixel in the white channel is determined; and the overexposed pixel in the white channel is corrected according to the first interpolation value of the green channel.

[0036] In one possible implementation, the device further includes:

[0037] The merge readout module is used to merge and read out the native domain image to obtain white channel information and sampled images.

[0038] Correspondingly, the overexposure point detection module is used for:

[0039] Detect overexposed pixels in the white channel information.

[0040] Correspondingly, the overexposure point correction module includes:

[0041] The overexposure correction unit is used to correct the overexposure pixels in the white channel based on the pixel values ​​of each pixel in the image block of the sampling image corresponding to the overexposure pixel in the white channel information.

[0042] In one possible implementation, the overexposure point correction unit is specifically used for:

[0043] The gradient direction is determined based on the pixel values ​​of each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; the filter kernel is determined based on the type of the image block in the sampled image corresponding to the overexposed pixel in the white channel and the gradient direction; the correction value of the overexposed pixel in the white channel is determined based on the product of the filter kernel and each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; if the correction value of the overexposed pixel in the white channel is greater than the pixel value of the overexposed pixel in the white channel, the pixel value of the overexposed pixel in the white channel is corrected to the correction value.

[0044] In one possible implementation, the device further includes:

[0045] The scene type determination module is used to determine the scene type of the native domain image.

[0046] Correspondingly, the merge read module is specifically used for:

[0047] If the scene type of the native domain image is a dark scene, then the native domain image is merged and read out to obtain white channel information and a sampled image.

[0048] In one possible implementation, the device further includes:

[0049] The first interpolation module is used to determine the first interpolation value of the green channel corresponding to each pixel in the red and blue channels of the same image block as the overexposed white channel pixel, based on a first preset interpolation algorithm, if the scene of the native domain image is a bright scene; the overexposure correction module is used to correct the overexposed white channel pixel according to the first interpolation value of the green channel.

[0050] In one possible implementation, the overexposure correction module includes:

[0051] An interpolated image block acquisition unit is used to replace the pixel values ​​of the corresponding red or blue channel pixels with the first interpolation of each green channel to obtain a first interpolated image block; a second interpolation unit is used to determine the second interpolation of the green channel corresponding to the overexposed white channel pixels in the first interpolated image block based on a second preset interpolation algorithm; an overexposure correction unit is used to calculate the average red-green difference of the red channel and the average blue-green difference of the blue channel based on the difference between the first interpolation of the green channel and the pixel values ​​of the corresponding red and blue channel pixels; and to correct the overexposed white channel pixels based on the second interpolation of the green channel, the average red-green difference, and the average blue-green difference.

[0052] In one possible implementation, the overexposure point detection module is specifically used for:

[0053] The native domain image is traversed based on a preset window to obtain each image block; for each image block, overexposed pixels in the white channel are detected.

[0054] In one possible implementation, the image reconstruction module is specifically used for:

[0055] Based on the pixel values ​​of each pixel in the corrected pixel block, interpolation is performed on the white, red, green, and blue channels; based on the interpolation results, image reconstruction is performed.

[0056] In one possible implementation, the device further includes:

[0057] The image acquisition module is used to acquire the native domain image based on the RGBW sensor.

[0058] Thirdly, this application provides an image reconstruction assembly, comprising: a memory and at least one processor; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the image reconstruction method as provided in the first aspect of this application.

[0059] Fourthly, this application provides a terminal device, including an image sensor and an image remodeling assembly provided in the third aspect of this application; wherein the image sensor is used to acquire native domain images.

[0060] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image reconstruction method provided in the first aspect of this application.

[0061] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the image reconstruction method provided in the first aspect of this application.

[0062] The image reconstruction method, apparatus, module, device, medium, and program products provided in this application, for RGBW native domain images, when an overexposed white channel pixel is detected in the image, the overexposed white channel pixel is corrected based on the pixel values ​​of pixels in other channels surrounding the overexposed white channel pixel. Image reconstruction is then performed based on the corrected white channel pixel values ​​and the pixel values ​​of pixels in other channels to obtain a color image. By combining the pixel value distribution of the RGB channels, the overexposed W channel pixels are corrected, restoring the detail information lost due to overexposure, thereby improving the quality of image reconstruction. Moreover, overexposure correction is performed through image processing, which is less costly than correction using different overexposure times. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A schematic diagram of an image reconstruction process provided in one embodiment of this application;

[0065] Figure 2 A schematic flowchart of an embodiment of the image reconstruction method provided in this application;

[0066] Figure 3 For this application Figure 2 A schematic diagram of the native domain image in the illustrated embodiment;

[0067] Figure 4 A schematic diagram illustrating the merged readout results provided in one embodiment of this application;

[0068] Figure 5 A schematic flowchart of Embodiment 2 of the image reconstruction method provided in this application;

[0069] Figure 6A For this application Figure 5 A schematic diagram of an image block with center point R in the illustrated embodiment;

[0070] Figure 6B For this application Figure 5 A schematic diagram of an image block with center point G in the illustrated embodiment;

[0071] Figure 7 A schematic flowchart of Embodiment 3 of the image reconstruction method provided in this application;

[0072] Figure 8 A flowchart illustrating Embodiment 4 of the image reconstruction method provided in this application;

[0073] Figure 9 For this application Figure 7 A schematic diagram of the green channel interpolation process in the illustrated embodiment;

[0074] Figure 10 This is a schematic diagram of the structure of the image reconstruction apparatus provided in the embodiments of this application;

[0075] Figure 11 This is a schematic diagram of the structure of the image remodeling group provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] First, let me explain the terms used in this application:

[0078] CFA (Color Filter Array): A color filter array used before the CCD (Charge Coupled Device) sensor in a digital camera to remove some components of the spectrum, so that each pixel retains only the color components corresponding to specific wavelengths of light, such as red, blue, and green.

[0079] Byer CFA: The Byer color filter array arrangement is a commonly used CFA arrangement. In each 2×2 array, two green light filters are placed diagonally, and the remaining two positions are occupied by a red light filter and a blue light filter, respectively. That is, it consists of 1 / 2 green light filter, 1 / 4 red light filter and 1 / 4 blue light filter.

[0080] Demosaicking: A technique used to convert native domain images in Byer format into RGB domain images using an interpolation algorithm.

[0081] In some scenarios, imaging systems need to acquire images in darker environments. To improve the signal-to-noise ratio of acquired images in darker scenes, RGBW CFA technology, which allows for more light intake, has been introduced. Figure 1 This is a schematic diagram of an image reconstruction process provided in one embodiment of this application, as shown below. Figure 1 As shown, the native domain image 101 acquired by the photosensitive element of the RGBW color filter array (CFA) of the imaging system consists of pixels from four channels: R, G, B, and W. Figure 1 Taking RGBW CFA as an example of Kodak layout, the smallest unit collected is as follows: Figure 1The original domain image 101 is shown in the figure. After obtaining the original domain image 101, it needs to be reconstructed into a color image 102 based on the demosaicing technique. In this process, the missing channel information of each pixel needs to be supplemented by the interpolation algorithm. That is, firstly, the information of each channel (including R channel, G channel, B channel and W channel) in the original domain image 101 is extracted, and then the complete W channel information is obtained by interpolation of the W channel. Based on the complete W channel information, the RGB three-channel interpolation is guided to obtain the complete R channel information, G channel information and B channel information. The complete RGB three-channel information is then merged to obtain the color image 102. In this application, "G" indicates that the color of the pixel at the corresponding position is green, "R" indicates that the color of the pixel at the corresponding position is red, "B" indicates that the color of the pixel at the corresponding position is blue, and "W" indicates that the color of the pixel at the corresponding position is gray.

[0082] When acquiring native domain image 101, the W channel receives more light than the RGB three channels. Under the same exposure time, the W channel is prone to overexposure, which leads to the loss of texture details around overexposed pixels during interpolation and image reconstruction, thus reducing the quality of image reconstruction.

[0083] To correct overexposed pixels in the W channel, the exposure time is typically shortened, or different exposure times are used for native domain image acquisition. Shortening the exposure time results in an overall darker image, reducing the signal-to-noise ratio; using different exposure times increases the cost of image acquisition.

[0084] To overcome the problems caused by overexposed pixel correction based on exposure time, this application provides an overexposed pixel correction method based on image processing. Under the premise of maintaining a consistent exposure time, the overexposed pixel value is corrected based on the distribution of pixel values ​​around the overexposed point, which better restores the details of the W channel. This provides more accurate channel information for the interpolation process during subsequent demosaicing, thereby improving the quality of image reconstruction.

[0085] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0086] Figure 2 This is a flowchart illustrating an embodiment of the image reconstruction method provided in this application. This image reconstruction method can be executed by an image reconstruction assembly, such as an image reconstruction chip or processor, or by a terminal device, such as the aforementioned imaging system. Figure 2 As shown, the image reconstruction method specifically includes the following steps:

[0087] S201, detects overexposed pixels in the white channel of the native domain image.

[0088] The native domain image is in Bayer format, and includes information from four channels: white (or W channel), green (or G channel), red (or R channel), and blue (or B channel). Overexposed pixels in the white channel are those whose pixel values ​​exceed a preset threshold.

[0089] The preset threshold can be a fixed value, such as 200, 225, 250, etc., or it can be the product of the maximum value and a preset coefficient less than 1, such as 0.9, 0.95, 0.99, or other values. This maximum value can be global or local; for example, it can be the maximum pixel value of the white channel pixels, or it can be the maximum pixel value of the white channel pixels in the native domain image.

[0090] For example, the native domain image can be any RGBW layout, such as Kodak, HexW, PenTile, etc.

[0091] Specifically, native domain images can be acquired based on RGBW sensors, and the acquired native domain images can be sent to the image remodeling group for native domain image overexposure processing and color image reconstruction.

[0092] For example, Figure 3 For this application Figure 2 The illustrated embodiment is a schematic diagram of a native domain image, which includes information from four channels, arranged as follows: Figure 3 As shown, "W" indicates that the pixel at the corresponding location is gray (the grayscale value can be between 0 and 255). Compared to traditional native domain images captured by RGB sensors, native domain images captured by RGBW sensors have additional white channel information, i.e. Figure 3 The information corresponding to W in the text. Figure 3 Taking the native domain image acquired by the RGBW sensor in HexW CFA format as an example, its smallest unit is 8 rows and 8 columns. The arrangement of the first to eighth rows of the smallest unit is as follows: BWBWGWGW, WBWBWGWG, BWBWGWGW, WBWBWGWG, GUGWRWRW, WGWGWRWR, GUGWRWRW and WGWGWRWR.

[0093] Specifically, it can iterate through each pixel in the white channel of the native domain image, and for each pixel, determine whether the pixel value of the pixel is greater than a preset threshold. If so, then the pixel is determined to be an overexposed pixel in the white channel.

[0094] Specifically, the native domain image can be traversed based on a preset window to obtain each image block; for each white channel pixel in the image block, it is determined whether the pixel value of the white channel pixel is greater than a preset threshold. If so, the pixel is determined to be an overexposed white channel pixel.

[0095] In one embodiment, it may be necessary to determine whether each white channel pixel used in white channel interpolation is an overexposed white channel pixel.

[0096] Since the white channel pixels receive more light than other channels in the native domain image, to facilitate subsequent white channel overexposure correction and interpolation calculations, white balance processing can be performed on the white channel pixels of the native domain image after acquisition. This involves multiplying the pixel value of the white channel pixels by a white balance coefficient that is less than 1 and greater than 0. The white balance coefficient can be 0.5, 0.6, 0.7, or other coefficients, determined based on the white balance algorithm used. This allows for the detection and correction of overexposed white channel pixels based on the white balance processed white channel pixels.

[0097] S202, the overexposed pixel in the white channel is corrected based on the pixel values ​​of pixels in other channels that are in the same image block as the overexposed pixel in the white channel.

[0098] Since the white channel suffers from overexposure, its gradient is unreliable. Therefore, overexposed pixels in the white channel can be corrected based on the pixels in the other channels, namely the RGB channels. The other channels are the R, G, and B channels, and the image block can contain M rows and N columns of pixels. After detecting an overexposed pixel in the white channel, an M*N image block is obtained centered on that pixel. The pixel values ​​of the other three channels within this image block are extracted. Based on the distribution of the pixel values ​​of the other three channels, such as color difference and mean, the overexposed pixels in the white channel are corrected.

[0099] Specifically, the correction value for overexposed pixels in the white channel can be calculated based on the pixel values ​​of the other three channels. Since the values ​​of overexposed pixels in the white channel are all reduced to a peak value that is less than the actual value, in order to restore the original pixel value of the overexposed pixels in the white channel, the correction value can be greater than the corresponding overexposed pixel in the white channel. The correction value can then be used to replace the corresponding overexposed pixel in the white channel, thereby restoring the texture details at the overexposed pixel in the white channel.

[0100] If the correction value is less than or equal to the corresponding overexposed white channel pixel, then the corresponding overexposed white channel pixel will not be corrected.

[0101] S203, Reconstruct the image based on the pixel values ​​of the corrected white channel pixels.

[0102] In one embodiment, steps related to white channel overexposure pixel correction can be performed before white channel interpolation. For example, by embedding a white channel overexposure correction module (used to perform steps related to white channel overexposure pixel correction, such as steps S201 and S202 above) into the white channel interpolation module, the white channel interpolation algorithm can be executed to correct white channel overexposure pixels.

[0103] Specifically, based on the pixel values ​​of the corrected white channel pixels, white channel interpolation can be performed to obtain complete white channel information. Then, based on the completed white channel information and the pixel values ​​of the RGB three-channel pixels, RGB three-channel interpolation can be guided to obtain complete RGB three-channel information. Finally, based on the complete RGB three-channel information, image reconstruction can be performed to obtain a full-color image for display.

[0104] Specifically, the white channel interpolation module first traverses the native domain image based on a preset window to obtain each pixel block, where the preset window is the window corresponding to the white channel interpolation. The white channel overexposure correction module detects the overexposed white channel pixels in each preset window and corrects the overexposed white channel pixels based on the pixel values ​​of pixels in other channels within the same image block as the overexposed white channel pixels, obtaining corrected pixel blocks. The white channel interpolation module performs white channel interpolation based on the corrected pixel blocks and the corresponding interpolation algorithm to obtain complete white channel information. Then, based on the complete white channel information, it guides the interpolation of the other three channels to obtain complete RGB three-channel information. Finally, based on the complete RGB three-channel information, it performs image reconstruction to obtain a full-color image for display.

[0105] The preset window size can be 9*9, 7*7, or other sizes.

[0106] The image reconstruction method provided in this embodiment targets RGBW native domain images. When overexposed white channel pixels are detected in the image, the overexposed white channel pixels are corrected based on the pixel values ​​of pixels in other channels surrounding the overexposed white channel pixels. Image reconstruction is then performed based on the corrected white channel pixel values ​​and the pixel values ​​of pixels in other channels to obtain a color image. By combining the pixel value distribution of the RGB channels, the overexposed W channel pixels are corrected, restoring the detail information lost due to overexposure, thereby improving the quality of image reconstruction. Furthermore, overexposure correction is performed through image processing, which is less costly than using different overexposure times for correction.

[0107] Optionally, the overexposed pixels in the white channel of the native domain image are corrected based on the pixel values ​​of pixels in other channels within the same image block as the overexposed white channel pixel, including:

[0108] Based on the first preset interpolation algorithm, the first interpolation value of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixel in the white channel is determined; and the overexposed pixel in the white channel is corrected according to the first interpolation value of the green channel.

[0109] Among them, the first interpolation value of the green channel is an interpolation value corresponding to the green channel, that is, the interpolation value of the green channel at the position corresponding to the red channel or blue channel.

[0110] The first preset interpolation algorithm can be any interpolation algorithm, such as interpolation algorithms based on color difference method, color ratio method, direction weighting, machine learning or deep learning methods.

[0111] When interpolating the sampling points based on the first preset interpolation algorithm, the corresponding window can be a p*q window centered on the sampling point. The values ​​of p and q can be equal or unequal. Both p and q are odd numbers, such as 7*7, 3*5, etc.

[0112] For each image block, based on a first preset interpolation algorithm, the pixels of the red and blue channels in that image block are used as sampling points, and each sampling point is interpolated to the corresponding green channel pixels, i.e., the first green channel interpolation. This results in an image block composed of 50% white channel and 50% green channel. Then, based on the first green channel interpolation in the image block composed of 50% white channel and 50% green channel, overexposed pixels in the white channel are corrected.

[0113] Specifically, the overexposed pixels in the white channel can be corrected based on the average difference between the first interpolation value of the green channel and the corresponding sampling point pixel value.

[0114] By fully considering the distribution of pixels around the sampling point through the interpolation process, the overexposed white channel pixels are corrected based on the difference results, thereby improving the accuracy of the overexposed white channel pixel correction.

[0115] In some embodiments, the native domain image is the merged readout image, including white channel information and a sampled image. Overexposed pixels in the white channel are the overexposed pixels in the white channel information.

[0116] Optionally, the overexposed pixels in the white channel of the native domain image are corrected based on the pixel values ​​of pixels in other channels within the same image block as the overexposed white channel pixel, including:

[0117] For each overexposed pixel in the white channel information, the overexposed pixel in the white channel is corrected according to the pixel value of each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel.

[0118] The sampled image is an image that includes RGB three-channel information.

[0119] Specifically, pixels in the same channel within a pixel of a native domain image can be merged into a single pixel in that channel, thereby achieving downsampling of the native domain image to obtain white channel information and a sampled image.

[0120] A single pixel may include 2 rows and 2 columns of pixels from the native domain image, or it may include 4 rows and 4 columns of pixels from the native domain image, or it may be a pixel of other sizes.

[0121] The native domain image can be merged and read out based on the Binning mode, such as the 2x Binning or 4x Binning mode, to obtain white channel information and sampled images.

[0122] For example, Figure 4 This is a schematic diagram of the merged readout results provided in one embodiment of this application, as shown below. Figure 4 As shown, for an m*n row native domain image, Figure 4 The native domain image is an RGBW image with a Kodak layout. Figure 4 The original domain image is represented by a minimum unit (4*4) of the original domain image. Through 2x Binning, we obtain... White channel information and The sampled image of the Bayer arrangement can be arranged in a Quad Bayer manner. It can also be obtained through quadruple Binning. White channel information and The sampled image is arranged in a Bayer pattern. The pixel size is 2*2 when Binning is 2x and 4*4 when Binning is 4x. The pixel values ​​of the white channel pixels in the pixel are averaged to obtain one element of the corresponding white channel information. The pixel values ​​of the pixels belonging to the same channel in the RGB channel in the pixel are averaged to obtain one pixel of that channel in the sampled image. Figure 4 Taking 2x Binning as an example.

[0123] After obtaining the white channel information and the sampled image through the aforementioned downsampling method, for each pixel in the white channel information, it is determined whether the pixel value of that pixel is greater than a preset threshold. If so, the pixel is determined to be an overexposed pixel in the white channel. For each overexposed pixel in the white channel, a pixel block of a preset size is obtained, centered on the pixel in any channel of the RGB channel corresponding to the overexposed pixel in the sampled image, such as a 3*5 pixel block, a 5*7 pixel block, etc. Based on the pixel values ​​of each pixel in this pixel block, the pixel value of the overexposed pixel in the white channel is corrected.

[0124] Specifically, the correction value for overexposed pixels in the white channel can be represented by the weighted average of the gradients of each pixel in the corresponding pixel block.

[0125] In one embodiment, the gradient direction of the pixel block can be determined first, and the pixels within the pixel block can be weighted and averaged based on the filter kernel corresponding to the gradient direction of the pixel block to obtain the correction value of the overexposed pixels in the white channel.

[0126] The correspondence between various gradient directions and filter kernels can be established in advance, and then the filter kernel corresponding to the gradient direction of the pixel block can be determined based on the correspondence.

[0127] In one embodiment, the gradient direction of a pixel block can also be determined based on the color difference variance of the pixels in the pixel block.

[0128] The above-mentioned method for correcting overexposed pixels in the white channel is compatible with the native domain image merging and readout mode, thus expanding the application scope of overexposed point correction.

[0129] To enable readers to more deeply understand the implementation principles of the embodiments of this application, the following will be used as a reference. Figures 5 to 9 The above embodiments will be further detailed or explained.

[0130] Figure 5 This is a flowchart illustrating a second embodiment of the image reconstruction method provided in this application. This embodiment targets the original domain image as a merged readout image, including white channel information and a sampled image. The detected overexposed pixels in the white channel are those pixels in the white channel information that are overexposed. Figure 5 As shown, the above S202 can be achieved through the following steps:

[0131] S501, for each overexposed pixel in the white channel information, determine the gradient direction based on the pixel value of each pixel in the image block of the sampling image corresponding to the overexposed pixel in the white channel.

[0132] The gradient direction can include the horizontal direction, the vertical direction, the 45° direction (diagonal direction), the 135° direction (anti-diagonal direction), etc.

[0133] Taking the gradient direction, which includes both horizontal and vertical directions, as an example, the horizontal gradient can be determined based on the pixel values ​​of the pixels in each channel located in the same row of the image patch, and the vertical gradient can be determined based on the pixel values ​​of the pixels in each channel located in the same column of the image patch. Then, the direction corresponding to the gradient with the smaller value between the horizontal and vertical gradients is determined as the gradient direction.

[0134] Taking a native domain image with the Kodak layout as an example, there are two types of pixel blocks (RGB blocks) in the sampled image: one type has a pixel center point in either the R or B channel (simply referred to as a pixel with center point R or B), and the other type has a pixel center point in either the G channel (simply referred to as a pixel with center point G). The calculation methods for the horizontal and vertical gradients of pixel blocks with center points R or B are similar, only R and B are interchanged. Figure 6A For this application Figure 5 The illustrated embodiment is a schematic diagram of an image block with center point R, as shown below. Figure 6A As shown, the color of the pixel at the center of the image patch is red (R). Therefore, the horizontal gradient Grad of this image patch is... H for:

[0135] Grad H =|G1-G2|+|G2-G3|+|G4-G5|+|G6-G7|+|G7-G8|+|R1-R2|+|R2-R3|+|B1-B2|+|B3-B4|

[0136] The vertical gradient Grad of this image patch V for:

[0137] Grad v =|G1-G6|+|G2-G7|+|G3-G8|+|B1-B3|+|B2-B4|

[0138] Correspondingly, the horizontal gradient Grad of the image patch centered on the pixels of the B channel. H for:

[0139] Grad H =|G1-G2|+|G2-G3|+|G4-G5|+|G6-G7|+|G7-G8|+|R1-R2|+|R2-R3|+|B1-B2|+|B3-B4|

[0140] The vertical gradient Grad of this image patch V for:

[0141] Grad V =|G1-G6|+|G2-G7|+|G3-G8|+|R1-R3|+|R2-R4|

[0142] Figure 6B For this application Figure 5 The illustrated embodiment is a schematic diagram of an image block with center point G, as shown below. Figure 6B As shown, if the color of the pixel at the center of the image patch is green (G), then the horizontal gradient Grad of that image patch is... H for:

[0143] Grad H=|G1-G2|+|G3-G4|+|G4-G5|+|G6-G7|+|R1-R2|+|B1-B2|+|B2-B3|+|B4-B5|+|B5-B6|

[0144] The vertical gradient Grad of this image patch V for:

[0145] Grad V =|G1-G6|+|G2-G7|+|B1-B4|+|B2-B5|+|B3-B6|

[0146] After calculating the gradient of the image patch in each direction, the direction corresponding to the smallest gradient value is determined as the gradient direction of that image patch. Taking the gradient direction as including both horizontal and vertical directions as an example, if Grad... H >Grad V If the gradient direction is vertical, then the gradient direction is determined to be vertical; otherwise, it is determined to be horizontal.

[0147] S502, determine the filter kernel based on the type of the image block in the sampling image corresponding to the overexposed pixel in the white channel and the gradient direction.

[0148] S503, determine the correction value of the overexposed pixel in the white channel based on the product of the filter kernel and the pixels in the image block of the sampled image corresponding to the overexposed pixel in the white channel.

[0149] Specifically, a correspondence can be established in advance between the type of image patch, the gradient direction, and the filter kernel, such as a first correspondence. Then, based on the first correspondence, the type of image patch in the sampled image corresponding to the white channel overexposed pixel and the gradient direction, the filter kernel corresponding to the white channel overexposed pixel is determined. The pixel values ​​in the image patch are then filtered based on the elements in the filter kernel to obtain the correction value of the corresponding white channel overexposed pixel.

[0150] By accumulating the product of the corresponding element in the filter kernel and the corresponding element in the image block corresponding to the overexposed pixel in the white channel, the weighted average value of the pixels in the image block is obtained, which is the correction value of the overexposed pixel in the white channel.

[0151] Table 1 is a first correspondence table provided in the embodiments of this application. As shown in Table 1, the gradient direction includes horizontal gradient and vertical gradient. There are two types of image blocks, so there are four types of filter kernels. Table 1 takes a filter kernel matrix of 3*5 or 5*3 as an example, that is, takes an image block size of 3*5 in the sampled image as an example. The values ​​of the weighting coefficients in each filter kernel are shown in Table 1. By weighting and averaging the pixels in the image block of the sampled image through the corresponding filter kernel, the correction value of the corresponding white channel overexposed pixel is obtained.

[0152] Table 1 First Correspondence Table

[0153]

[0154] S504, if the correction value of the overexposed pixel in the white channel is greater than the pixel value of the overexposed pixel in the white channel, then the pixel value of the overexposed pixel in the white channel is corrected to the correction value.

[0155] In one embodiment, the pixel value of the overexposed pixel in the white channel can be directly corrected to its corresponding correction value, without the need to compare the correction value with the pixel value of the overexposed pixel in the white channel.

[0156] To improve the accuracy of pixel value correction for overexposed pixels in the white channel, the pixel value of an overexposed pixel in the white channel can be corrected to the corresponding correction value only if the correction value is greater than the pixel value of the corresponding overexposed pixel in the white channel.

[0157] In this embodiment, for Binning mode, the native domain image is separated into continuous white channel information and RGB sampling image. By traversing each white channel pixel in the continuous white channel information, overexposed white channel pixels are found, improving the efficiency of overexposed pixel detection. Furthermore, the overexposed white channel pixels are corrected based on the weighted average of the image blocks corresponding to the overexposed white channel pixels in the sampling image, realizing the correction of overexposed pixels in Binning mode, expanding the application scope. Moreover, since the RGB channels have already been averaged during the merge readout, the logic of the correction value calculation is simplified, improving the efficiency of the correction value calculation.

[0158] Figure 7 This is a schematic flowchart of Embodiment 3 of the image reconstruction method provided in this application. This embodiment is... Figure 2 Based on the illustrated embodiment, a step of determining the scene type is added, and step S202 is further refined, such as... Figure 7 As shown, the image reconstruction method provided in this embodiment may include the following steps:

[0159] S701, determine the scene type of the native domain image.

[0160] The scene types include two categories: bright scenes and dark scenes. In dark scenes, the native domain image is generally darker and has a relatively low signal-to-noise ratio; in bright scenes, the native domain image is generally brighter and has a high signal-to-noise ratio.

[0161] The scene type of a native domain image can be determined using any method. For example, it can be determined based on the value range and average value of the white channel pixels in the native domain image. Alternatively, it can be determined based on parameters used when acquiring the native domain image, which may include exposure levels.

[0162] For native domain images in different scene types, different overexposure correction methods can be used to correct overexposed pixels in the white channel.

[0163] S702, if the scene type of the native domain image is a dark scene, then the native domain image is merged and read out to obtain white channel information and sampling image.

[0164] For native domain images in dark scenes, binning mode can be used to merge and read out the native domain images, reducing the image resolution and improving image processing speed without changing the image aspect ratio.

[0165] S703 detects overexposed pixels in the white channel information.

[0166] For each pixel in the white channel information, determine whether its pixel value is greater than a preset threshold. If so, the pixel is determined to be an overexposed pixel in the white channel. The specific method is similar to detecting overexposed pixels in the white channel of the native domain image, except that the detection object is replaced by the white channel information instead of the native domain image, which will not be described in detail here.

[0167] S704, for each overexposed pixel in the white channel information, the overexposed pixel in the white channel is corrected according to the pixel value of each pixel in the image block of the sampling image corresponding to the overexposed pixel in the white channel.

[0168] Specifically, it can be based on Figure 5 The correction method provided by steps S502 to S504 in the illustrated embodiment corrects overexposed pixels in the white channel.

[0169] S705, if the scene of the native domain image is a bright scene, then detect overexposed pixels in the white channel of the native domain image.

[0170] When the scene in the native domain image is a bright scene, since the native domain image contains enough information, there is no need to merge and read out the native domain image, and the overexposed pixels in the white channel of the native domain image can be detected directly.

[0171] Optionally, detect overexposed pixels in the white channel of the image to be processed, including:

[0172] The image to be processed is traversed based on a preset window to obtain each image block; for each image block, overexposed pixels in the white channel are detected.

[0173] S706, based on the first preset interpolation algorithm, determine the first interpolation of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixels in the white channel.

[0174] For example, interpolation of each pixel in the red and blue channels of an image patch can be performed based on an interpolation algorithm of gradient direction.

[0175] S707, Correct the overexposed pixels in the white channel according to the first interpolation of the green channel.

[0176] S708 performs interpolation of the white, red, green, and blue channels based on the corrected pixel values ​​of each channel.

[0177] After correcting the overexposed white channel pixels, a de-mosaic process is performed. Based on the corrected pixel values ​​of each channel, interpolation is performed on the white, red, green, and blue channels to obtain complete white, red, green, and blue channel information.

[0178] Specifically, the white channel can be interpolated first based on the pixel values ​​of the corrected white channel pixels and the pixel values ​​of the other channels to obtain complete white channel information; then, the interpolation of the other channels can be guided based on the complete white channel information to obtain complete red, green and blue channel information.

[0179] S709 performs image reconstruction based on the interpolation results.

[0180] By merging the complete white, red, green, and blue channel information, the reconstructed color image is obtained.

[0181] In this embodiment, different overexposure correction methods are automatically selected based on the scene type of the native domain image, improving the intelligence and accuracy of overexposure correction. For native domain images in dark scenes, the signal-to-noise ratio of the image is improved by merging and reading out native domain images, thereby improving the imaging quality in dark scenes. Furthermore, the number of pixels in the detection object is reduced by merging and reading out, thereby improving the efficiency of overexposure point detection and correction. For native domain images in bright scenes, overexposure correction processing is performed on the native domain image as the object, improving the accuracy of overexposure correction.

[0182] Figure 8 A schematic flowchart of Embodiment 4 of the image reconstruction method provided in this application is shown below. Figure 8 As shown, the above S707 can be implemented through the following steps:

[0183] S801, the pixel values ​​of the corresponding red or blue channel pixels are replaced by the first interpolation of each green channel to obtain the first interpolated image block.

[0184] After obtaining the first interpolation of the green channel corresponding to the pixels of the R / B channel in the image block, the pixel values ​​of the corresponding R / B channel pixels in the image block are replaced with the first interpolation of the green channel, thus obtaining a pixel block composed of 50% white channel pixels and 50% green channel pixels, which is the first interpolated pixel block.

[0185] Figure 9 For this application Figure 7 The schematic diagram of the green channel interpolation process in the illustrated embodiment is as follows: Figure 9 As shown, taking the interpolation of pixel R in the red channel of an image patch as an example, for ease of distinction, this pixel is denoted as pixel R0. Using pixel R0 as the center, the surrounding RGB pixels are obtained. Figure 9 The pixel values ​​of each pixel (with subscripts) are used to calculate the gradient in each direction. For simplicity, taking the horizontal and vertical gradients as examples, we can obtain the horizontal gradient Grad of pixel R0. H and vertical gradient Grad v The horizontal gradient Grad of pixel R0 H The expression is: Grad H =|G1-G2|+|B1-B2|+|R2-R3|, the vertical gradient Grad of pixel R0 v The expression is: Grad v =|G3-G4|+|B1-B3|+|R1-R3|. The gradient direction is determined based on the gradient, and then the green channel is interpolated according to the gradient direction. The gradient direction can include the horizontal direction (G3-G4|+|B1-B3|+|R1-R3|). v -Grad H >Thresh case), vertical direction (Grad) H -Grad v >In the case of Thresh (and in the case of no obvious direction, else), the expression for the first interpolation G0 of the green channel corresponding to pixel R0 is as follows:

[0186]

[0187] Thresh represents a small threshold, such as 0.1, 0.01, or other values.

[0188] The interpolation method for pixels in the blue channel is the same as that for pixels in the red channel. Simply replace the pixels in the red channel with pixels in the blue channel. This will not be elaborated further here.

[0189] After obtaining the first interpolation of the green channel for each pixel in the red and blue channels within the pixel block using the above method, the corresponding red or blue channel pixels within the pixel block are replaced with the first interpolation of the green channel, thus obtaining a pixel block composed of 50% white channel pixels and 50% green channel pixels, i.e., the first interpolated pixel block.

[0190] S802, based on the second preset interpolation algorithm, determine the second interpolation of the green channel corresponding to the overexposed white channel pixel in the first interpolated image block.

[0191] The second preset interpolation algorithm can be based on any interpolation algorithm, which can be the same as or different from the first interpolation algorithm, such as interpolation algorithms based on color difference method, color ratio method, direction weighting, machine learning or deep learning methods.

[0192] For example, an interpolation algorithm based on the gradient direction can be used to interpolate the overexposed pixels in the white channel of an image block to obtain the second interpolation of the green channel corresponding to the overexposed pixels in the white channel.

[0193] You can select the white channel overexposed pixels (W0) and the four green channel pixels (G, G, W0 ... u1 G d1 G 11 and G r1 Horizontal gradient Grad from four pixels v and vertical gradient Grad H The gradient calculation can include the horizontal direction (Grad). V -Grad H >Thresh case), vertical direction (Grad) H -Grad V >In the case of Thresh (and in the case of no obvious direction, the second interpolation G of the green channel corresponding to the overexposed pixel W0 in the white channel) W0 The expression is as follows:

[0194]

[0195] S803, based on the difference between the first interpolation of the green channel and the pixel values ​​of the corresponding red and blue channels, calculate the average red-green difference of the red channel and the average blue-green difference of the blue channel.

[0196] Specifically, taking image patches as units, for each image patch, the difference between the first interpolation value of each green channel and the pixel value of the corresponding red or blue channel pixel in that image patch is calculated and recorded as the color difference. Then, the average value of the color difference corresponding to each red channel pixel (or the sampling point corresponding to each red channel) in the image patch is calculated to obtain the above-mentioned red-green color difference average value. Calculate the average color difference of each pixel in the green channel (or the corresponding sampling point in the red channel) in the image patch to obtain the above-mentioned average blue-green color difference.

[0197] S804, based on the second interpolation of the green channel, the average red-green difference, and the average blue-green difference, the overexposed pixels in the white channel are corrected.

[0198] Correction value for overexposed pixels in the white channel The expression can be:

[0199]

[0200] Among them, G W0 This is the second interpolation of the green channel corresponding to the overexposed pixels in the white channel.

[0201] Specifically, the correction value for overexposed pixels in the white channel If the pixel value W0 is greater than the overexposed pixel value in the white channel, then the overexposed pixel value in the white channel is corrected to the corrected value for the overexposed pixel value in the white channel. That is, the corrected pixel value of the overexposed pixels in the white channel.

[0202] In this embodiment, the pixel values ​​of overexposed pixels in the white channel are corrected by using the average of the RGB three channels. The R value (pixel value of the R channel) and B value (pixel value of the B channel) in the image block are interpolated to obtain a more accurate G value (first interpolation of the green channel). Based on the average GR color difference and average GB color difference corresponding to the conversion, as well as the second interpolation of the green channel corresponding to the overexposed pixel in the white channel, the correction value is calculated. This fully considers the more accurate information of each channel and improves the accuracy of the correction value calculation.

[0203] Figure 10 This is a schematic diagram of the structure of the image reconstruction apparatus provided in the embodiments of this application, such as... Figure 10 As shown, the image reconstruction device includes: an overexposure point detection module 1010, an overexposure point correction module 1020, and an image reconstruction module 1030.

[0204] The overexposure point detection module 1010 is used to detect overexposed white channel pixels in the native domain image; the overexposure point correction module 1020 is used to correct the overexposed white channel pixels in the native domain image based on the pixel values ​​of pixels in other channels that are in the same image block as the overexposed white channel pixels; and the image reconstruction module 1030 is used to reconstruct the image based on the corrected pixel values ​​of the white channel pixels. The overexposed white channel pixels are white channel pixels whose pixel values ​​are greater than a preset threshold.

[0205] In one possible implementation, the overexposure point correction module 1020 is specifically used for:

[0206] Based on the first preset interpolation algorithm, the first interpolation value of the green channel corresponding to each pixel in the red and blue channels that are in the same image block as the overexposed pixel in the white channel is determined; and the overexposed pixel in the white channel is corrected according to the first interpolation value of the green channel.

[0207] In one possible implementation, the native domain image is the merged readout image, including white channel information and a sampled image.

[0208] Correspondingly, the overexposure point detection module 1010 is used for:

[0209] Detect overexposed pixels in the white channel information.

[0210] Correspondingly, the overexposure point correction module 1020 includes:

[0211] The overexposure correction unit is used to correct the overexposure pixels in the white channel based on the pixel values ​​of each pixel in the image block of the sampling image corresponding to the overexposure pixel in the white channel information.

[0212] In one possible implementation, the overexposure point correction unit is specifically used for:

[0213] The gradient direction is determined based on the pixel values ​​of each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; the filter kernel is determined based on the type of the image block in the sampled image corresponding to the overexposed pixel in the white channel and the gradient direction; the correction value of the overexposed pixel in the white channel is determined based on the product of the filter kernel and each pixel in the image block of the sampled image corresponding to the overexposed pixel in the white channel; if the correction value of the overexposed pixel in the white channel is greater than the pixel value of the overexposed pixel in the white channel, the pixel value of the overexposed pixel in the white channel is corrected to the correction value.

[0214] In one possible implementation, the device further includes:

[0215] The scene type determination module is used to determine the scene type of the native domain image.

[0216] The merge readout module is used to merge and read out the native domain image if the scene type of the native domain image is a dark scene, so as to obtain white channel information and sampling image.

[0217] In one possible implementation, the device further includes:

[0218] The first interpolation module is used to determine the first interpolation value of the green channel corresponding to each pixel in the red and blue channels of the same image block as the overexposed white channel pixel, based on a first preset interpolation algorithm, if the scene of the native domain image is a bright scene; the overexposure correction module is used to correct the overexposed white channel pixel according to the first interpolation value of the green channel.

[0219] In one possible implementation, the overexposure correction module includes:

[0220] An interpolated image block acquisition unit is used to replace the pixel values ​​of the corresponding red or blue channel pixels with the first interpolation of each green channel to obtain a first interpolated image block; a second interpolation unit is used to determine the second interpolation of the green channel corresponding to the overexposed white channel pixels in the first interpolated image block based on a second preset interpolation algorithm; an overexposure correction unit is used to calculate the average red-green difference of the red channel and the average blue-green difference of the blue channel based on the difference between the first interpolation of the green channel and the pixel values ​​of the corresponding red and blue channel pixels; and to correct the overexposed white channel pixels based on the second interpolation of the green channel, the average red-green difference, and the average blue-green difference.

[0221] In one possible implementation, the overexposure point detection module 1010 is specifically used for:

[0222] The native domain image is traversed based on a preset window to obtain each image block; for each image block, overexposed pixels in the white channel are detected.

[0223] In one possible implementation, the image reconstruction module 1030 is specifically used for:

[0224] Based on the pixel values ​​of each pixel in the corrected pixel block, interpolation is performed on the white, red, green, and blue channels; based on the interpolation results, image reconstruction is performed.

[0225] In one possible implementation, the device further includes:

[0226] The image acquisition module is used to acquire the native domain image based on the RGBW sensor.

[0227] The image reconstruction apparatus provided in this application embodiment can be used to execute the image reconstruction method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0228] Figure 11 This is a schematic diagram of the structure of the image remodeling group provided in the embodiments of this application, such as... Figure 11 The image remodeling assembly shown includes: a memory 1110 and at least one processor 1120.

[0229] The memory 1110 stores computer execution instructions; at least one processor 1120 executes the computer execution instructions stored in the memory 1110, causing the at least one processor 1120 to perform the image reconstruction method provided in any embodiment of this application.

[0230] The memory 110 and the processor 1120 are connected via a bus 1130.

[0231] The relevant explanations can be understood by referring to the relevant descriptions and effects of the steps provided in the embodiments of the image reconstruction method of this application, and will not be elaborated further here.

[0232] This application also provides a terminal device, including an image sensor and an image remodeling group provided in the above embodiments of this application; wherein the image sensor is used to acquire native domain images.

[0233] In one embodiment, the image sensor is an RGBW sensor.

[0234] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image reconstruction method provided in any embodiment of this application.

[0235] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image reconstruction method provided in any embodiment of this application.

[0236] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. In the embodiments of this application, the order of the above-mentioned process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0237] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method of image reconstruction, characterized by, The method comprises the following steps: determining the scene type of the original domain image; wherein the scene type comprises a bright scene and a dark scene; if the scene type of the original domain image is a dark scene, performing merged readout on the original domain image to obtain white channel information and a sampling map; detecting white channel overexposed pixel points in the original domain image; correcting the white channel overexposed pixel points in the original domain image according to the pixel values of the pixel points of other channels in the same image block as the white channel overexposed pixel points; if the scene of the original domain image is a bright scene, detecting white channel overexposed pixel points in the original domain image; determining a green channel first interpolation value corresponding to each pixel point of a red channel and a blue channel in the same image block as the white channel overexposed pixel points based on a first preset interpolation algorithm; correcting the white channel overexposed pixel points according to the green channel first interpolation value; performing image reconstruction according to the pixel value of the corrected white channel pixel point; wherein the white channel overexposed pixel point is a white channel pixel point with a pixel value greater than a preset threshold.

2. The method of claim 1, wherein, The white channel overexposed pixel point is an overexposed pixel point in the white channel information; correcting the white channel overexposed pixel points in the original domain image according to the pixel values of the pixel points of other channels in the same image block as the white channel overexposed pixel points comprises: for each white channel overexposed pixel point in the white channel information, correcting the white channel overexposed pixel point according to the pixel values of each pixel point in the image block in the sampling map corresponding to the white channel overexposed pixel point.

3. The method of claim 2, wherein, Correcting the white channel overexposed pixel point according to the pixel values of each pixel point in the image block in the sampling map corresponding to the white channel overexposed pixel point comprises: determining a gradient direction according to the pixel values of each pixel point in the image block in the sampling map corresponding to the white channel overexposed pixel point; determining a filter kernel according to the type of the image block in the sampling map corresponding to the white channel overexposed pixel point and the gradient direction; determining a correction value of the white channel overexposed pixel point according to the product of the filter kernel and each pixel point in the image block in the sampling map corresponding to the white channel overexposed pixel point; if the correction value of the white channel overexposed pixel point is greater than the pixel value of the white channel overexposed pixel point, correcting the pixel value of the white channel overexposed pixel point to the correction value.

4. The method of claim 1, wherein, Correcting the white channel overexposed pixel point according to the green channel first interpolation value comprises: replacing the pixel values of the pixel points of the corresponding red channel or blue channel with each green channel first interpolation value to obtain a first interpolation image block; determining a green channel second interpolation value corresponding to the white channel overexposed pixel point in the first interpolation image block based on a second preset interpolation algorithm; calculating a red-green color difference mean value corresponding to the red channel and a blue-green color difference mean value corresponding to the blue channel according to the difference between the green channel first interpolation value and the pixel values of the pixel points of the corresponding red channel and blue channel, respectively; correcting the white channel overexposed pixel point according to the green channel second interpolation value, the red-green color difference mean value and the blue-green color difference mean value.

5. An image reconstruction apparatus, characterized by comprising: The method comprises the following steps: The scene type determination module is configured to determine a scene type of the original domain image; wherein the scene type comprises a bright scene and a dark scene; If the scene type of the original domain image is a dark scene, The overexposure point detection module is configured to perform merged readout on the original domain image to obtain white channel information and a sampling map; and detect white channel overexposure pixel points in the original domain image; and the overexposure point correction module is configured to correct the white channel overexposure pixel points in the original domain image according to pixel values of pixel points of other channels in a same image block as the white channel overexposure pixel points; If the scene of the original domain image is a bright scene, The overexposure point detection module is configured to detect white channel overexposure pixel points in the original domain image; and determine, based on a first preset interpolation algorithm, a first interpolation of a green channel corresponding to each pixel point of a red channel and a blue channel in a same image block as the white channel overexposure pixel points; and the overexposure point correction module is configured to correct the white channel overexposure pixel points according to the first interpolation of the green channel. The image reconstruction module is configured to perform image reconstruction according to pixel values of the corrected white channel pixel points. The white channel overexposure pixel points are white channel pixel points with pixel values greater than a preset threshold.

6. An image reconstruction module, characterized by The image reconstruction apparatus comprises: a memory and at least one processor; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor performs the image reconstruction method according to any one of claims 1 to 4.

7. A terminal device, characterized by, The image reconstruction apparatus comprises an image sensor and the image reconstruction module according to claim 6. The image sensor is configured to acquire an original domain image.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the image reconstruction method according to any one of claims 1 to 4 is implemented.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the image reconstruction method according to any one of claims 1 to 4.

Images