Image sensor, image processing method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses an image sensor which comprises at least one two-dimensional pixel array, the pixel array comprises pixel units arranged in a 2 * 2 mode, and the pixel units are used for sensing light and collecting target color components so that the image sensor can collect color images and monochromatic images, the target color component comprises a first color component and a second color component; the first color component is one of the color components; the second color component is a monochromatic color component, and the monochromatic color component is used for representing the exposure of the color component collected by the pixel unit. The embodiment of the invention further provides an image processing method and device, electronic equipment and a computer storage medium.

Description

Technical Field

[0001] The present application relates to image processing technologies, and in particular to an image sensor, an image processing method, an apparatus, an electronic device, and a computer storage medium. Background Art

[0002] Currently, most manufacturers use image sensors (Sensors) arranged in RGGB to construct color images. After demosaicing the RGGB images, automatic white balance (AWB) and lens shading correction (LSC) are performed. Performing these two operations will cause a lot of information loss in areas where highlights are close to overexposure.

[0003] However, if the exposure is reduced for protection, then information will be lost in dark areas. The light intake of the RGGB color filter itself is already very low. Reducing the exposure further will make it difficult to achieve good image quality in dark areas. In recent years, the RYYB arrangement and the RGBW arrangement have been introduced. However, demosaicing of these two arrangements is very difficult, and the light intake of the Y component and the W component is very large. Limited by the well capacity, it is extremely easy to overexpose, resulting in poor image quality. Summary of the Invention

[0004] Embodiments of the present application provide an image sensor, an image processing method, an apparatus, an electronic device, and a computer storage medium, which can improve the image quality of images.

[0005] The technical solution of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image sensor, including: at least one two-dimensional pixel array, where the pixel array includes: at least four pixel units arranged in a 2×2 layout; where

[0007] The pixel unit is configured to sense light and collect a target color component, so that the image sensor collects a color image and a monochrome image;

[0008] where the target color component includes a first color component and a second color component; the first color component is one of the color components in the color color components; the second color component is a monochrome color component; the monochrome color component is used to represent the exposure amount of the color color component collected by the pixel unit.

[0009] In a second aspect, an embodiment of the present application provides an image processing method, including:

[0010] Receiving the color image and the monochrome image output by the image sensor;

[0011] Determine a white balance gain image based on the color image and the monochrome image;

[0012] Process the color image by using the white balance gain image to obtain an output image.

[0013] In a third aspect, an embodiment of the present application provides an image processing apparatus, including:

[0014] A receiving module, configured to receive a color image and a monochrome image output by the image sensor;

[0015] A determining module, configured to determine a white balance gain image based on the color image and the monochrome image;

[0016] A processing module, configured to process the color image by using the white balance gain image to obtain an output image.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor and a storage medium storing processor-executable instructions; the storage medium depends on the processor to execute operations through a communication bus, and when the instructions are executed by the processor, execute the image processing method described in the above one or more embodiments.

[0018] In a fifth aspect, an embodiment of the present application provides a computer storage medium storing executable instructions, and when the executable instructions are executed by one or more processors, the processor executes the image processing method described in the above one or more embodiments.

[0019] An embodiment of the present application provides an image sensor, an image processing method, apparatus, electronic device, and computer storage medium, including: at least one two-dimensional pixel array, the pixel array including: pixel units arranged in a 2×2 layout, the pixel units being configured to sense light and collect target color components, so that the image sensor collects a color image and a monochrome image, the target color components including a first color component and a second color component, the first color component being one of the color components in the color color components, and the second color component being a monochrome color component, the monochrome color component being used to characterize the exposure amount of the color color component collected by the pixel unit; that is to say, in the embodiment of the present application, each pixel unit in the two-dimensional pixel array of the image sensor is configured to sense light and collect target color components. Since the target color components include one of the color components in the color color components and a monochrome color component, the image sensor adopting the above pixel array enables the image sensor to independently collect a color image and a monochrome image for characterizing the exposure amount of the color image, thereby improving the image quality of the output image after image processing. Description of the Drawings

[0020] Figure 1ais the first pixel unit arranged as RGBW in the related art;

[0021] Figure 1b is the second pixel unit arranged as RGBW in the related art;

[0022] Figure 2a is the rendering effect diagram after demosaicing the RGGB image in the related art;

[0023] Figure 2b is the rendering effect diagram after demosaicing the RGBW image in the related art;

[0024] Figure 3a is the image of the RAW image without white balance in the related art;

[0025] Figure 3b is the image of the RAW image after white balance in the related art;

[0026] Figure 4 is the schematic structural diagram of an optional image sensor provided by an embodiment of the present application;

[0027] Figure 5 is the schematic structural diagram of an example of an optional pixel array provided by an embodiment of the present application;

[0028] Figure 6 is the schematic flow diagram of an optional image processing method provided by an embodiment of the present application;

[0029] Figure 7 is the schematic flow diagram of Example 1 of an optional image processing method provided by an embodiment of the present application;

[0030] Figure 8 is the schematic flow diagram of Example 2 of an optional image processing method provided by an embodiment of the present application;

[0031] Figure 9 is the schematic diagram of an optional gray scale calibration provided by an embodiment of the present application;

[0032] Figure 10 is the schematic diagram of an optional image identifying the overexposed area provided by an embodiment of the present application;

[0033] Figure 11 is the schematic diagram of an optional output image provided by an embodiment of the present application;

[0034] Figure 12 is the schematic diagram of an optional response curve of an image sensor provided by an embodiment of the present application;

[0035] Figure 13A schematic flowchart of an optional process for demosaicing an image provided by an embodiment of the present application;

[0036] Figure 14 A schematic structural diagram of an optional image processing apparatus provided by an embodiment of the present application;

[0037] Figure 15 A schematic structural diagram of an optional ISP provided by an embodiment of the present application;

[0038] Figure 16 A schematic structural diagram one of an optional electronic device provided by an embodiment of the present application;

[0039] Figure 17 A schematic structural diagram two of an optional electronic device provided by an embodiment of the present application. Detailed implementation manners

[0040] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0041] In the related art, RYYB images and RGBW images can be obtained for image processing. However, RYYB images and RGBW images are not friendly to demosaicing, and new demosaicing algorithms need to be designed for these formats, which is relatively difficult to process.

[0042] Figure 1a The first pixel unit arranged as RGBW in the related art, as Figure 1a shown, is a common first pixel unit of RGBW. The RGBW image needs to be converted into an RGGB image for demosaicing. During the conversion process, the information used when calculating the G channel at the W position in the black frame 1a1 is exactly the same as the calculation of the G channel in 1a2, and the position information of 1a1 is missing.

[0043] Figure 1b The second pixel unit arranged as RGBW in the related art, as Figure 1b shown, is the second pixel unit of RGBW. A new solution is given, using diagonal fusion to construct RGGB. However, in this way, all the relative positions in the pixel unit are diagonal, and there are no relative horizontal and relative vertical pixels. Therefore, after demosaicing, these arrangements are prone to color misregistration and generate a lot of color noise.

[0044] Figure 2a The effect diagram after demosaicing the RGGB image in the related art, Figure 2b The effect diagram after demosaicing the RGBW image in the related art. Obviously, Figure 2b Compared withFigure 2a The effect deteriorates.

[0045] In addition, for RYYB images and RGBW images, the Y channel and the W channel are extremely prone to overexposure, and for the additional incident light, the information in the bright area is not sufficiently protected.

[0046] Figure 3a For an image of a RAW image without white balance in the related art, such as Figure 3a As shown, the information of a RAW image may be complete without performing WB. Figure 3b For an image of a RAW image after white balance in the related art, such as Figure 3b As shown, a lot of detailed information is lost after performing WB. It should be noted that, similarly, the same problem will occur with LSC.

[0047] Aiming at the technical problem that the image quality of the output image after image processing is poor due to the images collected by the existing image sensors, the embodiments of the present application provide an image sensor. Figure 4 As a schematic structural diagram of an optional image sensor provided by the embodiments of the present application, such as Figure 4 As shown, the image sensor 400 may include:

[0048] At least one two-dimensional pixel array 41, and the pixel array 41 includes: pixel units 411 arranged in a 2×2 layout; wherein,

[0049] The pixel unit 411 is used for photosensing and collecting target color components, so that the image sensor 400 can collect color images and monochromatic images.

[0050] In the embodiments of the present application, the image sensor 400 includes at least one two-dimensional pixel array 41, and the pixel array 41 includes pixel units 411 arranged in a 2×2 layout; that is to say, it includes 4 pixel units 411. For each pixel unit 411, it can photosense and collect target color components, so that the image sensor 400 can collect color images and monochromatic images.

[0051] For example, the color components in a color image may include one red (Red, R) component, two green (Green, G) components, and one blue (Blue, B) component, and the color component of the monochromatic channel may be a white (White, W) component. Then, the image sensor collects a color image with RGGB color components and a monochromatic image with W color components respectively.

[0052] Among them, the target color component includes a first color component and a second color component; the first color component is one of the color components in the color color component; the second color component is a monochromatic color component; the monochromatic color component is used to characterize the exposure amount of the color color component collected by the pixel unit.

[0053] That is to say, each pixel unit 411 can collect one of the color components in the color color component and a monochromatic color component used to characterize the exposure amount of the color color component collected by the pixel unit. Taking an image sensor arranged in RGBW as an example, the first color component may include: one R component, two G components, and a B component, and the second color component is a W component.

[0054] Then, for the pixel units arranged in 2×2, the pixel unit at the 1×1 position can collect the R component and the W component, the pixel unit at the 1×2 position can collect the G component and the W component, the pixel unit at the 2×1 position can collect the G component and the W component, and the pixel unit at the 2×2 position can collect the B component and the W component. In this way, a color image arranged in RGGB and a monochromatic image of W pixels can be obtained.

[0055] In order to accurately obtain the exposure amount of the color image, in an optional embodiment, the photosensitive element of the first color component in the pixel unit surrounds the photosensitive unit of the second color component.

[0056] It can be understood that for each pixel unit, the photosensitive unit of the first color component can surround the photosensitive unit of the second color component. In this way, the second color component can more accurately characterize the exposure amount of the color color component collected by the pixel unit.

[0057] For the above-mentioned surrounding method, there can be various arrangement methods such that the photosensitive unit of the first color component surrounds the photosensitive unit of the second color component. Here, the embodiments of the present application do not make specific limitations on this.

[0058] In the embodiments of the present application, the following method can be adopted to implement: Figure 5 It is a schematic diagram of an example of an optional pixel array provided by the embodiments of the present application, as Figure 5As shown, the pixel unit 51 includes a photosensitive unit in the shape of a nine-square grid. Among them, the photosensitive unit at the center position of the nine-square grid is the photosensitive unit for the W color component, and the photosensitive units at other positions are the photosensitive units for the R color component. The pixel unit 52 includes a photosensitive unit in the shape of a nine-square grid. Among them, the photosensitive unit at the center position of the nine-square grid is the photosensitive unit for the W color component, and the photosensitive units at other positions are the photosensitive units for the G color component. The pixel unit 53 includes a photosensitive unit in the shape of a nine-square grid. Among them, the photosensitive unit at the center position of the nine-square grid is the photosensitive unit for the W color component, and the photosensitive units at other positions are the photosensitive units for the G color component. The pixel unit 54 includes a photosensitive unit in the shape of a nine-square grid. Among them, the photosensitive unit at the center position of the nine-square grid is the photosensitive unit for the W color component, and the photosensitive units at other positions are the photosensitive units for the B color component.

[0059] In this way, by adopting the above-mentioned surrounding method, the second color component collected by the pixel unit can more accurately represent the exposure amount of the first color component collected by the pixel unit, which helps to collect more accurate color images and monochrome images, so as to improve the image quality of the output image.

[0060] In order to improve the overexposure of the monochrome image, in an optional embodiment, the filter in the photosensitive element of the pixel unit in the image sensor that senses the monochrome channel of the monochrome image is a gray filter.

[0061] It can be understood that for the photosensitive element in the pixel unit of the image sensor that senses the monochrome channel of the monochrome image, there is a risk of overexposure when using a colorless filter. Therefore, here, the filter in the photosensitive element of the pixel unit in the image sensor that senses the monochrome channel of the monochrome image can be a gray filter.

[0062] In this way, by using a gray filter in the photosensitive element that senses the monochrome channel of the monochrome image, the collected monochrome image will not be excessive, thus preventing the problem of overexposure.

[0063] In order to select a suitable gray filter, in an optional embodiment, the gray filter is selected according to the calibrated gray value.

[0064] It can be understood that the above-mentioned gray filter can be selected according to the calibrated radian value. Among them, the calibrated gray value is obtained based on the average value of the response values obtained after the pure white light source passes through the filter of the preset color.

[0065] That is to say, first, based on a pure white light source passing through a filter of a preset color, the response values of each color channel in the preset color can be obtained. Then, the average value of the response values of all color channels is calculated, and the gray value is calibrated using this average value. Next, the optical density is calculated using the gray value. Finally, a gray filter is selected based on the optical density to select a more suitable gray filter.

[0066] In this way, by selecting the gray filter in the above manner, a suitable gray filter can be selected for the photosensitive element that senses the monochromatic channel of the monochromatic image in the image sensor, which helps to determine a more accurate white balance gain image.

[0067] An embodiment of the present application provides an image sensor, including: at least one two-dimensional pixel array. The pixel array includes: pixel units arranged in a 2×2 pattern. The pixel units are used for photosensing and collecting target color components, so that the image sensor can collect color images and monochromatic images. The target color components include a first color component and a second color component. The first color component is one of the color components in the color color components, and the second color component is a monochromatic color component. The monochromatic color component is used to characterize the exposure amount of the color color component collected by the pixel unit. That is to say, in the embodiment of the present application, each pixel unit in the two-dimensional pixel array of the image sensor is used for photosensing and collecting target color components. Since the target color components include one of the color components in the color color components and a monochromatic color component, the image sensor using the above pixel array enables the image sensor to independently collect a color image and a monochromatic image used to characterize the exposure amount of the color image, thereby improving the image quality of the output image after image processing.

[0068] Aiming at the technical problem of poor image effects in RGBW image processing, an embodiment of the present application provides a method for processing an image. Figure 6 For a schematic flowchart of an optional method for processing an image provided by an embodiment of the present application, as Figure 6 shown, the method for processing the image may include:

[0069] S601: Receive a color image and a monochromatic image output by the image sensor;

[0070] The method for processing an image provided by an embodiment of the present application can be applied to the processor of an electronic device or to the image signal processor (ISP) of an electronic device. Here, the embodiment of the present application does not make a specific limitation on this.

[0071] Taking the ISP as the execution entity, the ISP can receive the color image and the monochrome image output by the image sensor described in one or more of the above embodiments. Among them, in the pixel unit of the image sensor, the color channels of the color image and the monochrome channels of the monochrome image are respectively photosensitive.

[0072] That is to say, the pixel unit in the image sensor can be photosensitive to the color channels of the color image, so that the image sensor can collect the color components of each color channel in the color image. Moreover, the image sensor can also be photosensitive to the monochrome channels of the monochrome image, so that the image sensor can collect the color components of the monochrome channels.

[0073] Among them, the monochrome image is used to represent the exposure amount of the color image. It can be seen that the image sensor can know the exposure information of the color image by collecting the monochrome image.

[0074] S602: Determine the white balance gain image according to the color image and the monochrome image;

[0075] After obtaining the color image and the monochrome image through the above S601, in S602, the white balance gain image can be determined according to the color image and the monochrome image together. Among them, here, the artificial intelligence (AI) model method can be used to determine the white balance gain image according to the color image and the monochrome image; the preset calculation method can also be used to determine the white balance gain image according to the color image and the monochrome image.

[0076] It should be noted that the white balance gain image can be directly determined according to the color image and the monochrome image; the color image can also be subjected to other image processing and then determined together with the monochrome image. Here, the embodiments of the present application do not make specific limitations in this regard.

[0077] S603: Process the color image by using the white balance gain image to obtain the output image.

[0078] The white balance gain image can be determined through the above S602. In S603, the determined white balance gain image can be used to process the color image, so that the output image can be obtained.

[0079] Similarly to the above S602, the white balance gain image can be used to process the color image to obtain the output image; the color image can also be subjected to other image processing and then the white balance gain image is used to process the image after other image processing to obtain the output image; here, the embodiments of the present application do not make specific limitations in this regard.

[0080] It should be noted that in the embodiments of the present application, a color image obtained by an image sensor and a monochromatic image for characterizing the exposure amount of the color image are used to determine a white balance gain image, so that the obtained white balance gain image takes into account both the color information of the image and the exposure situation of the image. When performing white balance using the white balance gain image, the overexposure problem of the output image can be improved and the image quality can be enhanced.

[0081] To further improve the accuracy of the determined white balance gain image, in an alternative embodiment, S602 may include:

[0082] Perform demosaicing on the color image to obtain the demosaiced color image;

[0083] Determine the white balance gain image according to the demosaiced color image and the monochromatic image.

[0084] It can be understood that after the color image is obtained, demosaicing can be first performed on the color image. Here, a demosaicing algorithm can be used to implement demosaicing of the color image, so as to obtain the demosaiced color image.

[0085] After the demosaiced color image is obtained, the demosaiced color image and the monochromatic image are then used to determine the white balance gain image. Here, the white balance gain image can be determined according to the demosaiced color image and the monochromatic image in the manner of an AI model; or a preset calculation method can also be used to determine the white balance gain image according to the demosaiced color image and the monochromatic image. Here, the embodiments of the present application do not make specific limitations in this regard.

[0086] It should be noted that in the embodiments of the present application, in addition to using the above-mentioned demosaicing process, other image processing methods can also be used. Here, the embodiments of the present application do not make specific limitations in this regard.

[0087] In this way, by performing demosaicing on the color image, due to the independence between the color image and the monochromatic image, the image quality of the demosaiced image can be improved, and based on this, white balance can be performed to improve the image quality after white balance.

[0088] For the demosaiced color image, in an alternative embodiment, S603 may include:

[0089] Process the demosaiced color image using the white balance gain image to obtain an output image.

[0090] Understandably, based on the demosaicked color image, here, using the determined white balance gain image, the demosaicked color image can be processed, so that an output image after white balance can be obtained.

[0091] In this way, the white balance gain image is determined from the demosaicked color image and used for white balance, so that white balance is further achieved for the demosaicked color image, thereby improving the image quality.

[0092] Furthermore, in order to determine the white balance gain image, in an optional embodiment, determining the white balance gain image according to the demosaicked color image and the monochrome image may include:

[0093] Perform white balance processing on the demosaicked color image to obtain the color image after white balance processing;

[0094] Determine the white balance gain image according to the color image after white balance processing and the monochrome image.

[0095] Understandably, taking the color image as an RGGB image as an example, after demosaicking it, the demosaicked color image is an RGB image. For this image, a white balance algorithm can be used for white balance processing, so that the color image after white balance processing can be obtained, and then the white balance gain image is determined using the color image after white balance processing and the monochrome image.

[0096] In order to implement determining the white balance gain image according to the color image after white balance processing and the monochrome image. Here, the way of an AI model can be adopted to determine the white balance gain image according to the color image after white balance processing and the monochrome image; a preset calculation method can also be adopted to determine the white balance gain image according to the color image after white balance processing and the monochrome image. Here, the embodiments of the present application do not make specific limitations on this.

[0097] In the white balance processing of the demosaicked color image, there may be a problem that the pixel values of the obtained color image after white balance processing cannot be stored due to a small bit width. Here, the bit width can be enlarged, so that the pixel values of the obtained color image after white balance processing can be stored.

[0098] That is to say, here, the white balance gain image is determined using the color image after white balance processing and the monochrome image, so that the determined white balance gain image not only considers the white balance gain of the demosaicked color image, but also considers the exposure of the color image. In this way, the determined white balance gain image is more accurate and helps to improve the image quality.

[0099] In order to determine a white balance gain image based on a color image and a monochrome image after white balance processing, in an alternative embodiment, determining a white balance gain image based on a color image and a monochrome image after white balance processing may include:

[0100] Obtaining a digital gain based on the color image after white balance processing, the white balance gain during white balance processing, and the monochrome image;

[0101] Determining a white balance gain image based on the digital gain and the white balance gain during white balance processing.

[0102] It can be understood that after obtaining the color image after white balance processing, not only can the color image after white balance processing be known, but also the white balance gain during white balance processing can be known. In addition, the digital gain can be obtained by combining the two with the monochrome image. The digital gain is used to correct the white balance gain during white balance processing, so that the white balance gain image can be determined.

[0103] Among them, in obtaining the digital gain based on the color image after white balance processing, the white balance gain during white balance processing, and the monochrome image, the digital gain can be determined based on the color image after white balance processing, the white balance gain during white balance processing, and the monochrome image by using an AI model; the digital gain can also be determined based on the color image after white balance processing, the white balance gain during white balance processing, and the monochrome image by using a preset calculation method. Here, the embodiments of the present application do not make specific limitations in this regard.

[0104] In this way, the white balance gain during white balance processing is corrected by the above method for determining the digital gain, so that the obtained white balance gain image is more accurate, which is beneficial to obtaining an output image with better image quality.

[0105] In addition, in the embodiments of the present application, in addition to performing white balance processing on the image, other processing methods for the image can also be implemented. In an alternative embodiment, the above method may further include:

[0106] Performing LSC processing on the output image to obtain an output image after LSC processing;

[0107] Determining an attenuation region based on the overexposed region of the output image after LSC processing and the texture region of the monochrome image;

[0108] Updating the monochrome image by using the luminance value of the attenuation region to obtain an updated monochrome image;

[0109] Obtaining an attenuation image by using the updated monochrome image and the output image after LSC processing;

[0110] Processing the output image by using the attenuation image in combination with the LSC processing parameters to update and obtain an output image.

[0111] Understandably, after obtaining the output image, LSC processing can also be performed on the output image to obtain the output image after LSC processing. To prevent the overexposure problem of the image after LSC processing, in the embodiments of the present application, an attenuation region can be determined according to the overexposed region of the output image after LSC processing and the texture region of the monochrome image, and the luminance value of the attenuation region is used to update the monochrome image, thereby obtaining the updated monochrome image.

[0112] Then, the attenuated monochrome image and the output image after LSC processing are used to determine the attenuation image, and the attenuation image is combined with the LSC processing parameters to process the output image. Among them, the attenuation image and the LSC processing parameters can be calculated to determine the processing parameters for attenuation processing and LSC processing, and the output image is processed using this parameter to update the output image. It can also be to first use the attenuation image to process the output image to obtain the output image after attenuation processing, and then perform LSC processing on the output image after attenuation processing to update and obtain the output image. Here, the embodiments of the present application do not make specific limitations in this regard.

[0113] In this way, through the above method of determining the attenuation image, attenuation and LSC processing of the output image are realized to update and obtain the output image, so that the overexposure problem of the updated output image is improved, and the dark information of the output image is enhanced, thereby improving the image quality of the output image.

[0114] In an optional embodiment, to determine the overexposed region, the above method may further include:

[0115] Perform overexposure detection on the output image after LSC processing to obtain the first target region;

[0116] The region in the first target region where the pixel value is greater than the preset threshold is determined as the overexposed region.

[0117] Understandably, overexposure detection can be first performed on the output image after LSC processing, so that the first target region can be obtained. Here, in addition to directly using the first target region as the overexposed region, the pixel values in the first target region can also be screened, and the region in the first target region where the pixel value is greater than the preset threshold is used as the overexposed region.

[0118] Among them, the overexposure detection algorithm can be used to implement overexposure detection on the output image after LSC processing; the AI model method can also be used to implement overexposure detection on the output image after LSC processing; here, the embodiments of the present application do not make specific limitations in this regard.

[0119] In this way, the overexposed area can be obtained through the above overexposure detection and comparison with the preset threshold, making the obtained overexposed area more accurate and facilitating the determination of the accurate attenuation area.

[0120] In an optional embodiment, to determine the texture area, the above method may further include:

[0121] Performing texture detection on the monochromatic image to obtain a second target area;

[0122] Performing Gaussian convolution processing on the second target area to obtain the texture area.

[0123] It can be understood that texture detection can be first performed on the monochromatic image, so as to obtain a second target area. Here, in addition to directly using the second target area as the texture area, the pixel values in the second target area can also be screened, and the area where the pixel values in the first target area are greater than the preset value can be used as the texture area; the pixel values in the second target area or the area after screening the second target area can also be expanded. For example, Gaussian convolution processing is performed on the second target area to expand the second target area, so as to obtain the texture area.

[0124] Among them, the following formula can be used to determine the texture area:

[0125]

[0126] Area Coutour =∫∫ I(x,y) Gauss(x,y)Area Coutour (x,y)dxdy (2)

[0127] Among them, Area Coutour represents the texture area.

[0128] Among them, a texture detection algorithm can be used to implement texture detection on the output image after LSC processing; an AI model can also be used to implement texture detection on the output image after LSC processing; here, the embodiments of the present application do not make specific limitations in this regard.

[0129] In this way, the texture area can be obtained through the above texture detection and Gaussian convolution. Using Gaussian convolution to expand the second target area makes the obtained texture area more accurate and facilitates the determination of the accurate attenuation area.

[0130] In an optional embodiment, to determine the attenuation area, according to the overexposed area of the output image after LSC processing and the texture area of the monochromatic image, determining the attenuation area may include:

[0131] Determining the attenuation area according to the intersection of the overexposed area and the texture area.

[0132] Understandably, after obtaining the overexposed region and the texture region, first select the intersection of the overexposed region and the texture region, so that the attenuation region can be determined according to the intersection of the overexposed region and the texture region.

[0133] Here, the attenuation region can be determined based on the intersection of the overexposed region and the texture region. For example, the attenuation region can be determined by shrinking or expanding the intersection. Here, the embodiments of the present application do not make specific limitations in this regard.

[0134] In this way, the attenuation region is determined through the intersection of the overexposed region and the texture region, so that the determined attenuation region is related to the intersection of the overexposed region of the output image after LSC processing and the texture region of the monochrome image, thereby realizing the attenuation processing of the attenuation region, which helps to improve the image quality.

[0135] Furthermore, in order to determine the attenuation region, in an optional embodiment, according to the intersection of the overexposed region and the texture region, determining the attenuation region may include:

[0136] Determine the intersection of the overexposed region and the texture region as the attenuation region.

[0137] Understandably, the intersection of the overexposed region and the texture region can be directly used as the attenuation region. In this way, the determined attenuation region is both an overexposed region and a texture region. The region in the monochrome image that is both an overexposed region and a texture region is updated, so that the overexposure problem of the updated monochrome image is improved, which helps to improve the overexposure problem of the image after LSC processing.

[0138] In addition, in order to determine the attenuation region, in an optional embodiment, according to the intersection of the overexposed region and the texture region, determining the attenuation region may include:

[0139] Expand the intersection of the overexposed region and the texture region to obtain the attenuation region.

[0140] Understandably, in addition to directly using the intersection of the overexposed region and the texture region as the attenuation region, the intersection of the overexposed region and the texture region can also be expanded, so that the attenuation region can be obtained.

[0141] Among them, the expansion method here can be expanded according to a preset rule. For example, the region with a distance less than the preset distance from the intersection region is used as the attenuation region. Other methods can also be used to expand the intersection region to obtain the attenuation region. Here, the embodiments of the present application do not make specific limitations in this regard.

[0142] In this way, the attenuation region is obtained by the above method of further expanding the intersection, so that the determined attenuation region can include a more reasonable overexposed texture region as much as possible, which helps to update the monochrome image and further improve the image quality of the output image.

[0143] In an optional embodiment, to update the monochrome image, updating the brightness value of the attenuation region in the monochrome image to obtain an updated monochrome image may include:

[0144] Replacing the brightness value of the attenuation region in the monochrome image with a preset target brightness value to obtain an updated monochrome image.

[0145] It can be understood that here the brightness value of the attenuation region in the monochrome image is replaced with the target brightness value. The target brightness value is a preset value, and the brightness values of the monochrome image other than the attenuation region remain unchanged. In this way, an updated monochrome image can be obtained.

[0146] It should be noted that the color channel of the above monochrome image can be the luminance component.

[0147] In this way, by updating the brightness value of the attenuation region in the monochrome image above, the brightness value of the attenuation region becomes a preset brightness value. In this way, overexposure of the monochrome image is prevented, and further, the overexposure problem can be improved after the output image is subjected to LSC processing, and the image quality of the image after LSC processing is improved.

[0148] In an optional embodiment, to obtain the attenuation image, using the monochrome image and the output image after LSC processing to obtain the attenuation image may include:

[0149] Determining the attenuation image according to the ratio of the updated monochrome image to the output image after LSC processing.

[0150] It can be understood that after obtaining the updated monochrome image, the ratio between it and the output image after LSC processing can be calculated, so that the attenuation value corresponding to each pixel value in the LSC processing can be obtained, and then the attenuation image can be obtained.

[0151] Finally, the output image is processed using the attenuation image. Specifically, the attenuation value corresponding to each pixel value is multiplied by each pixel value of the output image, thereby realizing the attenuation of the output image.

[0152] In this way, the attenuation image determined by the above ratio method can update the brightness value of the attenuation region of the monochrome image and prevent the overflow of the highlight region. Then, when performing LSC processing on this basis, not only can the overexposure problem of the image be improved, but also the details of the dark part of the image can be enhanced, thereby improving the image quality.

[0153] The following is an example to describe the method for processing an image described in one or more of the above embodiments.

[0154] Figure 7 FIG. 4 is a schematic flowchart of the first example of an optional method for processing an image provided by an embodiment of the present application. As Figure 7 shown, the method for processing the image may include:

[0155] S701: Expose an image sensor (Sensor) arranged in an RGBW arrangement as Figure 7 shown, and read out eight pixel values towards the center (Binning) to obtain a Bayer map;

[0156] S702: Extract the W pixels to obtain a grayscale picture, a Mono map;

[0157] Among them, the Bayer map is equivalent to the above-mentioned color image. When exposing using a Sensor arranged in an RGBW arrangement as Figure 7 shown, a gray filter is covered on the W pixels.

[0158] In addition, extract the W pixel values obtained by exposure to obtain a black-and-white exposure map, a Mono map, and at the same time bin the eight pixels around the W pixels to obtain an RGGB Bayer map. Among them, the Mono map is equivalent to the above-mentioned monochromatic image.

[0159] S703: Generate per-pixel DGain according to the Mono map;

[0160] S704: Perform demosaicing on the Bayer map;

[0161] S705: After demosaicing (Demosaic), combine the gain (RGain) of the R color component, the gain (BGain) of the B color component, and the gain (DGain) during per-pixel white balance processing during white balance processing to obtain a GainMap;

[0162] Among them, the GainMap is equivalent to the above-mentioned white balance gain image. Specifically, after demosaicing the Bayer map, use the AWB algorithm to calculate the WBGain, and at the same time, based on the brightness of the Mono map, calculate different DGains for each pixel, combine the WBGain and the DGain, and pull back the brightness to the brightness of the Mono map.

[0163] S706: Perform white balance WB on the Bayer map using the GainMap to obtain a WB'd map;

[0164] Among them, the WB'd map is equivalent to the above-mentioned output image.

[0165] S707: Determine the overexposed area for the image after WB, determine the texture area for the Mono image, and calculate the decay value Decay based on the overexposed area and the texture area;

[0166] S708: Combine the look-up table of LSC and the decay value Decay to perform LSC, and obtain the RGB with information protection.

[0167] Among them, the above Decay constitutes a decay image.

[0168] Specifically, use the Sobel operator to detect edges on the Mono image, segment the highlight area, take the intersection of the highlight area and the texture-dense area, calculate the decay value Decay in this area, multiply Decay by the look-up table of LSC, and then perform LSC to obtain the final output image.

[0169] Figure 8 It is a schematic flowchart of the second example of an optional image processing method provided by the embodiments of the present application. As Figure 8 shown, the image processing method may include:

[0170] S801: Determine the image sensor;

[0171] Specifically, since it is very easy to overexpose when using a clear filter, the gray level of W is calibrated by RGB. Figure 9 It is a schematic diagram of an optional gray level calibration provided by the embodiments of the present application. As Figure 9 shown, pass a pure white light source through the filters of three colors, RGB, and resolve the response values of the three colors, RGB. Average these three colors to obtain the W value, determine the optical density according to the W value, and select a gray filter according to the optical density.

[0172] S802: Bin the pixels corresponding to RGB towards the center to obtain the corresponding RGGB image;

[0173] S803: Extract the W pixels to obtain the Mono image;

[0174] S804: Perform Demosaic on the RGGB image to obtain the RGB image;

[0175] S805: Calculate the WBGain for the RGB image after Demosaic, correspondingly expand the bit width of the RGB image according to the maximum value of WBGain, and apply WBGain to the RGB image to obtain the RGB image after intermediate white balance;

[0176] S806: Align the pixels of the RGB image after intermediate white balance with the Mono image to obtain the per-pixel luminance-aligned DGain. As shown in the following formula, DGain is a matrix as large as the resolution of the original image, and W represents the exposure image arranged in Mono.

[0177]

[0178] S807: Multiply DGain, RGain, and BGain to obtain the final GainMap. The calculation of GainMap is shown in the following formula.

[0179] GainMap = (RGain * DGain DGain BGain * DGain) (4)

[0180] Wherein, WBGain may include: RGain and BGain.

[0181] S808: Apply GainMap to the RGB image after Demosaic to obtain the RGB image after white balance, and this RGB image does not lose information.

[0182] S809: Apply the LSC table to the RGB image after white balance, and detect the overexposed area of the RGB image after LSC to obtain the overexposed area OverexposureMask;

[0183] Figure 10 For an optional schematic diagram of an image identifying the overexposed area provided by the embodiments of the present application, as Figure 10 shown, in the circled area, the area where the pixel value is greater than the maximum value is marked as the overexposed area.

[0184] S810: At the same time, use the Sobel operator on the Mono image for texture detection, and the area where the detected texture is greater than the threshold is the texture area CoutourMask;

[0185] Since the detected texture area of the edge is a narrow and thin edge, perform Gaussian convolution on the texture area to expand the area, as shown in the above formulas (1) and (2), where G represents the gradient at the current position.

[0186] S811: Take the intersection of the texture area and the overexposed area to obtain the attenuation area, and calculate the attenuation value;

[0187] As the following formula:

[0188] Area decay = Area Coutour ∩ (Area Exposure > Th Exposure ) (5)

[0189] Among them, Area decay represents the attenuation region, Area Coutour represents the texture region, Area Exposure represents the overexposed region, Th Exposure represents the preset threshold of the overexposed region.

[0190] Use the following formula to identify the neighborhood of the attenuation region:

[0191] U(Area decay , δ) = {(x, y)|||(x, y) - Area decay || < δ} (6)

[0192] Among them, δ represents a preset distance, and find the neighborhood on the Mono image.

[0193] Take the neighborhood brightness found on the Mono image as the target brightness to obtain an updated Mono image, and use the following formula to calculate the attenuation value of the attenuation region:

[0194]

[0195] Among them, represents the brightness value of the updated Mono image, and LSCGain * I(x, y) represents the pixel value of the RGB image after LSC.

[0196] S812: Combine Decay and LSC to obtain the final LSC;

[0197] S813: Apply LSC to the RGB image after white balance to obtain the output image.

[0198] Figure 11 is a schematic diagram of an optional output image provided by an embodiment of the present application. As Figure 11 shown, compared with Figure 10 , an RGB image with information recovery can be obtained after attenuation and then performing LSC.

[0199] In this example, the calibration of the W pixel can be extended to a non-linear response. Figure 12 is a schematic diagram of an optional response curve of an image sensor provided by an embodiment of the present application. As Figure 12 shown, the abscissa is: input brightness, and the ordinate is: output brightness.

[0200] Due to the limitation of the well capacity, when using a linear response curve, the Sensor image is likely to be overexposed. When using a non-linear response curve, when the light input is large, the Sensor image can be compressed to a smaller range, so that more exposure information can be carried.

[0201] In addition, since binning will lose a lot of resolution, the resolution directly output by the sensor will be much smaller. To solve this problem, Figure 13 FIG. is a schematic flowchart of an optional process for demosaicing an image provided by an embodiment of the present application. As Figure 13 shown, the missing RGGB pixels are filled by interpolation, and then the image of the RGGB pixels is remosaicked to obtain a RAW image with the original resolution; wherein, Remosaic is a technique used to recombine image pixels in image processing.

[0202] Finally, for the intersection of texture detection and overexposure detection, an AI model can be used for segmentation, and the calculation of the attenuation method and the execution of LSC can also be combined into one step to achieve highlight recovery using AI.

[0203] This example provides a sensor with an RGBW arrangement and a corresponding exposure strategy for information protection. When shooting a high-dynamic scene, the Mono image formed by the W channel can correct the AWB algorithm and the LSC algorithm, thereby preventing the highlight information of the sensor exposure from overflowing. At the same time, the exposure can also be increased to enhance the dark part information. This example is compatible with technologies such as single exposure, double exposure, triple exposure, and multi-transistor multiplexing exposure. Applying this method can achieve the purpose of information protection and facilitate the cooperation between the tone (Tone module) and the auto exposure (Auto Exposure, AE) module in the ISP.

[0204] This example uses RGBW pixels and cooperates with the WB and LSC modules for information protection, increasing the amount of information presented after the sensor exposure, improving the dynamic range of imaging. At the same time, the arrangement and binning strategy will not affect the existing demosaic. This solution only requires one exposure process in the sensor, allows multiple exposures, and can be used for the main and secondary cameras on the camera and different ISPs.

[0205] This example uses the brightness of the W channel to control the final brightness. The final imaging brightness is controlled by AE and will not be changed by AWB. The exposure brightness is completely determined by the light input. This is more conducive to the execution of the tone module in the ISP, and both the input brightness and the target brightness can be controlled by AE. On the other hand, due to the recovery of highlight information, the exposure of the dark part can be appropriately increased, so that the details of the dark part are presented more fully and the highlight information is also preserved.

[0206] An embodiment of the present application provides an image processing method, which receives a color image and a monochrome image output by an image sensor, determines a white balance gain image according to the color image and the monochrome image, and uses the white balance gain image to process the color image to obtain an output image; that is to say, in the embodiment of the present application, a color image and a monochrome image representing the exposure amount are obtained through the image sensor, so that the obtained color image and monochrome image are both independently obtained. Then, based on this, the determined white balance gain image not only considers the color of the image, but also considers the exposure amount of the image, so that the output image obtained by processing the color image with the white balance gain image improves the problems of overexposure of the image or insufficient presentation of dark details, and improves the image quality.

[0207] Based on the same inventive concept as the foregoing embodiment, an embodiment of the present application provides an image processing apparatus Figure 14 which is a schematic structural diagram of an optional image processing apparatus provided by an embodiment of the present application, as Figure 14 shown. The image processing apparatus includes: a receiving module 141, a determining module 142, and a processing module 143; wherein,

[0208] The receiving module 141 is configured to receive a color image and a monochrome image output by an image sensor;

[0209] The determining module 142 is configured to determine a white balance gain image according to the color image and the monochrome image;

[0210] The processing module 143 is configured to use the white balance gain image to process the color image to obtain an output image.

[0211] In an optional embodiment, the determining module 142 is specifically configured to: perform demosaicing processing on the color image to obtain a demosaiced color image; determine a white balance gain image according to the demosaiced color image and the monochrome image.

[0212] In an optional embodiment, the processing module 143 is specifically configured to: use the white balance gain image to process the demosaiced color image to obtain an output image.

[0213] In an optional embodiment, when the determining module 142 determines the white balance gain image according to the demosaiced color image and the monochrome image, it includes: performing white balance processing on the demosaiced color image to obtain a white balance processed color image; determining a white balance gain image according to the white balance processed color image and the monochrome image.

[0214] In an alternative embodiment, the determination module 142 determines a white balance gain image based on the color image and the monochrome image after white balance processing, including: obtaining a digital gain based on the color image after white balance processing, the white balance gain during white balance processing, and the monochrome image; and determining the white balance gain image based on the digital gain and the white balance gain during white balance processing.

[0215] In an alternative embodiment, the apparatus is further configured to: perform LSC processing on the output image to obtain an output image after LSC processing; determine an attenuation region based on the overexposed region of the output image after LSC processing and the texture region of the monochrome image; update the brightness value of the attenuation region in the monochrome image to obtain an updated monochrome image; obtain an attenuation image by using the updated monochrome image and the output image after LSC processing; and process the output image by using the attenuation image in combination with the LSC processing parameters to update and obtain the output image.

[0216] In an alternative embodiment, the apparatus is further configured to: perform overexposure detection on the output image after LSC processing to obtain a first target region; and determine the region where the pixel value in the first target region is greater than a preset threshold as the overexposed region.

[0217] In an alternative embodiment, the apparatus is further configured to: perform texture detection on the monochrome image to obtain a second target region; and perform Gaussian convolution processing on the second target region to obtain a texture region.

[0218] In an alternative embodiment, when the apparatus determines the attenuation region based on the overexposed region of the output image after LSC processing and the texture region of the monochrome image, it includes: determining the attenuation region based on the intersection of the overexposed region and the texture region.

[0219] In an alternative embodiment, when the apparatus determines the attenuation region based on the intersection of the overexposed region and the texture region, it includes: determining the intersection of the overexposed region and the texture region as the attenuation region.

[0220] In an alternative embodiment, when the apparatus determines the attenuation region based on the intersection of the overexposed region and the texture region, it includes: expanding the intersection of the overexposed region and the texture region to obtain the attenuation region.

[0221] In an alternative embodiment, when the apparatus updates the brightness value of the attenuation region in the monochrome image to obtain an updated monochrome image, it includes: replacing the brightness value of the attenuation region in the monochrome image with a preset target brightness value to obtain an updated monochrome image.

[0222] In an alternative embodiment, the apparatus uses the updated monochrome image and the output image processed by the LSC to obtain an attenuation image, including: determining the attenuation image according to the ratio of the updated monochrome image to the output image processed by the LSC.

[0223] In practical applications, the above-mentioned receiving module 141, determining module 142, and processing module 143 can be implemented by a processor located on an image processing device, specifically implemented by a CPU, a microprocessor (Microprocessor Unit, MPU), a digital signal processor (Digital Signal Processing, DSP), or a field programmable gate array (Field Programmable Gate Array, FPGA), etc.

[0224] An embodiment of the present application provides an ISP. Figure 15 As shown in the structural schematic diagram of an alternative ISP provided by an embodiment of the present application, Figure 15 as shown, an embodiment of the present application provides an ISP1500, and ISP1500 includes:

[0225] A processor 151, configured to call and run a computer program from a memory, so that a device installed with the ISP1500 executes the method described in one or more of the above embodiments;

[0226] A transceiver 152, configured to receive and send information during the process of receiving and sending information between a device and the ISP1500.

[0227] Figure 16 As shown in the first structural schematic diagram of an alternative electronic device provided by an embodiment of the present application, Figure 16 as shown, an embodiment of the present application provides an electronic device 1600, including:

[0228] The ISP1500, a processor 161, and a storage medium 162 storing instructions executable by the processor; the storage medium 162 operates depending on the processor 161 through a communication bus 163.

[0229] Figure 17 As shown in the second structural schematic diagram of an alternative electronic device provided by an embodiment of the present application, Figure 17 as shown, an embodiment of the present application provides an electronic device 1700, including:

[0230] A processor 171 and a storage medium 172 storing processor-executable instructions; the storage medium 172 operates depending on the processor 171 via a communication bus 173. When the instructions are executed by the processor, the image processing method executed on the processor side in the above one or more embodiments is executed.

[0231] It should be noted that in actual application, each component in the computer device is coupled together via the communication bus 173. It can be understood that the communication bus 173 is used to implement the connection and communication between these components. In addition to the data bus, the communication bus 173 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 17 all kinds of buses are labeled as the communication bus 173.

[0232] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in the above one or more embodiments.

[0233] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.

[0234] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0235] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0236] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0238] As described above, it is only a preferred embodiment of the present application and is not used to limit the protection scope of the present application.

Claims

1. An image sensor, characterized in that: include: At least one two-dimensional pixel array, the pixel array comprising: pixel units arranged in a 2×2 arrangement; wherein, The pixel unit is used to sense light and collect target color components, so that the image sensor collects color images and monochrome images; Among them, the target color component includes a first color component and a second color component; the first color component is one of the color components of the color component; the second color component is a monochrome color component; the monochrome color component is used to characterize the exposure of the color component collected by the pixel unit.

2. The image sensor according to claim 1, characterized in that The photosensitive elements of the first color component in the pixel unit surround the photosensitive units of the second color component.

3. The image sensor according to claim 1 or 2, characterized in that: The filter of the photosensitive element of the second color component in the pixel unit is a gray filter.

4. The image sensor according to claim 3, characterized in that: The gray filter is selected according to the calibrated gray value; The calibrated grayscale value is obtained based on the average value of the response value obtained after the pure white light source passes through the filter of the preset color.

5. A method for processing an image, characterized in that: include: Receiving a color image and a monochrome image output by the image sensor according to any one of claims 1 to 4; Determine a white balance gain image according to the color image and the monochrome image; The color image is processed using the white balance gain image to obtain an output image.

6. The method according to claim 5, characterized in that The step of determining a white balance gain image according to the color image and the monochrome image comprises: Performing a demosaicing process on the color image to obtain a demosaiced color image; The white balance gain image is determined according to the color image after the demosaicing process and the monochrome image.

7. The method according to claim 6, characterized in that The step of processing the color image using the white balance gain image to obtain an output image includes: The de-mosaiced color image is processed using the white balance gain image to obtain the output image.

8. The method according to claim 6, characterized in that The step of determining the white balance gain image according to the color image and the monochrome image after the demosaicing process comprises: Performing white balance processing on the color image after the demosaicing processing to obtain a color image after the white balance processing; The white balance gain image is determined according to the color image after the white balance processing and the monochrome image.

9. The method according to claim 8, characterized in that The step of determining the white balance gain image according to the color image after white balance processing and the monochrome image comprises: Obtaining a digital gain according to the color image after the white balance processing, the white balance gain during the white balance processing, and the monochrome image; The white balance gain image is determined according to the digital gain and the white balance gain during the white balance processing.

10. The method according to claim 5, characterized in that The method further comprises: Performing LSC processing on the output image to obtain an output image after LSC processing; Determining an attenuation area according to an overexposed area of ​​the output image after the LSC processing and a texture area of ​​the monochrome image; Updating the brightness value of the attenuation area in the monochrome image to obtain an updated monochrome image; Obtaining an attenuation image using the updated monochrome image and the output image after LSC processing; The output image is processed by using the attenuation image in combination with the LSC processing parameters to update the output image.

11. The method according to claim 10, characterized in that The method further comprises: Performing overexposure detection on the output image after the LSC processing to obtain a first target area; An area in the first target area where the pixel value is greater than a preset threshold is determined as the overexposed area.

12. The method according to claim 10, characterized in that The method further comprises: Performing texture detection on the monochrome image to obtain a second target area; Performing Gaussian convolution processing on the second target area to obtain the texture area.

13. The method according to claim 10, characterized in that The determining of the attenuation area according to the overexposed area of ​​the output image after the LSC processing and the texture area of ​​the monochrome image comprises: The attenuation area is determined according to an intersection of the overexposed area and the texture area.

14. The method according to claim 13, characterized in that The determining the attenuation area according to the intersection of the overexposed area and the texture area includes: The intersection of the overexposed area and the texture area is determined as the attenuation area.

15. The method according to claim 13, characterized in that The determining the attenuation area according to the intersection of the overexposed area and the texture area includes: The intersection of the overexposed area and the texture area is enlarged to obtain the attenuated area.

16. The method according to claim 10, characterized in that The step of updating the brightness value of the attenuated area in the monochrome image to obtain an updated monochrome image includes: The brightness value of the attenuated area in the monochrome image is replaced with a preset target brightness value to obtain the updated monochrome image.

17. The method according to claim 10, characterized in that The step of obtaining an attenuated image by using the updated monochrome image and the output image after LSC processing comprises: The attenuation image is determined according to a ratio of the updated monochrome image to the output image after LSC processing.

18. An image processing device, characterized in that: include: A receiving module, configured to receive a color image and a monochrome image output by the image sensor according to any one of claims 1 to 4; A determination module, used for determining a white balance gain image according to the color image and the monochrome image; A processing module is used to process the color image using the white balance gain image to obtain an output image.

19. An electronic device, characterized in that: include: A processor and a storage medium storing instructions executable by the processor; The storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in any one of claims 5 to 17 is executed.

20. A computer storage medium, characterized in that Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the image processing method described in any one of claims 5 to 17.

Images