Image processing method and device

Abstract

The invention discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the steps that a first image is acquired based on first exposure parameters, and the first exposure parameters comprise first exposure time and first conversion gain; determining a target pixel point needing glare calibration in the first image; determining a second conversion gain of the target pixel point based on the first pixel value of the target pixel point and the glare calibration parameter; wherein the first pixel value is the pixel value of the target pixel point in the first image; reading a second pixel value corresponding to the target pixel point based on the second conversion gain to obtain a second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processing method and apparatus. Background Technology

[0002] As electronic devices become increasingly feature-rich in their photography capabilities, consumers' demands for the photography experience are also constantly rising, posing extremely high challenges to the functional integrity, performance stability, and imaging effects of electronic devices.

[0003] In traditional camera shooting, especially in scenes with strong light sources such as night scenes, optical interference phenomena such as ghosting and glare are very likely to occur. These phenomena arise because when light enters from within or outside the field of view, it undergoes multiple reflections within the lens and module. The imaging interference formed by reflected light and stray light is collectively referred to as glare. Typically, this phenomenon is most noticeable when the strong light source is in the center of the frame or just away from the edge of the frame.

[0004] Currently, flare is mainly mitigated through hardware or lens optimization. However, this can only reduce the impact of flare to a certain extent, not eliminate it completely. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method and apparatus that can solve the problem of glare in captured images.

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0007] A first image is acquired based on first exposure parameters, the first exposure parameters including first exposure time and first conversion gain;

[0008] Identify the target pixels in the first image that require glare calibration;

[0009] The second conversion gain of the target pixel is determined based on the first pixel value of the target pixel and the glare calibration parameters; wherein the first pixel value is the pixel value of the target pixel in the first image;

[0010] The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

[0011] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0012] The acquisition module is used to acquire a first image based on first exposure parameters, the first exposure parameters including first exposure time and first conversion gain;

[0013] The first determining module is used to determine the target pixel points in the first image that need to be glare calibration.

[0014] The second determining module is used to determine the second conversion gain of the target pixel based on the first pixel value of the target pixel and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixel in the first image;

[0015] The reading module is used to read the second pixel value corresponding to the target pixel based on the second conversion gain to obtain the second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

[0016] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions implementing the steps of the method as described in the first aspect when executed by the processor.

[0017] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, they implement the steps of the method as described in the first aspect.

[0018] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method as described in the first aspect.

[0019] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect.

[0020] In the embodiments of this application, a first image is acquired based on first exposure parameters, including a first exposure time and a first conversion gain. The first conversion gain determines the amplification factor by which the sensor converts light signals into electrical signals, and the first exposure time controls the duration of light incidence. The first exposure time and the first conversion gain jointly affect the intensity of glare and the real scene signal in the image. Target pixels in the first image that require glare calibration are identified, focusing on areas in the first image that are significantly affected by glare, providing objects for subsequent targeted processing. A second conversion gain for the target pixels is determined based on the first pixel value and the glare calibration parameters. The first pixel value is the pixel value of the target pixels in the first image. Since the response characteristics of glare and the real scene signal on the sensor are consistent and both change with the conversion gain, the second conversion gain that can effectively suppress glare can be determined by combining the first pixel value with the pre-established glare calibration parameters.

[0021] The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain. The second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time. The target pixel is resampled by the second conversion gain, so that the glare signal is effectively suppressed while the real scene signal is preserved. Finally, the second image with glare interference is output, thereby realizing efficient calibration of glare problem in the captured image at the software level and improving the image quality. Attached Figure Description

[0022] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application;

[0023] Figure 2a This is a schematic diagram of a three-channel pixel provided in an embodiment of this application;

[0024] Figure 2b This is one of the schematic diagrams of a single-channel pixel provided in the embodiments of this application;

[0025] Figure 2c This is a second schematic diagram of a single-channel pixel provided in an embodiment of this application;

[0026] Figure 2d This is the third schematic diagram of a single-channel pixel provided in the embodiments of this application;

[0027] Figure 3a This is a schematic diagram of the structure of an image sensor pixel circuit provided in an embodiment of this application;

[0028] Figure 3b This is a schematic diagram of an integrating analog-to-digital converter circuit provided in an embodiment of this application;

[0029] Figure 4 This is a structural diagram of an image processing apparatus provided in an embodiment of this application;

[0030] Figure 5 This is one of the hardware structure diagrams of the electronic device according to an embodiment of this application;

[0031] Figure 6 This is the second schematic diagram of the hardware structure of the electronic device according to an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0033] The terms "first," "second," etc., used in this specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0034] The technical terms used in the embodiments of this application are explained below:

[0035] The CMOS camera module (CCM) is currently the mainstream camera module for mobile phones. It mainly consists of a lens, a voice coil motor (VCM), an infrared filter (IRFilter), an image sensor (CMOS Image Sensor), a digital signal processor (DSP), and a flexible printed circuit (FPC).

[0036] The workflow of a CMOS camera module includes: a voice coil motor driving the lens to achieve precise focusing; external light being focused by the lens and then filtered out by an infrared filter to remove infrared energy before illuminating the image sensor's photodiode (pixel) array; the photodiodes converting the light signal into an electrical signal, which is then amplified by a front-end amplifier circuit and converted into a digital signal matrix by an analog-to-digital converter (ADC); the digital signal processor performing noise reduction, white balance calibration, edge enhancement, and other algorithm processing on the raw data, ultimately compressing and storing the image.

[0037] Stray light is a common optical phenomenon in photography, specifically referring to strong light, such as sunlight or strong artificial light sources. When these light sources enter the lens, they are reflected, scattered, or reflected multiple times by the surface of the optical elements inside the lens, resulting in non-target light distribution on the imaging plane. This causes the image to appear washed out, have halos, or have reduced contrast, which is especially noticeable in backlit shooting scenarios.

[0038] Ghosting, a typical form of flare, refers to the phenomenon where, when a point light source enters the lens, the light undergoes multiple reflections at the interface between the lens air and glass, ultimately forming clear bright spots on the imaging plane that are symmetrically distributed with the position of the light source. Their shape is often similar to the geometric characteristics of the light source, severely affecting the visual representation of the subject in the image.

[0039] To facilitate understanding of the application scenarios of the embodiments of this application, the imaging principle of the camera module is explained below:

[0040] After light passes through the lens, it first passes through an infrared filter to filter out non-visible light bands to avoid interference with visible light imaging, and then enters the microlens array. The function of the microlenses is to focus light onto the photosensitive areas of the pixels on the surface of the image sensor. However, the reflectivity of their optical interfaces and the non-ideal nature of their edge structures can cause light scattering or unexpected reflections: when light is incident at a large angle, diffuse stray light may be generated at the edges of the microlenses, and some light rays may deviate from the target pixel and enter adjacent pixels, forming pixel crosstalk.

[0041] After passing through microlenses, light must pass through a color filter array (CFA) to reach the pixel photodiode. Color filters typically employ a Bayer array structure, consisting of red (R), green (G), and blue (B) filter units arranged in a specific ratio. Due to the non-ideal spectral transmission characteristics of the filter material, non-target wavelengths of light may penetrate the filter, causing wavelength crosstalk. Simultaneously, physical gaps between filter units and defects in the surface coating can lead to light diffraction or scattering, further introducing stray light.

[0042] In the pixel structure of an image sensor, structures such as the metal wiring around the photodiode, the pixel isolation layer, and the back glass may cause specular or diffuse reflection of incident light. For example, the high reflectivity of the metal wiring can cause light to be reflected twice between pixels, while the reflected light from the interface between the back glass and the air may pass through the CFA and microlenses again, forming cyclically reflected stray light.

[0043] Stray light caused by reflection, scattering, and diffraction from the lens, IR filter, microlens, CFA, and sensor structure is significantly amplified in point light source scenarios. The high-energy-density light from the point source is reflected multiple times between the lens elements, forming ghost images symmetrically distributed with respect to the light source position. Essentially, this is a structured superposition of stray light on the imaging plane. Due to lens aberrations and microlens focusing errors, the propagation path of stray light at different incident angles exhibits highly nonlinear characteristics, resulting in a complex distribution of stray light that varies with the light source angle, making it difficult to accurately describe with a single model. When the stray light energy exceeds the image sensor's noise floor, it leads to decreased image contrast, color distortion, and overexposure in localized areas, severely impacting image quality.

[0044] The image processing method provided in this application embodiment can be applied to at least the following application scenarios, which will be described below.

[0045] Currently, in traditional camera imaging systems, ghosting and flare are mainly caused by multiple reflections in unexpected optical paths, especially noticeable in scenes with strong point light sources. Their physical essence is Fresnel reflection that occurs when light encounters interfaces between different media along its optical path. Specifically, these can be categorized into four key reflection paths:

[0046] Interface between cover glass and lens element: Incident light undergoes multiple reflections between the rear surface of the cover glass and the front surface of the lens. Such reflected light usually appears as large-area hazy glare, resulting in a decrease in overall image contrast.

[0047] Air interface between adjacent lenses within a lens group: Light is crosstalked and reflected at the uncoated or ineffective glass-air interface within the lens group, forming ring-shaped or polygonal light spots, the shape of which is directly related to the structure of the lens aperture blades.

[0048] Interface between the end surface of the lens and the front surface of the infrared filter: The reflected light from this interface superimposes with the main optical path to form an off-axis ghost image on the sensor plane, which usually appears as a virtual bright spot symmetrical to the light source.

[0049] Interface between the back surface of the infrared filter and the microlens layer of the image sensor: The sensor surface produces strong back reflection due to the high refractive index of silicon. This reflected light passes through the infrared filter in the opposite direction and is reflected back to the sensor, forming a "sensor ghost" located close to the light source. Its intensity is limited by the microlens array design and the anti-reflective coating performance of the sensor.

[0050] When a strong light source is located in the center of the image or slightly beyond the edge of the field of view, the angle between the aforementioned reflection path and the main imaging light path is the smallest, making it easiest for the reflected light energy to be captured by the sensor, resulting in particularly significant ghosting and glare phenomena. Currently, such stray light is mainly suppressed through hardware-level anti-reflection technology, but current solutions cannot completely eliminate flare, only minimize it.

[0051] Taking a four-bayer pixel sensor as an example to illustrate the circuit function, this sensor achieves phase detection autofocus (PDAF) through a pixel array structure design. In its physical structure, every four pixels of the same color share a microlens. This type of complementary metal-oxide-semiconductor image sensor (CIS) is called a quad pixel diagonal complementary metal-oxide-semiconductor image sensor (QPDCIS).

[0052] The physical structure of a Quad Bayer sensor, taking four red (R) pixels as an example:

[0053] The circuit structure corresponding to pixels R1 and R2 is as follows: photodiodes PD1 and PD2 are independent photosensitive units, with their photosensitive time controlled by transmission gate transistors TG1 and TG2, respectively. TG1 and TG2 control the switching of the photosensitive process of PD1 and PD2 through gate voltage signals. RST1 is a reset transistor used to clear residual photogenerated electrons in PD1, PD2, and the floating diffusion (FD1) to ensure the accuracy of the pixel signal. FD1 is a floating diffusion node shared by the two R pixels, equivalent to a charge storage capacitor, which receives the charge transferred from PD1 and PD2 through the TG5 transmission gate transistor. SF1 is a source follower (SF), which acts as a buffer amplifier to convert the charge signal in FD1 into a voltage signal and transmit it to the output. SET1 is a row select transistor. When SET1 is turned on, the voltage signal in FD1 is transmitted to the ultra-analog-to-digital converter (Ultra-Digital Converter) through SF1. The ADC module converts the output voltage into a digital signal. The circuit structures of pixels R3 and R4 below are completely symmetrical with those of R1 and R2, and they process the photosensitive signal through independent transmission gate transistors, reset transistors, and a shared floating diffuser.

[0054] In PDAF implementation, the Quad Bayer sensor can output either left-right or top-bottom phase difference information: the left-right phase difference is calculated based on the signal difference between the R1+R3 pixel group and the R2+R4 pixel group, suitable for focusing in vertical stripe scenes; the top-bottom phase difference is calculated based on the signal difference between the R1+R2 pixel group and the R3+R4 pixel group, suitable for focusing in horizontal stripe scenes. Both phase difference calculation methods are based on the optical path difference principle of pixel pairs. By comparing the phase difference of the light-sensing signals of pixels at different positions, the voice coil motor (VCM) is driven to adjust the lens position, achieving fast and accurate focusing.

[0055] In response to the problems in related technologies, this application provides an image processing method and apparatus that can solve the problem of glare in captured images in related technologies.

[0056] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0057] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application.

[0058] like Figure 1 As shown, the image processing method may include steps 110-140, and the method is applied to an image processing device, as detailed below:

[0059] Step 110: Obtain a first image based on first exposure parameters, wherein the first exposure parameters include a first exposure time and a first conversion gain;

[0060] The first exposure parameter is the core parameter for controlling sensor imaging. The first exposure time is the duration for which the sensor receives light, usually in milliseconds. The first conversion gain is the amplification factor by which the sensor converts photogenerated charge into an electrical signal, affecting signal strength and noise level.

[0061] The first image is acquired based on the first exposure parameters. At this time, the first exposure time determines the amount of light accumulated on the sensor, and the first conversion gain determines the degree of signal amplification. Since glare is essentially the imaging of stray light on the sensor, its signal is the same as the real scene signal. It will be enhanced with the extension of exposure time and amplified with the increase of conversion gain. Therefore, the first image contains both real scene information and superimposed glare signal.

[0062] Step 120: Determine the target pixels in the first image that require glare calibration;

[0063] The target pixel refers to the pixel in the first image that is significantly affected by glare, usually manifested as an abnormally bright pixel, a pixel with color distortion, or a pixel with light spots.

[0064] Image analysis is used to identify target pixels and regions affected by glare. Image analysis methods include brightness threshold determination and neighboring pixel differences. The first pixel value of the target pixel often contains excessively high glare components, causing details of the real scene to be obscured.

[0065] Step 130: Determine the second conversion gain of the target pixel based on the first pixel value of the target pixel and the glare calibration parameters; wherein, the first pixel value is the pixel value of the target pixel in the first image;

[0066] Glare calibration parameters are pre-measured experimentally and are used to describe the adjustment law of conversion gain under different glare intensities; the second conversion gain is a new conversion gain determined for the target pixel to suppress glare, and the difference between the second conversion gain and the first conversion gain is the key to achieving calibration.

[0067] Glare signals and real scene signals have different response characteristics under different conversion gains. When the conversion gain is reduced, the amplification of glare, a high-intensity interference signal, will be significantly reduced. Although the real scene signal will also be reduced, it can be recovered through subsequent processing. Therefore, by combining the glare intensity reflected by the first pixel value, the second conversion gain that can effectively suppress glare can be determined.

[0068] Step 140: Read the second pixel value corresponding to the target pixel based on the second conversion gain to obtain a second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

[0069] The second pixel value of the target pixel is read based on the second conversion gain to obtain the second image. At this time, the first exposure time is kept unchanged to ensure that the amount of light accumulation is consistent with the first image. The signal amplification factor is adjusted only by changing the conversion gain, so that the glare component of the target pixel is significantly suppressed, and the real scene signal is clearly presented after deducting the excessively amplified glare.

[0070] By dynamically adjusting the conversion gain, the amplification of glare signals is specifically reduced, achieving effective separation between glare and real signals. Compared to traditional hardware optimization, which can only passively reduce glare, dynamic parameter adjustment at the software level can accurately calibrate according to the glare intensity of different pixels, significantly improving the image quality in strong light source scenes. At the same time, it avoids the problem of the overall image being too dark or too bright due to changes in exposure time, taking into account both the brightness balance and detail preservation of the image.

[0071] In one possible embodiment, the following steps may be included before step 130:

[0072] Multiple point light sources at different angles are captured to obtain calibration images using preset reference conversion gain and reference exposure time; the pixel coordinates affected by glare in each color channel of the calibration images are identified, and the corresponding glare calibration parameters are recorded; a glare parameter database is established based on the pixel coordinates and glare calibration parameters.

[0073] Using a pre-set reference gain and reference exposure time, multiple point light sources at different angles are photographed to obtain calibration images. By selecting point light sources at different locations, various glare scenarios in actual shooting can be simulated. The central light source simulates glare when facing the light source directly, while the edge light sources simulate the effect of oblique light sources. Uniform illumination intensity ensures the consistency and comparability of the collected data and avoids interference with subsequent analysis due to differences in illumination.

[0074] After shooting, each color channel of the calibration image is analyzed independently. Image processing algorithms are used to identify the coordinates of pixels affected by glare in each channel. For example, a threshold-based method can be used, setting a reasonable brightness threshold and marking pixels exceeding the threshold as affected by glare; alternatively, machine learning models can be employed to accurately identify the locations of glare-affected pixels by learning from a large number of labeled glare images. Simultaneously with determining the pixel coordinates, the glare calibration parameters corresponding to each pixel are recorded; this value represents the degree of glare interference experienced by the pixel under standard conditions.

[0075] By identifying the coordinates of pixels affected by glare in each color channel of the calibration image and recording the corresponding glare calibration parameters, and since the calibration image consists of multiple color channels, identifying the coordinates of pixels affected by glare in each channel allows for more detailed and comprehensive capture of glare information. The glare calibration parameters for each pixel reflect the degree of glare interference experienced by that pixel under standard conditions.

[0076] Finally, a glare parameter database is established based on the pixel coordinates affected by glare in each color channel and the glare calibration parameters. This database acts like a dictionary, storing pixel coordinates and their corresponding glare calibration parameters in a one-to-one correspondence, providing accurate reference data for subsequent processing of target pixels in the current captured frame.

[0077] After collecting pixel coordinates and glare calibration parameters, a glare parameter database can be established. Specifically, sub-databases are created for the red, green, and blue channels to achieve fine-grained management of glare data for different color channels. Taking the red channel sub-database as an example, a suitable data structure such as a hash table or array is used to store the identified pixel coordinates affected by glare in the red channel. Each coordinate corresponds to a record, which also stores the glare calibration parameters for the corresponding pixel. The green and blue channel sub-databases are established in the same way to ensure that each sub-database completely records the glare data for its corresponding color channel. Finally, these three sub-databases, each corresponding to a different color channel, together constitute the complete glare parameter database.

[0078] Therefore, the differences in light sensitivity and response characteristics of different color channels are fully considered. Since each color channel behaves differently when affected by glare, glare data can be stored and retrieved more accurately. In subsequent glare compensation processing of target pixels, glare calibration parameters can be quickly and accurately extracted from the corresponding sub-database based on the color channel of the target pixel. These parameters are then combined with the gain and exposure time under the current shooting conditions for compensation calculation, thereby achieving refined glare processing in different color channels.

[0079] Among them, the multiple point light sources at different angles include at least one of the following: a central point light source at the center of the image sensor's field of view, a first edge point light source at the upper left corner of the image plane, a second edge point light source at the upper right corner, a third edge point light source at the lower left corner, and a fourth edge point light source at the lower right corner; the central point light source, the first edge point light source, the second edge point light source, the third edge point light source, and the fourth edge point light source have the same illumination intensity.

[0080] The multiple point light sources at different angles include at least one of the following: a central point light source at the center of the image sensor's field of view; a first edge point light source at the upper left corner of the image plane; a second edge point light source at the upper right corner; a third edge point light source at the lower left corner; and a fourth edge point light source at the lower right corner. All point light sources have the same illumination intensity. By selecting these point light sources at different locations, the area of ​​the image sensor that may be affected by glare can be fully covered. The central point light source can simulate the glare situation when facing the light source directly, while the edge point light sources can simulate the glare effect caused by oblique light sources.

[0081] It is understood that multiple point light sources at different angles may further include at least one of the following: a fifth edge point light source above the image plane, a sixth edge point light source on the left, a seventh edge point light source on the right, and an eighth edge point light source below. This application does not limit the number or position of multiple point light sources at different angles.

[0082] Maintaining consistent light intensity is crucial for acquiring glare data from different locations under a unified standard, avoiding interference from variations in light intensity. Preset baseline gain and exposure time lay the foundation for establishing a unified reference standard, ensuring data consistency and comparability.

[0083] In one possible embodiment, the step of establishing a glare parameter database based on the pixel coordinates and reference glare calibration parameters mentioned above may specifically include the following steps:

[0084] Sub-databases are established for the red, green, and blue channels respectively. Each sub-database stores the pixel coordinates and glare calibration parameters of the corresponding color channel that are affected by glare.

[0085] Preview images are obtained by capturing multiple point light sources at different angles under preset reference gain and reference exposure time. These point light sources include the center point light source at the center of the image sensor's field of view, as well as the edge point light sources at the upper left, upper right, lower left, and lower right corners of the image plane, while maintaining consistent illumination intensity for all point light sources, thus comprehensively simulating various glare scenes.

[0086] Each color channel of the preview image is analyzed independently, and image processing algorithms are used to identify the coordinates of pixels affected by glare in each channel. For example, in the red channel, specific threshold judgments or machine learning models are used to mark the locations of pixels whose brightness is abnormal due to glare; similar identification methods are used in the green and blue channels to determine the affected pixels. While identifying the pixel coordinates, the glare calibration parameters corresponding to each pixel are recorded; this value reflects the degree of glare interference to the pixel under standard conditions.

[0087] After completing the acquisition of pixel coordinates and glare calibration parameters, as follows: Figures 2a-2d As shown, taking the red channel sub-database as an example, the coordinates of pixels affected by glare in the red channel are identified and stored according to a specific data structure. Each coordinate corresponds to a record, which also stores the glare calibration parameters for the corresponding pixel. The green and blue channel sub-databases are established in a similar manner, ensuring that each sub-database completely stores the glare data for its corresponding color channel. Ultimately, these three sub-databases for different color channels together form a complete glare parameter database.

[0088] In Flare images, the Flare ranges of the red, green, and blue (R / G / B) channels differ, which is essentially related to the characteristics of the sensor's color filters, differences in light wavelengths, and the dispersion effect of the optical system. Different colors of light propagate differently in the lens and sensor, and individual calibration of each channel can precisely eliminate glare interference in full-color images.

[0089] Because the sensor's pixel array covers the R / G / B filters via a Bayer array, each filter only allows light of a specific wavelength to pass through:

[0090] Red light (R): The filter allows light with a wavelength of 600-700nm to pass through, while blocking other wavelengths by more than 90%.

[0091] Green light (G): Allows light with a wavelength of 500-600nm to pass through, while blocking red and blue light;

[0092] Blue light (B): Allows light with a wavelength of 400-500nm to pass through, while blocking red and green light.

[0093] When a point light source generates flare, stray light is filtered during reflection / scattering within the lens and sensor. For example, stray light generated by the blue light component of a point light source will only be detected by pixels using a blue filter, while red / green pixels are almost unaffected; stray light with the red light component is primarily received by pixels using a red filter. Therefore, stray light of different colors can only activate pixels in their corresponding channels, resulting in a direct correlation between the flare range of each channel and the spectral composition of the light source and the transmittance of the filter, inevitably leading to differences.

[0094] Red light: strong penetrability, weak scattering but wide reflection range, so the flame of the red channel may exhibit a diffuse characteristic of "large range, low brightness"; Blue light: weak penetrability, strong scattering but narrow reflection range, so the flame of the blue channel may exhibit a concentrated characteristic of "small range, high brightness"; Green light: characteristics are between red and blue, and the flame range and brightness distribution are also in the middle.

[0095] In the 2×2OCL (On Chip Lens) structure, the microlenses have different focusing accuracies for different wavelengths: the focal length of red light is slightly longer and that of blue light is slightly shorter, which causes the stray light to fall at different points on the pixels of each channel, further expanding the difference in the three-channel Flare range.

[0096] By establishing separate sub-databases, the differences in light sensitivity and response characteristics of different color channels are fully considered. Since each color channel behaves differently when affected by glare, creating separate tables allows for more accurate storage and retrieval of glare data.

[0097] When performing glare compensation on the target pixel in the subsequent process, the glare calibration parameters can be quickly and accurately extracted from the corresponding sub-database based on the color channel of the target pixel. The compensation calculation is then performed in combination with the gain and exposure time under the current shooting conditions, thereby achieving refined processing of glare in different color channels and significantly improving the accuracy and effectiveness of image glare compensation.

[0098] In one possible embodiment, step 130 may specifically include the following steps:

[0099] From the glare parameter database, determine the glare calibration parameters corresponding to the first pixel value of the target pixel; the glare parameter database includes: multiple sets of corresponding pixel coordinates and glare calibration parameters affected by glare obtained based on the reference conversion gain and the reference exposure time; determine the gain ratio of the first conversion gain and the reference conversion gain; determine the exposure ratio of the first exposure time and the reference exposure time; determine the product of the gain ratio, the exposure ratio and the glare calibration parameters as a dynamic compensation value; determine the second conversion gain based on the dynamic compensation value.

[0100] The glare calibration parameters corresponding to the first pixel value of the target pixel are determined from the glare parameter database. This glare parameter database is pre-built based on the reference conversion gain and reference exposure time and contains multiple sets of mapping relationships between pixel coordinates affected by glare and corresponding glare calibration parameters. Based on the first pixel value, the basic calibration data applicable to the pixel can be accurately matched.

[0101] Calculate the gain ratio of the first conversion gain to the reference conversion gain, and the exposure ratio of the first exposure time to the reference exposure time. The gain ratio quantifies the degree of deviation of the conversion gain from the reference conditions during actual shooting, and the exposure ratio quantifies the degree of deviation of the exposure time from the reference conditions. Since the intensity of the glare signal will be amplified as the conversion gain increases and accumulated as the exposure time increases, the gain ratio and exposure ratio directly reflect the enhancement or reduction factor of the glare signal from the reference state in the actual scene.

[0102] The gain ratio and exposure ratio are multiplied by the glare calibration parameters obtained from the database to obtain the dynamic compensation value. The dynamic compensation value integrates the difference between the reference calibration data and the actual shooting parameters, and can accurately characterize the calibration range required for the target pixel under the current conditions.

[0103] The second conversion gain is determined based on the dynamic compensation value. For example, when the dynamic compensation value is high, it indicates that the glare effect is strong. The second conversion gain will be adjusted to a value lower than the first conversion gain to weaken the amplification of the glare signal. In this way, the conversion gain can be accurately and dynamically adapted, laying the foundation for the subsequent effective elimination of glare.

[0104] In one possible embodiment, step 140 may specifically include the following steps:

[0105] The second conversion gain is converted into a compensation voltage value; a target reference voltage is obtained based on the compensation voltage value and the reference voltage value corresponding to the first pixel value; the pixel voltage of the target pixel is converted from analog to digital based on the target reference voltage to obtain the second pixel value.

[0106] The second conversion gain is converted into a compensation voltage value because the sensor's conversion gain is essentially achieved through the reference voltage in the circuit. Different conversion gains correspond to different voltage amplification coefficients. Through the preset gain-voltage mapping relationship, the abstract parameter of the second conversion gain can be converted into a specific compensation voltage value, which directly reflects the voltage adjustment range required to suppress glare.

[0107] Based on the compensation voltage value and the reference voltage value corresponding to the first pixel value read out, the target reference voltage is calculated. The reference voltage value when the first pixel value is read out is the original voltage parameter corresponding to the first conversion gain. By combining the compensation voltage value and the reference voltage value, the target reference voltage that adapts to the second conversion gain can be obtained, ensuring that the amplification factor at the circuit level matches the second conversion gain.

[0108] The pixel voltage of the target pixel is converted from analog to digital based on the target reference voltage to obtain the second pixel value. Pixel voltage is an analog electrical signal generated by the sensor after light sensing, containing both real signal and glare signal. Analog-to-digital conversion is the process of converting analog voltage into digital pixel value. Since the target reference voltage corresponds to a lower conversion gain, and the conversion gain is directly related to the signal amplification factor of the analog-to-digital conversion, a lower gain means that the same analog voltage is amplified less when converted into a digital signal. Therefore, during the conversion process, the excessively high analog voltage caused by glare is converted into a digital signal with a lower amplification factor, thus significantly reducing its proportion in the second pixel value. Although the real scene signal is also weakened accordingly, clear detail is still maintained because the glare component is effectively suppressed. Ultimately, the target pixel in the second image is free from glare interference, achieving accurate glare calibration.

[0109] Specifically, the step of performing analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value may include the following steps:

[0110] The pixel voltage is input to the integrator of the integrating analog-to-digital converter circuit; the target reference voltage is input to the reference voltage terminal of the integrating analog-to-digital converter circuit; the output voltage of the integrator is compared with the threshold voltage by a comparator, and a comparison signal is output to a binary counter; the second pixel value is generated based on the digital signal of the binary counter.

[0111] The pixel voltage of the target pixel is input to the integrator of the integrating analog-to-digital converter circuit. The pixel voltage is the analog electrical signal generated by the sensor after light sensing, which includes the superposition of the real scene signal and the glare signal. The integrator performs integration on the analog voltage over a period of time, converting the voltage signal into a time-dependent accumulated charge signal, thus transforming the instantaneous voltage signal into a more stable and processable signal.

[0112] The previously determined target reference voltage is input to the reference voltage terminal of the integrating analog-to-digital converter circuit. The target reference voltage directly determines the discharge rate and conversion accuracy of the integrator. Its value is matched with the second conversion gain. By adjusting the reference voltage, the amplification ratio of the analog-to-digital conversion can be changed, thereby suppressing the glare signal.

[0113] The comparator in the circuit continuously compares the output voltage of the integrator with a preset threshold voltage. When the output voltage of the integrator reaches the threshold voltage, the comparator outputs a comparison signal to the binary counter. This signal serves as the stop signal for the counter, marking the end of the integration process.

[0114] The second pixel value is generated based on the digital signal accumulated by the binary counter during the integration process. The counting result of the counter is proportional to the integration time, which is related to the ratio of the pixel voltage to the target reference voltage. Since the target reference voltage is calibrated to adapt to the second conversion gain, the integration time corresponding to the excessively high pixel voltage caused by glare is compressed during the conversion process. The proportion of glare component in the digital signal accumulated by the counter is significantly reduced. The final generated second pixel value can more realistically reflect the real scene information of the target pixel, and effectively eliminate glare interference.

[0115] In one possible embodiment, the integrating analog-to-digital converter circuit includes:

[0116] The glare parameter calculation module is used to determine, from the glare parameter database, a glare calibration parameter corresponding to the first pixel value of the target pixel; and to determine, based on the first pixel value of the target pixel and the glare calibration parameter, a second conversion gain of the target pixel.

[0117] A voltage control module, connected to the glare parameter calculation module, is used to convert the second conversion gain into a compensation voltage value; and to obtain a target reference voltage based on the compensation voltage value and the reference voltage value corresponding to the first pixel value when it is read out.

[0118] The analog-to-digital conversion module is connected to the glare parameter calculation module and the voltage control module, respectively, and is used to perform analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value.

[0119] As the core control unit, the glare parameter calculation module mainly undertakes the following tasks: First, it accurately matches the corresponding glare calibration parameters from the pre-built glare parameter database based on the first pixel value of the target pixel. This database stores the correlation between the pixel coordinates affected by glare and the calibration parameters under the reference conversion gain and reference exposure time. Second, based on the first pixel value of the target pixel and the matched glare calibration parameters, it calculates the dynamic compensation value by combining the actual gain ratio and exposure ratio with the reference, and finally determines the second conversion gain adapted to the current scene, providing a basis for subsequent voltage adjustment.

[0120] The voltage control module is responsible for converting abstract gain parameters into voltage signals that the circuit can recognize. First, it converts the second conversion gain output by the glare parameter calculation module into a specific compensation voltage value. Then, it uses this compensation voltage value and the reference voltage value corresponding to the first pixel value to generate a target reference voltage. The target reference voltage determines the signal amplification ratio in the analog-to-digital conversion process and is a key circuit parameter for suppressing glare.

[0121] The analog-to-digital converter (ADC) module is responsible for converting analog signals to digital signals. Internally, it comprises three core components: an integrator, a comparator, and a binary counter. The integrator receives the pixel voltage of the target pixel and performs integration, converting the voltage signal into accumulated charge. The comparator continuously compares the integrator's output voltage with a preset threshold voltage; when the threshold is reached, it outputs a stop signal to the binary counter. The binary counter counts based on the duration from the start of integration to the stop signal, ultimately converting the count result into a digital signal, which is the second pixel value obtained after eliminating glare interference. Through the coordinated operation of these three modules, the ADC module can perform glare calibration in real time during signal conversion without additional post-processing steps. This ensures calibration efficiency and significantly improves imaging quality in strong light source scenarios through dynamic parameter adjustment at the hardware level.

[0122] In embodiments of this application, a first image is acquired based on first exposure parameters, including a first exposure time and a first conversion gain. The first conversion gain determines the amplification factor by which the sensor converts light signals into electrical signals, and the first exposure time controls the duration of light incidence. The first exposure time and the first conversion gain jointly affect the intensity of glare and real-scene signals in the image. Target pixels in the first image that require glare calibration are identified, focusing on areas in the first image significantly affected by glare, providing targets for subsequent targeted processing. A second conversion gain for the target pixels is determined based on the first pixel value and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixels in the first image; since glare and real-scene signals have the same response characteristics on the sensor, both changing with the conversion gain, a second conversion gain that effectively suppresses glare can be determined by combining the first pixel value with pre-established glare calibration parameters.

[0123] The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain. The second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time. The target pixel is resampled by the second conversion gain, so that the glare signal is effectively suppressed while the real scene signal is preserved. Finally, the second image with glare interference is output, thereby realizing efficient calibration of glare problem in the captured image at the software level and improving the image quality.

[0124] This application provides a schematic diagram of the structure of an image sensor pixel circuit, as shown in the embodiment. Figure 3a As shown:

[0125] Multiple photodiodes, specifically PD1, PD2, PD3, and PD4, are used for photoelectric conversion to generate photocurrent;

[0126] Multiple transistors, including TG1, TG2, TG3, TG4, TGS, TG6, RST1, SF1, and SET1;

[0127] TG1, TG2, TG3 and TG4 are connected to PD1-PD4 respectively, serving as the selection control unit for photodiodes; TGS and TG6 constitute the transmission control branch for pixel signals; RST1, SF1 and SET1 form the amplification and reset circuits related to signal reading and reset.

[0128] Among them, the source / drain of TG1-TG4 are connected to the corresponding PD1-PD4 respectively, and the drain / source are connected to the pixel signal transmission node; TGS and TG6 are used to construct the transmission path of pixel signal, RST1 is used to perform a reset operation on node FD1, and SF1 and SET1 form a source follower amplification structure to realize the initial amplification output of the signal.

[0129] FD1, as a temporary storage and relay node for pixel signals, is connected between TGS, RST1, and SF1;

[0130] An adaptive calibration ADC module, whose signal input terminal is connected to the output terminal of the source follower amplifier structure, is used to perform adaptive calibration and analog-to-digital conversion on the pixel analog signal and output a digital signal.

[0131] The ISP receives the digital signal output by the adaptive calibration ADC module and performs image signal processing;

[0132] The MIPI module connects to the ISP and is used to encode and transmit processed image signals according to the MIPI protocol.

[0133] The power supply VDD provides the operating voltage for the transistors, photodiodes, and other circuit units.

[0134] The multiple photodiodes PD1-PD4 and their corresponding selection transistors TG1-TG4 form a pixel signal acquisition, transmission, and processing link through TGS, TG6, FD1, and subsequent reset, amplification, ADC, and other circuits, realizing the conversion and transmission from optical signals to digital image signals.

[0135] In this pixel circuit, PD1, PD2, PD3, and PD4 are photodiodes that generate photogenerated carriers when illuminated, thus achieving photoelectric conversion. TG1-TG4 are selection transistors that control whether PD1-PD4 are connected to the pixel signal path. When TG1 is turned on, the photogenerated current generated by PD1 can be transmitted to subsequent circuits. Similarly, TG2-TG4 control PD2-PD4 respectively.

[0136] TGS and TG6 constitute the transmission control branch for pixel signals, used to regulate the transmission of signals from PD1 to PD4 to node FD1 after gating. RST1 is a reset transistor, which can reset the voltage of node FD1 to its initial level at the beginning of the pixel cycle and other times to eliminate residual signal interference. SF1 is a source follower transistor and SET1 is an enable transistor. Together with FD1, they form a source follower amplifier circuit to perform preliminary voltage follower amplification of the pixel signals on FD1, and output the analog pixel signal through the Vout terminal.

[0137] The Vout terminal is connected to an adaptive calibration ADC module. This ADC module has an adaptive calibration function, which can adaptively adjust and calibrate parameters such as amplitude and bias of the input analog pixel signal to improve the accuracy of analog-to-digital conversion and output a digital signal to the ISP. After performing image processing operations such as noise reduction and enhancement on the digital pixel signal, the ISP transmits it to the MIPI module. The MIPI module completes signal encoding and transmission according to the MIPI protocol, realizing system-level interaction of image data.

[0138] This application provides a schematic diagram of the structure of an integrating analog-to-digital converter circuit, as shown in the embodiment. Figure 3b As shown,

[0139] The switching unit includes S1 and S2; S1 is used to switch the connection state between the pixel analog input signal and the sample-and-hold module SH1 to realize signal sampling or path switching; S2 is used to select the reference voltage Vref1 or Vref to provide a reference for the circuit.

[0140] The sample-and-hold module SH1, connected to S1, is used to perform sampling and holding operations on the input pixel analog signal and temporarily store the signal voltage.

[0141] The integrator has its signal input terminal connected to the common node of the switching unit and SH1 via a resistor R, and is used to perform integration operations on the input analog signal; the integrator is also connected to a capacitor C to form an integration feedback path and realize the voltage integration function.

[0142] Comparator 1 receives the output signal Uc of the integrator at its non-inverting input terminal. It is used to compare the integrated analog signal with the internal threshold or feedback signal and output a comparison level, that is, the output signal of comparator 1 is Uo.

[0143] The voltage control module has its signal input terminal connected to the output terminal of the glare parameter calculation module. It is used to perform voltage conversion and control the switching unit based on the output of the glare parameter calculation module to achieve adaptive calibration.

[0144] The glare parameter calculation module receives feedback signals from the counter, calculates glare calibration parameters related to the image, and provides control basis for the voltage control module.

[0145] The clock module provides clock signals for the 10-bit binary counter and circuit timing.

[0146] A 10-bit binary counter receives the comparison level signal output by the comparator 1 and the clock signal from the clock module, counts the cycles corresponding to the comparison result, and outputs a 10-bit binary digital signal.

[0147] The MIPI encoding module receives the digital signal output from the 10-bit binary counter, encodes it according to the MIPI protocol, and then outputs it.

[0148] The switching unit, sample and hold module SH1, integrator, comparator 1, voltage control module, glare parameter calculation module, clock module, 10-bit binary counter, and MIPI encoding module form a complete signal processing link from analog pixel signal input, through adaptive calibration, analog-to-digital conversion, encoding, and transmission.

[0149] S1 is a signal selection switch that switches between SH1 and the integrator. S2 is a reference voltage selection switch that can be connected to either Vref1 or Vref to provide a reference voltage for the integrator, comparator, etc., to meet the calibration requirements of different signal ranges.

[0150] SH1 is a sample-and-hold circuit that samples and holds the voltage of the analog signal selected by S1. One end of SH1 is connected to the voltage control module. Resistor R, integrator, and capacitor C form an integration circuit. The input signal flows into the integrator through R, and the integrator outputs voltage Uc, realizing the integration conversion of the analog signal.

[0151] Comparator 1 compares the integrator output Uc with the internal reference level, and outputs the comparison result U (high / low level). O The voltage conversion module receives calibration information from the glare parameter calculation module, performs voltage conversion based on this information, and simultaneously outputs switching control signals to S1 and S2, reset signals, enable signals, or integral parameter adjustment signals to the integrator to adjust the circuit operating parameters and achieve adaptive calibration.

[0152] The clock module provides a clock signal for the 10-bit binary counter and timing logic. The counter counts the cycles corresponding to the comparator output level, generating a 10-bit binary digital signal. This digital signal is transmitted to the MIPI encoding module, encoded according to the MIPI protocol, and then output, achieving adaptation of the ADC digital signal to the system transmission interface. The glare parameter calculation module extracts and calculates characteristic parameters related to image glare from the circuit feedback signal, providing a basis for the adaptive calibration strategy of the voltage control module and optimizing the ADC conversion accuracy under different lighting and glare scenarios.

[0153] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0154] Figure 4 This is a block diagram of an image processing apparatus provided in an embodiment of this application. The apparatus 400 includes:

[0155] The acquisition module 410 is used to acquire a first image based on a first exposure parameter, wherein the first exposure parameter includes a first exposure time and a first conversion gain;

[0156] The first determining module 420 is used to determine the target pixel points in the first image that need to be glare calibration.

[0157] The second determining module 430 is used to determine the second conversion gain of the target pixel based on the first pixel value of the target pixel and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixel in the first image.

[0158] The reading module 440 is used to read the second pixel value corresponding to the target pixel based on the second conversion gain to obtain a second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

[0159] In one possible embodiment, the device 400 further includes:

[0160] The imaging module is used to capture multiple point light sources from different angles to obtain calibration images with preset reference conversion gain and reference exposure time;

[0161] The identification module is used to identify the pixel coordinates affected by glare in each color channel of the calibration image and record the corresponding glare calibration parameters.

[0162] A module is established to create a glare parameter database based on the pixel coordinates and glare calibration parameters.

[0163] In one possible embodiment, the multiple point light sources at different angles include at least one of the following:

[0164] The image sensor has a central point light source at the center of its field of view, a first edge point light source at the upper left corner of the image plane, a second edge point light source at the upper right corner, a third edge point light source at the lower left corner, and a fourth edge point light source at the lower right corner; the central point light source, the first edge point light source, the second edge point light source, the third edge point light source, and the fourth edge point light source have the same illumination intensity.

[0165] In one possible embodiment, the module is specifically used for:

[0166] Sub-databases are established for the red, green, and blue channels respectively. Each sub-database stores the pixel coordinates and glare calibration parameters of the corresponding color channel that are affected by glare.

[0167] In one possible embodiment, the second determining module 430 is specifically used for:

[0168] From the glare parameter database, determine the glare calibration parameters corresponding to the first pixel value of the target pixel; the glare parameter database includes: multiple sets of corresponding pixel coordinates and glare calibration parameters affected by glare obtained based on the reference conversion gain and the reference exposure time;

[0169] Determine the gain ratio between the first conversion gain and the reference conversion gain;

[0170] Determine the exposure ratio between the first exposure time and the reference exposure time;

[0171] The product of the gain ratio, the exposure ratio, and the glare calibration parameter is determined as the dynamic compensation value;

[0172] The second conversion gain is determined based on the dynamic compensation value.

[0173] In one possible embodiment, the reading module 440 is specifically used for:

[0174] The second conversion gain is converted into a compensation voltage value;

[0175] Based on the compensation voltage value and the reference voltage value corresponding to the first pixel value read out, the target reference voltage is obtained;

[0176] The pixel voltage of the target pixel is converted from analog to digital based on the target reference voltage to obtain the second pixel value.

[0177] In one possible embodiment, the reading module 440 is specifically used for:

[0178] The pixel voltage is input to the integrator of the integrating analog-to-digital converter circuit;

[0179] The target reference voltage is input to the reference voltage terminal of the integrating analog-to-digital converter circuit;

[0180] The comparator compares the output voltage of the integrator with the threshold voltage and outputs a comparison signal to the binary counter.

[0181] The second pixel value is generated based on the digital signal from the binary counter.

[0182] In one possible embodiment, the integrating analog-to-digital converter circuit includes:

[0183] The glare parameter calculation module is used to determine, from the glare parameter database, a glare calibration parameter corresponding to the first pixel value of the target pixel; and to determine, based on the first pixel value of the target pixel and the glare calibration parameter, a second conversion gain of the target pixel.

[0184] A voltage control module, connected to the glare parameter calculation module, is used to convert the second conversion gain into a compensation voltage value; and to obtain a target reference voltage based on the compensation voltage value and the reference voltage value corresponding to the first pixel value when it is read out.

[0185] The analog-to-digital conversion module is connected to the glare parameter calculation module and the voltage control module, respectively, and is used to perform analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value.

[0186] In embodiments of this application, a first image is acquired based on first exposure parameters, including a first exposure time and a first conversion gain. The first conversion gain determines the amplification factor by which the sensor converts light signals into electrical signals, and the first exposure time controls the duration of light incidence. The first exposure time and the first conversion gain jointly affect the intensity of glare and real-scene signals in the image. Target pixels in the first image that require glare calibration are identified, focusing on areas in the first image significantly affected by glare, providing targets for subsequent targeted processing. A second conversion gain for the target pixels is determined based on the first pixel value and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixels in the first image; since glare and real-scene signals have the same response characteristics on the sensor, both changing with the conversion gain, a second conversion gain that effectively suppresses glare can be determined by combining the first pixel value with pre-established glare calibration parameters.

[0187] The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain. The second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time. The target pixel is resampled by the second conversion gain, so that the glare signal is effectively suppressed while the real scene signal is preserved. Finally, the second image with glare interference is output, thereby realizing efficient calibration of glare problem in the captured image at the software level and improving the image quality.

[0188] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0189] The image processing apparatus in this application embodiment can be a device with a motion system. This motion system can be an Android motion system, an iOS motion system, or other possible motion systems; this application embodiment does not specifically limit the specific motion system.

[0190] The image processing apparatus provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0191] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 510, including a processor 511, a memory 512, and a program or instructions stored in the memory 512 and executable on the processor 511. When the program or instructions are executed by the processor 511, they implement the various steps of any of the above-described image processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0192] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0193] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0194] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0195] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0196] The processor 610 is used to acquire a first image based on a first exposure parameter, wherein the first exposure parameter includes a first exposure time and a first conversion gain.

[0197] The processor 610 is also configured to determine the target pixels in the first image that require glare calibration;

[0198] Processor 610 is further configured to determine a second conversion gain of the target pixel based on a first pixel value and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixel in the first image;

[0199] The processor 610 is further configured to read the second pixel value corresponding to the target pixel based on the second conversion gain to obtain a second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

[0200] Optionally, the input unit 604 is also used to capture multiple point light sources at different angles with preset reference conversion gain and reference exposure time to obtain calibration images;

[0201] The processor 610 is also used to identify the pixel coordinates affected by glare in each color channel of the calibration image and record the corresponding glare calibration parameters;

[0202] The processor 610 is also used to establish a glare parameter database based on the pixel coordinates and glare calibration parameters.

[0203] Optionally, the multiple point light sources at different angles include at least one of the following:

[0204] The image sensor has a central point light source at the center of its field of view, a first edge point light source at the upper left corner of the image plane, a second edge point light source at the upper right corner, a third edge point light source at the lower left corner, and a fourth edge point light source at the lower right corner; the central point light source, the first edge point light source, the second edge point light source, the third edge point light source, and the fourth edge point light source have the same illumination intensity.

[0205] Optionally, the processor 610 is also configured to establish sub-databases for the red channel, green channel and blue channel respectively, each of the sub-databases storing the pixel coordinates affected by glare and glare calibration parameters for the corresponding color channel.

[0206] Optionally, the processor 610 is further configured to determine, from a glare parameter database, a glare calibration parameter corresponding to a first pixel value of the target pixel; the glare parameter database includes: multiple sets of corresponding pixel coordinates affected by glare and glare calibration parameters obtained based on a reference conversion gain and a reference exposure time;

[0207] The processor 610 is also configured to determine the gain ratio between the first conversion gain and the reference conversion gain;

[0208] The processor 610 is also configured to determine the exposure ratio between the first exposure time and the reference exposure time;

[0209] The processor 610 is further configured to determine the product of the gain ratio, the exposure ratio, and the glare calibration parameter as a dynamic compensation value;

[0210] The processor 610 is also configured to determine the second conversion gain based on the dynamic compensation value.

[0211] Optionally, the processor 610 is also configured to convert the second conversion gain into a compensation voltage value;

[0212] The processor 610 is further configured to obtain a target reference voltage based on the compensation voltage value and the reference voltage value corresponding to the first pixel value when it is read out;

[0213] The processor 610 is further configured to perform analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value.

[0214] Optionally, the processor 610 is also used to control the integrator of the pixel voltage input integrating analog-to-digital converter circuit;

[0215] The processor 610 is also configured to control the input of the target reference voltage to the reference voltage terminal of the integrating analog-to-digital converter circuit;

[0216] The processor 610 is also used to control the comparator to compare the output voltage of the integrator with the threshold voltage and output a comparison signal to the binary counter;

[0217] The processor 610 is also used to control the digital signal of the binary counter to generate the second pixel value.

[0218] Optionally, the integrating analog-to-digital converter circuit includes:

[0219] Glare parameter calculation module, wherein the processor 610 is configured to control the glare parameter calculation module to determine, from the glare parameter database, a glare calibration parameter corresponding to a first pixel value of the target pixel; and to determine a second conversion gain of the target pixel based on the first pixel value of the target pixel and the glare calibration parameter.

[0220] A voltage control module is connected to the glare parameter calculation module. The processor 610 is used to control the voltage control module to convert the second conversion gain into a compensation voltage value; and to obtain a target reference voltage based on the compensation voltage value and the reference voltage value corresponding to the first pixel value when it is read out.

[0221] The analog-to-digital conversion module is connected to the glare parameter calculation module and the voltage control module respectively. The processor 610 is used to control the analog-to-digital conversion module to perform analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value.

[0222] In embodiments of this application, a first image is acquired based on first exposure parameters, including a first exposure time and a first conversion gain. The first conversion gain determines the amplification factor by which the sensor converts light signals into electrical signals, and the first exposure time controls the duration of light incidence. The first exposure time and the first conversion gain jointly affect the intensity of glare and real-scene signals in the image. Target pixels in the first image that require glare calibration are identified, focusing on areas in the first image significantly affected by glare, providing targets for subsequent targeted processing. A second conversion gain for the target pixels is determined based on the first pixel value and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixels in the first image; since glare and real-scene signals have the same response characteristics on the sensor, both changing with the conversion gain, a second conversion gain that effectively suppresses glare can be determined by combining the first pixel value with pre-established glare calibration parameters.

[0223] The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain. The second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time. The target pixel is resampled by the second conversion gain, so that the glare signal is effectively suppressed while the real scene signal is preserved. Finally, the second image with glare interference is output, thereby realizing efficient calibration of glare problem in the captured image at the software level and improving the image quality.

[0224] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or video images obtained by an image capture device (such as a camera) in video image capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 609 can be used to store software programs and various data, including but not limited to applications and motion systems. Processor 610 can integrate an application processor and a modem processor. The application processor mainly handles the action system, user page, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 610.

[0225] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or it may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0226] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0227] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0228] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0229] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0230] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0231] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0232] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0233] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0234] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: A first image is acquired based on a first exposure parameter, wherein the first exposure parameter includes a first exposure time and a first conversion gain; Identify the target pixels in the first image that require glare calibration; The second conversion gain of the target pixel is determined based on the first pixel value of the target pixel and the glare calibration parameters; wherein, the first pixel value is the pixel value of the target pixel in the first image; The second image is obtained by reading the second pixel value corresponding to the target pixel based on the second conversion gain; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

2. The method according to claim 1, characterized in that, Before determining the second conversion gain of the target pixel based on the first pixel value and glare calibration parameters of the target pixel, the method further includes: Calibration images are obtained by capturing point light sources from multiple angles using preset reference conversion gain and reference exposure time; Identify the pixel coordinates affected by glare in each color channel of the calibration image and record the corresponding glare calibration parameters; A glare parameter database is established based on the pixel coordinates and the glare calibration parameters.

3. The method according to claim 2, characterized in that, The plurality of point light sources at different angles include at least one of the following: The image sensor has a central point light source at the center of its field of view, a first edge point light source at the upper left corner of the image plane, a second edge point light source at the upper right corner, a third edge point light source at the lower left corner, and a fourth edge point light source at the lower right corner; the central point light source, the first edge point light source, the second edge point light source, the third edge point light source, and the fourth edge point light source have the same illumination intensity.

4. The method according to claim 2, characterized in that, The step of establishing a glare parameter database based on the pixel coordinates and the glare calibration parameters includes: Sub-databases are established for the red, green, and blue channels respectively. Each sub-database stores the pixel coordinates and glare calibration parameters of the corresponding color channel that are affected by glare.

5. The method according to claim 1, characterized in that, The step of determining the second conversion gain of the target pixel based on the first pixel value and glare calibration parameters includes: From the glare parameter database, determine the glare calibration parameters corresponding to the first pixel value of the target pixel; the glare parameter database includes: multiple sets of corresponding pixel coordinates and glare calibration parameters affected by glare obtained based on the reference conversion gain and the reference exposure time; Determine the gain ratio between the first conversion gain and the reference conversion gain; Determine the exposure ratio between the first exposure time and the reference exposure time; The product of the gain ratio, the exposure ratio, and the glare calibration parameter is determined as the dynamic compensation value; The second conversion gain is determined based on the dynamic compensation value.

6. The method according to claim 1, characterized in that, The step of reading the second pixel value corresponding to the target pixel based on the second conversion gain includes: The second conversion gain is converted into a compensation voltage value; Based on the compensation voltage value and the reference voltage value corresponding to the first pixel value read out, the target reference voltage is obtained; The pixel voltage of the target pixel is converted from analog to digital based on the target reference voltage to obtain the second pixel value.

7. The method according to claim 6, characterized in that, The step of performing analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value includes: The pixel voltage is input to the integrator of the integrating analog-to-digital converter circuit; The target reference voltage is input to the reference voltage terminal of the integrating analog-to-digital converter circuit; The comparator compares the output voltage of the integrator with the threshold voltage and outputs a comparison signal to the binary counter. The second pixel value is generated based on the digital signal from the binary counter.

8. The method according to claim 7, characterized in that, The integrating analog-to-digital converter circuit includes: The glare parameter calculation module is used to determine, from the glare parameter database, a glare calibration parameter corresponding to the first pixel value of the target pixel; and to determine, based on the first pixel value of the target pixel and the glare calibration parameter, a second conversion gain of the target pixel. A voltage control module, connected to the glare parameter calculation module, is used to convert the second conversion gain into a compensation voltage value; and to obtain a target reference voltage based on the compensation voltage value and the reference voltage value corresponding to the first pixel value when it is read out. The analog-to-digital conversion module is connected to the glare parameter calculation module and the voltage control module, respectively, and is used to perform analog-to-digital conversion on the pixel voltage of the target pixel based on the target reference voltage to obtain the second pixel value.

9. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first image based on first exposure parameters, wherein the first exposure parameters include a first exposure time and a first conversion gain; The first determining module is used to determine the target pixel points in the first image that need to be glare calibration. The second determining module is used to determine the second conversion gain of the target pixel based on the first pixel value of the target pixel and glare calibration parameters; wherein the first pixel value is the pixel value of the target pixel in the first image; The reading module is used to read the second pixel value corresponding to the target pixel based on the second conversion gain to obtain a second image; wherein the second conversion gain is different from the first conversion gain, and the exposure time corresponding to the second image is the first exposure time.

10. The apparatus according to claim 9, characterized in that, The device further includes: The imaging module is used to capture multiple point light sources from different angles to obtain calibration images with preset reference conversion gain and reference exposure time; The identification module is used to identify the pixel coordinates affected by glare in each color channel of the calibration image and record the corresponding glare calibration parameters. A module is established to create a glare parameter database based on the pixel coordinates and glare calibration parameters.

Images