Image processing method, system and device and storage medium
By acquiring and processing images in real time in the image acquisition device, dynamically adjusting the bright color uniformity intensity, the parasitic light sensitivity problem in the global shutter mode is solved, the image quality is improved, suitable for various scenarios without changing the hardware structure.
Patent Information
- Application Number
- CN202411358005.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-12-03
AI Technical Summary
The global shutter method has parasitic light sensitivity problems in the image acquisition device, which leads to image distortion and artifacts, affecting image quality.
Through the image processing method, images are collected in real time and divided into reference areas and influence areas, and white balance statistics are counted, parasitic light sensitivity influence is judged, bright chrominance uniformity intensity is dynamically adjusted, image processing is optimized using white balance coefficient to reduce parasitic light influence.
Effectively reduce the impact of parasitic light sensitivity on images, improve image quality, adapt to various shooting scenes, and does not require changes to the hardware structure, which is low in cost and good universality.
Smart Images

Figure CN119071648A_ABST
Abstract
Description
Background Art
[0002] Image acquisition devices using CMOS image sensors have two shutter modes: one is a rolling shutter that exposes the image sensor row by row or column by column, and the other is a global shutter that exposes all pixels on the image sensor at the same time. The global shutter has the advantages of avoiding image distortion, high speed and low noise, but also has the disadvantages of high cost, limited maximum frame rate and parasitic light interference.
[0003] For the global shutter method, it will be affected by parasitic light sensitivity (PLS). The main reason for the parasitic light sensitivity is that the microlens cannot focus all the light on the photosensitive area, resulting in a small number of photogenerated electrons being generated and collected in the storage capacitor of the global exposure pixel. This part of the photogenerated electrons is not generated by the normal photosensitivity process, but is caused by the scattering, reflection or refraction of light, so it is called "parasitic light". When the parasitic light sensitivity is large enough, the parasitic light generated in the current frame readout stage may have an impact on the next frame of the image, such as edge shadows, resulting in unnecessary residues or artifacts in the image. Summary of the invention
[0004] In view of the problems in the prior art, the purpose of the present application is to provide an image processing method, system, device and storage medium, which can reduce the impact of parasitic light sensitivity through software algorithm processing.
[0005] The present application provides an image processing method, comprising the following steps:
[0006] Acquire the image to be processed taken by the image acquisition device in real time, and count the first white balance statistical value of the reference area and the second white balance statistical value of the affected area in the image to be processed;
[0007] Determine whether parasitic light sensitivity influence occurs currently;
[0008] When the parasitic light sensitivity does not affect the image processing, the brightness and color uniformity adjustment function of the image processor is kept turned off, and the white balance coefficient is updated according to the first white balance statistical value and the second white balance statistical value, wherein the white balance coefficient is used to represent the relative relationship between the first white balance statistical value and the second white balance statistical value when the parasitic light sensitivity does not affect the image processing;
[0009] When the parasitic light sensitivity occurs, determining an allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient, determining a current uniformity adjustment strength according to a comparison result between the second white balance statistics and the allowable range of white balance statistics, enabling a brightness and chromaticity uniformity adjustment function of the image processor, and setting the strength of the brightness and chromaticity uniformity adjustment function according to the current uniformity adjustment strength;
[0010] An image is obtained after the image processor processes the image to be processed.
[0011] By adopting the image processing method of the present application, after obtaining the image to be processed, the image is divided into a reference area and an influence area, and the first white balance statistical value of the reference area and the second white balance statistical value of the influence area in the image to be processed are respectively counted to determine whether the parasitic light sensitivity influence occurs. When the parasitic light sensitivity influence does not occur, the brightness and chromaticity uniformity adjustment is not performed, and the white balance coefficient is updated. If the parasitic light sensitivity influence occurs, the brightness and chromaticity uniformity adjustment function is turned on to process the shadow situation in the influence area. At this time, it is necessary to combine the white balance statistical values of the reference area and the influence area and the latest updated white balance coefficient to control the intensity of the brightness and chromaticity uniformity adjustment function, so as to realize dynamic adjustment of the brightness and chromaticity uniformity intensity according to the severity of the shadow caused by the parasitic light sensitivity of the image captured by the image acquisition device, so as to realize dynamic adjustment of the brightness and chromaticity uniformity intensity when shooting in actual scenes, and effectively adapt to various shooting scenes. The method optimizes the image processing process from the software level between the time when the image acquisition device completes image capture and the time when the image processor processes the image, and obtains the processed image with reduced parasitic light sensitivity after brightness and chromaticity uniformity adjustment from the image processor. There is no need to change the hardware structure of the image acquisition device and the image processor, and the optimization cost is low and the universality is good.
[0012] In some embodiments, the calibration parameters of the brightness and chromaticity uniformity adjustment function are set by the following steps:
[0013] Setting shooting parameters of the image acquisition device under the calibration environment, wherein the shutter value of the image acquisition device is set to a minimum value and the aperture value is set to a maximum value;
[0014] Acquire a calibration image captured by the image acquisition device under a calibration environment;
[0015] Turning on the brightness and color uniformity adjustment function of the image processor to obtain the image after the image processor processes the calibration image;
[0016] The mean values of each color channel in the image after the calibration image is processed by the image processor are counted, and the calibration parameters of the brightness and chromaticity uniformity adjustment function are set according to the mean values of each color channel, wherein the calibration parameters include the values of each color channel when the intensity of the brightness and chromaticity uniformity adjustment function is maximum.
[0017] In some embodiments, the determining whether a parasitic light sensitivity impact is currently occurring includes: determining whether a shutter value of the image acquisition device is less than a preset shutter threshold; if so, determining that a parasitic light sensitivity impact is currently occurring.
[0018] In some embodiments, the first white balance statistic is a ratio of a mean red component to a mean green component or a ratio of a mean blue component to a mean green component in the reference area;
[0019] The second white balance statistic is a ratio of a red component mean value to a green component mean value or a ratio of a blue component mean value to a green component mean value in the affected area.
[0020] In some embodiments, updating the white balance coefficient according to the first white balance statistic and the second white balance statistic comprises the following steps:
[0021] Taking the white balance coefficient at the previous moment as the first component, calculating the ratio of the second white balance statistical value to the first white balance statistical value as the second component;
[0022] The first component and the second component are weightedly summed to obtain the updated white balance coefficient.
[0023] In some embodiments, determining the allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient comprises the following steps:
[0024] Determine a reference value according to a product of the first white balance statistic and the white balance coefficient;
[0025] Adding a first preset value to the reference value to obtain an upper limit value of the allowable range of the white balance statistical value;
[0026] The second preset value is subtracted from the reference value to obtain a lower limit value of the allowable range of the white balance statistical value.
[0027] In some embodiments, determining the current uniformity adjustment strength according to the comparison result between the second white balance statistic and the allowable range of the white balance statistic includes the following steps:
[0028] comparing the second white balance statistical value with the upper limit value and the lower limit value of the allowable range of the white balance statistical value;
[0029] If the second white balance statistic is greater than or equal to the upper limit of the allowable range of the white balance statistic, and the uniformity adjustment strength at the previous moment is less than the preset maximum strength value, increasing the uniformity adjustment strength at the previous moment to obtain the current uniformity adjustment strength;
[0030] If the second white balance statistic is less than or equal to the lower limit of the allowable range of the white balance statistic, and the uniformity adjustment intensity at the previous moment is greater than the preset minimum intensity value, the uniformity adjustment intensity at the previous moment is reduced to obtain the current uniformity adjustment intensity.
[0031] The embodiment of the present application further provides an image processing system, which is applied to the image processing method, and the system includes:
[0032] A statistical module, used for collecting in real time the image to be processed taken by the image acquisition device, and counting the first white balance statistical value of the reference area and the second white balance statistical value of the affected area in the image to be processed;
[0033] a coefficient updating module, configured to update a white balance coefficient according to the first white balance statistical value and the second white balance statistical value when no parasitic light sensitivity influence occurs, wherein the white balance coefficient is used to represent a relative relationship between the first white balance statistical value and the second white balance statistical value when no parasitic light sensitivity influence occurs;
[0034] an intensity setting module, configured to determine an allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient when parasitic light sensitivity occurs, and determine a current uniformity adjustment intensity according to a comparison result between the second white balance statistics and the allowable range of white balance statistics;
[0035] The uniformity control module is used to keep the brightness and chromaticity uniformity adjustment function of the image processor turned off when there is no parasitic light sensitivity impact, and to turn on the brightness and chromaticity uniformity adjustment function of the image processor when there is parasitic light sensitivity impact, and to set the intensity of the brightness and chromaticity uniformity adjustment function according to the current uniformity adjustment intensity, and to obtain the image after the image processor processes the image to be processed.
[0036] By adopting the image processing system of the present application, the various steps of the above-mentioned image processing method can be implemented when the functions of each module are realized to obtain the technical effect of the above-mentioned image processing method, which will not be described in detail here.
[0037] The present application also provides an image processing device, including:
[0038] processor;
[0039] a memory storing executable instructions of the processor;
[0040] The processor is configured to perform the steps of the image processing method by executing the executable instructions.
[0041] By adopting the image processing device provided by the present application, the processor executes the image processing method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned image processing method.
[0042] The embodiment of the present application also provides a computer-readable storage medium for storing a program, which implements the steps of the image processing method when executed by a processor.
[0043] By adopting the computer-readable storage medium provided in the present application, the program stored therein implements the steps of the image processing method when being executed, thereby obtaining the beneficial effects of the above-mentioned image processing method. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.
[0045] Figure 1 is a flow chart of an image processing method according to an embodiment of the present application;
[0046] Figure 2 This is a flow chart of setting calibration parameters of the brightness and chromaticity uniformity adjustment function according to an embodiment of the present application;
[0047] Figure 3 is a schematic diagram of an image processing system according to an embodiment of the present application;
[0048] Figure 4 is a structural schematic diagram of an image processing device according to an embodiment of the present application;
[0049] Figure 5 It is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present application. DETAILED DESCRIPTION
[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted. Although "first" or "second" etc. are used in this specification to represent certain features, they are only used to represent the function and are not used as a limitation on the number and importance of specific features.
[0051] like Figure 1As shown, in one embodiment, the present application provides an image processing method, comprising the following steps:
[0052] S100: acquiring an image to be processed captured by an image acquisition device in real time, and counting a first white balance statistical value of a reference area and a second white balance statistical value of an affected area in the image to be processed;
[0053] The image acquisition device includes a CMOS sensor for imaging, and the raw image captured by the image acquisition device is acquired in real time as the image to be processed, and each pixel in the image to be processed is analyzed, and a first white balance statistical value of a reference area and a second white balance statistical value of an image area in the image to be processed are counted;
[0054] The image acquisition device is used, for example, in monitoring scenes to continuously capture images in real time, but the present application is not limited thereto;
[0055] The reference area refers to an area in the image to be processed that is less affected by the parasitic light sensitivity or is not affected by the parasitic light sensitivity, and the affected area refers to an area in the image to be processed that is greatly affected by the parasitic light sensitivity;
[0056] Determined by the lens characteristics of the image acquisition device, the portion of the upper edge region in the image to be processed is least affected by the parasitic light sensitivity, and the portion of the lower edge region is most affected by the parasitic light sensitivity. Optionally, the region of the first row number at the upper edge of the image to be processed is used as the reference region, and the region of the second row number at the lower edge of the image to be processed is used as the affected region. The values of the first row number and the second row number can be selected as needed, and the first row number can be the same as or different from the second row number. For example, the first row number and the second row number are set to 6, then the upper edge region of the image to be processed, which is 6 rows downward from the top edge, is used as the reference region, and the lower edge region of the image to be processed, which is 6 rows upward from the bottom edge, is used as the affected region;
[0057] S200: Determine whether parasitic light sensitivity influence occurs currently;
[0058] When the parasitic light sensitivity does not affect the image processor, S300: keep the brightness uniformity adjustment function of the image processor turned off, and update the white balance coefficient according to the first white balance statistical value and the second white balance statistical value, where the white balance coefficient is used to represent the relative relationship between the first white balance statistical value and the second white balance statistical value when the parasitic light sensitivity does not affect the image processor;
[0059] Image processor refers to ISP (Image Signal Processing) chip, which is used to process the output signal of the image sensor of the image acquisition device. For example, the image processor can enhance the image contrast and clarity, perform white balance processing, reduce noise, etc., and can also be used to adjust the brightness and color uniformity of the image. The image acquisition device and the image processor can be integrated in one shooting device or can be two separate parts.
[0060] The image processor provides an interface for setting parameters of the brightness and chromaticity uniformity adjustment function. Users can pass parameter setting files to the image processor or set parameters in the user setting UI page of the image processor. The parameters of the brightness and chromaticity uniformity adjustment function include, for example, the switch of the brightness and chromaticity uniformity adjustment function, the intensity of the function implementation, and the calibration parameters when the function intensity is maximum. The brightness and chromaticity uniformity adjustment function will be described in detail below. The updated white balance coefficient is used to analyze and determine the intensity of the brightness and chromaticity uniformity adjustment function when parasitic light sensitivity is affected.
[0061] When the parasitic light sensitivity is affected, S410: determining the allowable range of the white balance statistic according to the first white balance statistic and the white balance coefficient, S420: determining the current uniformity adjustment strength according to the comparison result between the second white balance statistic and the allowable range of the white balance statistic, S430: starting the brightness and color uniformity adjustment function of the image processor, and setting the strength of the brightness and color uniformity adjustment function according to the current uniformity adjustment strength;
[0062] When the parasitic light sensitivity influence occurs, the updating of the white balance coefficient is stopped. Therefore, in step S410, the white balance statistical value allowable range is determined based on the current latest white balance coefficient (the white balance coefficient at the most recent moment when the parasitic light sensitivity influence does not occur);
[0063] S500: Obtaining an image after the image processor processes the image to be processed;
[0064] Specifically, a raw image captured by an image acquisition device is input into an image processor, and a processed image output by the image processor is obtained.
[0065] By adopting the image processing method of the present application, after the image to be processed is obtained through step S100, the image is divided into a reference area and an influence area, and the first white balance statistical value of the reference area in the image to be processed and the second white balance statistical value of the influence area are respectively counted. It is determined through step S200 whether the parasitic light sensitivity influence occurs. When the parasitic light sensitivity influence does not occur, the brightness and chromaticity uniformity adjustment is not performed through step S300, and the white balance coefficient is updated. If the parasitic light sensitivity influence occurs, it is necessary to enable the brightness and chromaticity uniformity adjustment function through steps S410 to S430 to process the shadow situation of the influence area. At this time, it is necessary to control the intensity of the brightness and chromaticity uniformity adjustment function in combination with the white balance statistical values of the reference area and the influence area and the latest updated white balance coefficient, so as to dynamically adjust the brightness and chromaticity uniformity intensity according to the severity of the shadow caused by the parasitic light sensitivity of the image captured by the image acquisition device, so as to realize the dynamic adjustment of the brightness and chromaticity uniformity intensity when shooting in actual scenes, and effectively adapt to various shooting scenes. The method optimizes the image processing process from the software level between the time when the image acquisition device completes image capture and the time when the image processor processes the image, and obtains the processed image with reduced parasitic light sensitivity after brightness and chromaticity uniformity adjustment from the image processor. There is no need to change the hardware structure of the image acquisition device and the image processor, and the optimization cost is low and the universality is good.
[0066] In this embodiment, the calibration parameters of the brightness uniformity adjustment function are set by the following steps. The following steps may be performed before the above step S100. Figure 2 As shown, setting the calibration parameters of the brightness and color uniformity adjustment function includes the following steps:
[0067] S010: Setting shooting parameters of the image acquisition device under the calibration environment, wherein the shutter value of the image acquisition device is set to the minimum value and the aperture value to the maximum value;
[0068] Set up a stable and uniform light source in the calibration environment, such as an integrating sphere, a uniform light box, etc., to ensure that the brightness and chromaticity of the light source remain unchanged. Use a center gray or all-white background plate, fix the image acquisition device on the bracket, ensure that the camera can clearly capture the background plate, and ensure that the light reflected from the background does not interfere with the brightness and chromaticity measurement of the image acquisition device. Keep the temperature, humidity and other conditions in the calibration environment stable to avoid the influence of external factors on the calibration results. Use calibration charts when shooting, such as the SineImage neutral gray test chart, all-white test chart or grayscale test chart, etc. These charts have known brightness and chromaticity values and can be used to calibrate the image acquisition device.
[0069] When setting shooting parameters, set the shutter value of the image acquisition device to the minimum value and the aperture value to the maximum value to simulate the shooting scene when the parasitic light sensitivity is most seriously affected. The shooting resolution can be set to the maximum resolution to improve the accuracy of calibration parameter calculation;
[0070] S020: Acquire a calibration image captured by an image acquisition device in a calibration environment;
[0071] The calibration image here is a raw image captured by the image acquisition device under the calibration environment based on the shooting parameters set in the previous step. When shooting, place the calibration card within the shooting field of view of the device to ensure that the card completely fills the shooting field of view.
[0072] S030: Turn on the brightness and color uniformity adjustment function of the image processor, and obtain the image processed by the image processor on the calibration image;
[0073] S040: Counting the mean values of each color channel in the image after the calibration image is processed by the image processor;
[0074] The entire image is divided into a plurality of small blocks (e.g., 1089 small blocks of 33×33, but the present application is not limited thereto), and the mean of each color channel in each small block is counted, and then the mean of each color channel of all small blocks is counted;
[0075] S050: setting calibration parameters of the brightness and chromaticity uniformity adjustment function according to the average value of each color channel, the calibration parameters including the value of each color channel when the intensity of the brightness and chromaticity uniformity adjustment function is maximum;
[0076] The type of color channel here is determined according to the type of color channel in the setting interface opened by the brightness and chromaticity uniformity adjustment function. For example, there are three color channels, namely, a red channel, a green channel and a blue channel. The calibration parameters are written into the image processor. When the image processor runs the brightness and chromaticity uniformity adjustment function to adjust the image to be processed, the image is processed based on the calibration parameters and the current real-time intensity. The red channel, the green channel and the blue channel of the image to be processed are respectively processed for uniformity, and the red pixel value, the green pixel value and the blue pixel value of each pixel point after the uniformity processing are obtained, so as to realize the chromaticity adjustment of the image to be processed, and the brightness adjustment of the image to be processed is realized through the chromaticity adjustment.
[0077] The brightness and chromaticity uniformity adjustment function is implemented, for example, by the LSP (Lens Shading Correction) algorithm in the ISP chip. The ISP chip provides an interface for setting the calibration parameters and intensity of the brightness and chromaticity uniformity adjustment function. The calibration parameters are pre-calibrated using the above steps S010 to S050, and the intensity is adjusted dynamically in real time using the above step S430. Taking the grid correction method as an example, the image to be processed is divided into multiple grids, and the processing coefficients of each color channel at each grid intersection are calculated. The processing coefficients of other pixel points are obtained by interpolation, and the pixel values of each pixel point are multiplied by the corresponding processing coefficients to obtain the processed image. The value of the calibration parameter (the value of each color channel when the intensity is maximum) determines the upper limit of the processing coefficient of each color channel when the intensity is maximum. The processing coefficients of the pixel values of each actual intersection are obtained by the intensity of the current brightness and chromaticity uniformity adjustment function, the calibration parameters, and the pixel distribution of the image itself.
[0078] By pre-calibrating the upper limit of the processing coefficient when the intensity is maximum when the parasitic light sensitivity is most seriously affected, and dynamically adjusting the adjustment intensity according to the parasitic light sensitivity in the actual shooting environment, and then dynamically adjusting the actual value of the processing coefficient, the adjustment effect of the brightness and color uniformity adjustment function is more in line with the need to eliminate the influence of parasitic light sensitivity. The intensity of the brightness and color uniformity adjustment function is represented by 100%, and is initially set to 0. When the intensity needs to be adjusted, the intensity is increased or decreased according to a preset step value, such as adjusting 5%, 10% each time, etc. The step value should not be set too large to avoid too abrupt changes in the output processed image.
[0079] In this embodiment, in step S200, determining whether parasitic light sensitivity influence currently occurs includes: determining whether a shutter value of the image acquisition device is less than a preset shutter threshold value, and if so, determining that parasitic light sensitivity influence currently occurs.
[0080] The preset shutter threshold here is a preset relatively small shutter value. When the shutter value of the image acquisition device is less than the preset shutter threshold, it is considered that the parasitic light sensitivity influence is currently occurring, and steps S410 to S430 are continued. If the shutter value of the image acquisition device is greater than or equal to the preset shutter threshold, it is considered that the parasitic light sensitivity influence is not currently occurring, and step S300 is continued.
[0081] The preset shutter threshold value may be an empirical value, for example, by observing the influence of parasitic light sensitivity in images captured by the image acquisition device over a period of time, when the parasitic light sensitivity influence is observed, the shutter value at that time is obtained as the preset shutter threshold value. In an actual application environment, the parasitic light sensitivity influence will increase with the increase of the brightness of the shooting environment, and decrease with the decrease of the brightness. Therefore, the preset shutter threshold value corresponds to a lower shutter value when the ambient brightness is relatively large.
[0082] In this embodiment, the first white balance statistic is a ratio of a mean red component to a mean green component or a ratio of a mean blue component to a mean green component in the reference area. The second white balance statistic is a ratio of a mean red component to a mean green component or a ratio of a mean blue component to a mean green component in the affected area. The first white balance statistic and the second white balance statistic are of the same type, and the following description is made by taking the example that both the first white balance statistic and the second white balance statistic are the ratio of the mean red component to the mean green component.
[0083] In the step S100, counting the first white balance statistics of the reference area and the second white balance statistics of the affected area in the image to be processed includes:
[0084] Divide the reference area into n uniform small blocks, calculate the ratio of the mean value of the red component to the mean value of the green component of each small block awg_rg1, awb_rg2, ... awb_rgn, and calculate the average ratio of all small blocks awb_rgsum_up = (awg_rg1 + awb_rg2 + ... + awb_rgn) / n as the first white balance statistical value;
[0085] The affected area is divided into n uniform small blocks, and the ratio of the mean red component to the mean green component of each small block is calculated respectively. The average ratio awb_rgsum_down of all small blocks is calculated as the second white balance statistic.
[0086] In this embodiment, in step S300, updating the white balance coefficient according to the first white balance statistic and the second white balance statistic includes the following steps:
[0087] Taking the white balance coefficient at the previous moment as the first component, calculating the ratio of the second white balance statistical value to the first white balance statistical value as the second component;
[0088] The first component and the second component are weightedly summed to obtain an updated white balance coefficient.
[0089] For example, the initial white balance coefficient is set to 1, the sum of the weight of the first component and the weight of the second component is 1, and here the weight of the first component is 0.99, and the weight of the second component is 0.01. The updated white balance coefficient ratio = ratio'×0.99+(awb_rgsum_down / awb_rgsum_up)×0.01, where ratio' represents the white balance coefficient at the previous moment. The weight value here is only an example and is not intended to limit the scope of protection of this application. The weight of the first component can also be set to 0.85, 0.9, 0.95, etc. as needed.
[0090] In this embodiment, in step S410, determining the allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient includes the following steps:
[0091] Determine a reference value according to a product of the first white balance statistic value and a white balance coefficient;
[0092] For example, the value of awb_rgsum_up×ratio is used as the reference value in the allowable range of white balance statistics;
[0093] Adding the first preset value to the reference value to obtain an upper limit value of the allowable range of the white balance statistical value;
[0094] Subtract the second preset value from the reference value to obtain a lower limit value of the allowable range of the white balance statistical value;
[0095] Therefore, the allowable range of the white balance statistical value is (reference value-second preset value, reference value+first preset value).
[0096] The specific values of the first preset value and the second preset value here can be set as needed, and can also be set by setting a percentage in combination with the reference value. For example, the first preset value = the first coefficient × the reference value, the second preset value = the second coefficient × the reference value, taking the first coefficient as 0.05 and the second coefficient as 0.05 as an example, the allowable range of the white balance statistical value is (reference value - 0.05 × reference value, reference value + 0.05 × reference value), that is, the lower limit value is equal to awb_rgsum_up × ratio × 0.95, and the upper limit value is equal to awb_rgsum_up × ratio × 1.05. The values of the first coefficient and the second coefficient here are only examples and are not intended to limit the scope of protection of this application.
[0097] In this embodiment, in step S420, determining the current uniformity adjustment strength according to the comparison result between the second white balance statistic value and the allowable range of the white balance statistic value includes the following steps:
[0098] comparing the second white balance statistical value with the upper limit value and the lower limit value of the allowable range of the white balance statistical value;
[0099] If the second white balance statistic value is greater than or equal to the upper limit of the allowable range of the white balance statistic value, and the uniformity adjustment strength at the previous moment is less than the preset maximum strength value, then the uniformity adjustment strength at the previous moment is increased to obtain the current uniformity adjustment strength;
[0100] If the second white balance statistic is less than or equal to the lower limit of the allowable range of white balance statistic, and the uniformity adjustment intensity at the previous moment is greater than the preset minimum intensity value, the uniformity adjustment intensity at the previous moment is reduced to obtain the current uniformity adjustment intensity.
[0101] The minimum intensity is, for example, 0, and the maximum intensity is, for example, 100%. By setting a suitable step value, each time the uniformity adjustment intensity is increased or decreased, the adjustment is performed based on the suitable step value to avoid abrupt changes in the images of the two frames before and after the adjustment.
[0102] During real-time image capture and processing, if the brightness and chromaticity uniformity adjustment function was turned on at the previous moment and there is no parasitic light sensitivity influence at the current moment (for example, the shutter opening at the current moment is only greater than or equal to the preset shutter threshold), the brightness and chromaticity uniformity adjustment function is turned off and the white balance coefficient is updated.
[0103] As mentioned above, in actual application environments, the influence of parasitic light sensitivity will increase with the increase of the brightness of the shooting environment, and decrease with the decrease of the brightness. When the image acquisition device is applied to continuous shooting in outdoor monitoring scenes, from morning to night, the brightness changes from weak to strong and then to weak, and the corresponding parasitic light sensitivity influence will also change from weak to strong and then to weak. Correspondingly, the intensity of the brightness and chromaticity uniformity adjustment function changes from low to high and then to low, until the parasitic light sensitivity influence completely disappears at night, and then the brightness and chromaticity uniformity adjustment function is turned off. This is only an example of an application scenario of the image acquisition device, and the image acquisition device can also be applied to other scenarios such as image capture of indoor scenes.
[0104] like Figure 3 As shown, the embodiment of the present application also provides an image processing system, which is applied to the image processing method, and the system includes:
[0105] The statistical module M100 is used to collect the image to be processed taken by the image acquisition device in real time, and to count the first white balance statistical value of the reference area and the second white balance statistical value of the affected area in the image to be processed;
[0106] A coefficient updating module M200, configured to update a white balance coefficient according to a first white balance statistic and a second white balance statistic when no parasitic light sensitivity influence occurs, wherein the white balance coefficient is used to represent a relative relationship between the first white balance statistic and the second white balance statistic when no parasitic light sensitivity influence occurs;
[0107] The intensity setting module M300 is used to determine the allowable range of the white balance statistic according to the first white balance statistic and the white balance coefficient when the parasitic light sensitivity is affected, and to determine the current uniformity adjustment intensity according to the comparison result between the second white balance statistic and the allowable range of the white balance statistic;
[0108] The uniformity control module M400 is used to keep the brightness and color uniformity adjustment function of the image processor turned off when there is no parasitic light sensitivity impact, and to turn on the brightness and color uniformity adjustment function of the image processor when there is parasitic light sensitivity impact, and to set the intensity of the brightness and color uniformity adjustment function according to the current uniformity adjustment intensity, and to obtain the image processed by the image processor on the image to be processed.
[0109] By adopting the image processing system of the present application, the functions of each module can be implemented to implement each step of the above-mentioned image processing method to obtain the technical effect of the above-mentioned image processing method, which is not described in detail here. The specific implementation method of the functions of each module in the image processing system can adopt the specific implementation method of each step in the above-mentioned image processing method, which is not described in detail here.
[0110] An embodiment of the present application also provides an image processing device, comprising a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to perform the steps of the image processing method by executing the executable instructions.
[0111] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0112] Refer to the following Figure 4 The electronic device 600 according to this embodiment of the present application is described. Figure 4 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0113] like Figure 4 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.
[0114] The storage unit stores a program code, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present application described in the above image processing method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .
[0115] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .
[0116] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.
[0117] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0118] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0119] By adopting the image processing device provided by the present application, the processor executes the image processing method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned image processing method.
[0120] The embodiment of the present application also provides a computer-readable storage medium for storing a program, which implements the steps of the image processing method when executed by a processor. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps of various exemplary implementations of the present application described in the above image processing method section of this specification.
[0121] refer to Figure 5 As shown, a program product 800 for implementing the above method according to an embodiment of the present application is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0122] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0123] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
[0124] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or cluster. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0125] By adopting the computer-readable storage medium provided in the present application, the program stored therein implements the steps of the image processing method when being executed, thereby obtaining the beneficial effects of the above-mentioned image processing method.
[0126] The above content is a further detailed description of the present application in combination with specific preferred implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application, which should be deemed to fall within the scope of protection of the present application.
Claims
1. An image processing method, characterized in that: The steps include: Acquire the image to be processed taken by the image acquisition device in real time, and count the first white balance statistical value of the reference area and the second white balance statistical value of the affected area in the image to be processed; Determine whether parasitic light sensitivity influence occurs currently; When the parasitic light sensitivity does not affect the image processing, the brightness and color uniformity adjustment function of the image processor is kept turned off, and the white balance coefficient is updated according to the first white balance statistical value and the second white balance statistical value, wherein the white balance coefficient is used to represent the relative relationship between the first white balance statistical value and the second white balance statistical value when the parasitic light sensitivity does not affect the image processing; When the parasitic light sensitivity occurs, determining an allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient, determining a current uniformity adjustment strength according to a comparison result between the second white balance statistics and the allowable range of white balance statistics, enabling a brightness and chromaticity uniformity adjustment function of the image processor, and setting the strength of the brightness and chromaticity uniformity adjustment function according to the current uniformity adjustment strength; An image is obtained after the image processor processes the image to be processed.
2. The image processing method according to claim 1, characterized in that: The method also includes setting the calibration parameters of the brightness and chromaticity uniformity adjustment function by adopting the following steps: Setting shooting parameters of the image acquisition device under the calibration environment, wherein the shutter value of the image acquisition device is set to a minimum value and the aperture value is set to a maximum value; Acquire a calibration image captured by the image acquisition device under a calibration environment; Turning on the brightness and color uniformity adjustment function of the image processor to obtain the image after the image processor processes the calibration image; The mean values of each color channel in the image after the calibration image is processed by the image processor are counted, and the calibration parameters of the brightness and chromaticity uniformity adjustment function are set according to the mean values of each color channel, wherein the calibration parameters include the values of each color channel when the intensity of the brightness and chromaticity uniformity adjustment function is maximum.
3. The image processing method according to claim 1, characterized in that: The determining whether parasitic light sensitivity influence currently occurs includes: determining whether a shutter value of the image acquisition device is less than a preset shutter threshold value; if so, determining that parasitic light sensitivity influence currently occurs.
4. The image processing method according to claim 1, characterized in that: The first white balance statistic is a ratio of a red component mean to a green component mean or a ratio of a blue component mean to a green component mean in the reference area; The second white balance statistic is a ratio of a red component mean value to a green component mean value or a ratio of a blue component mean value to a green component mean value in the affected area.
5. The image processing method according to claim 1, characterized in that: Updating a white balance coefficient according to the first white balance statistic and the second white balance statistic comprises the following steps: Taking the white balance coefficient at the previous moment as the first component, calculating the ratio of the second white balance statistical value to the first white balance statistical value as the second component; The first component and the second component are weightedly summed to obtain the updated white balance coefficient.
6. The image processing method according to claim 1, characterized in that: Determining an allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient comprises the following steps: Determine a reference value according to a product of the first white balance statistic and the white balance coefficient; Adding a first preset value to the reference value to obtain an upper limit value of the allowable range of the white balance statistical value; The second preset value is subtracted from the reference value to obtain a lower limit value of the allowable range of the white balance statistical value.
7. The image processing method according to claim 1, characterized in that: Determining the current uniformity adjustment intensity according to a comparison result between the second white balance statistic and the allowable range of the white balance statistic includes the following steps: comparing the second white balance statistical value with the upper limit value and the lower limit value of the allowable range of the white balance statistical value; If the second white balance statistic is greater than or equal to the upper limit of the allowable range of the white balance statistic, and the uniformity adjustment strength at the previous moment is less than the preset maximum strength value, increasing the uniformity adjustment strength at the previous moment to obtain the current uniformity adjustment strength; If the second white balance statistic is less than or equal to the lower limit of the allowable range of the white balance statistic, and the uniformity adjustment intensity at the previous moment is greater than the preset minimum intensity value, the uniformity adjustment intensity at the previous moment is reduced to obtain the current uniformity adjustment intensity.
8. An image processing system, characterized in that: The image processing method according to any one of claims 1 to 7, wherein the system comprises: A statistical module, used for collecting in real time the image to be processed taken by the image acquisition device, and counting the first white balance statistical value of the reference area and the second white balance statistical value of the affected area in the image to be processed; a coefficient updating module, configured to update a white balance coefficient according to the first white balance statistical value and the second white balance statistical value when no parasitic light sensitivity influence occurs, wherein the white balance coefficient is used to represent a relative relationship between the first white balance statistical value and the second white balance statistical value when no parasitic light sensitivity influence occurs; an intensity setting module, configured to determine an allowable range of white balance statistics according to the first white balance statistics and the white balance coefficient when parasitic light sensitivity occurs, and determine a current uniformity adjustment intensity according to a comparison result between the second white balance statistics and the allowable range of white balance statistics; The uniformity control module is used to keep the brightness and chromaticity uniformity adjustment function of the image processor turned off when there is no parasitic light sensitivity impact, and to turn on the brightness and chromaticity uniformity adjustment function of the image processor when there is parasitic light sensitivity impact, and to set the intensity of the brightness and chromaticity uniformity adjustment function according to the current uniformity adjustment intensity, and to obtain the image after the image processor processes the image to be processed.
9. An image processing device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to perform the steps of the image processing method according to any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.
