An image processing method and system, a color image acquisition method and a storage medium

Abstract

The application provides an image processing method and system, a color image acquisition method and a storage medium. The image processing method comprises the following steps: acquiring current color channel gray scale images of a calibration board collected by a black-and-white camera under red, green and blue light sources, respectively; calculating an initial gain coefficient of each channel gray scale image relative to a target gray scale; if there is discrete adjustment, calculating a residual error after the discrete adjustment as a digital gain based on the number of the discrete adjustment; otherwise, taking the initial gain coefficient as the digital gain, so that the black-and-white camera reaches white balance, and a corrected image of each channel is acquired. Based on the black-and-white camera and the red, green and blue light sources, the application realizes effective balance of colors in an early stage (a photoelectric conversion stage) of signal acquisition, thereby maximizing the use of the dynamic range of the image sensor, minimizing the rear-end noise, and improving the color restoration accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image processing, and particularly relates to an image processing method and system, a color image acquisition method and a storage medium. BACKGROUND

[0002] Ordinary color camera imaging is realized by using a Bayer Filter Pattern (Bayer array) filter to realize color image function, pixel values are calculated by using color interpolation algorithm of self pixel value and adjacent pixel value to obtain R, G, B values of each pixel, color saturation of the color camera using this method is low, and the color camera must use a filter, and the sensitivity and flexibility of the color camera to light are not strong. The current 3CCD camera basically overcomes these defects, but the manufacturing cost of 3CCD is high, which is not conducive to popularization and use.

[0003] In addition, there is a scheme of taking color images by a black and white camera: rotating emission of red, green and blue monochromatic light to irradiate the observed target, and obtaining color information of the observed object each time, and processing the color information into a color image by software. In this process, white balance is a key step to ensure the accuracy of color restoration, and the target is to make the standard white object photographed in the finally synthesized color image present a true and pure white color, and avoid color deviation.

[0004] The prior art usually adopts a pure digital gain method, that is, the average value proportion of the white area in the three-channel gray scale image is calculated, and the pixel values of each channel are uniformly scaled by multiplication.

[0005] However, this method has the following inherent defects:

[0006] 1. Signal-to-noise ratio deterioration: when the original signal of a certain channel (such as blue or red) is weak, excessive digital gain will simultaneously amplify the noise of the channel, resulting in a decline in the overall quality of the image.

[0007] 2. Loss of dynamic range: if the signal of a certain channel is too strong and saturated, the detailed information is permanently lost, and digital gain cannot recover.

[0008] 3. Low resource utilization: unable to fully utilize physical adjustment means such as light source and camera exposure, resulting in that the dynamic range of the camera is not optimally used.

[0009] Chinese patent CN 119155560 A discloses a method and system for acquiring color images based on black and white line-scan cameras under combined light field. The patent realizes the acquisition of color images through a black and white line-scan camera combined with three groups of three-color light sources; changing the illumination order and brightness of the three groups of light sources can combine three different working modes to provide a light field environment for image acquisition, which is suitable for different defect detection scenarios, and the technical problems solved by the present application are different.

[0010] Chinese patent CN 115460386 B discloses a method and system for acquiring color images using black and white cameras. The patent uses the gain adjustment of the black and white camera itself to make the system have lower requirements for light sources, and a fixed brightness illumination system can also realize the acquisition of color images of the inspected object by the black and white camera. Based on the image gray scale corresponding to the fixed illumination, the average of the multiple gain coefficients corresponding to multiple image gray scales is used as the digital gain of the black and white camera to realize white balance correction. The application still essentially belongs to the digital gain adjustment method and does not overcome the inherent defects of the pure digital gain method. In addition, the patent does not propose a white balance correction method for a non-fixed illumination system, and the technical problems and means are different from those of the present application.

[0011] Chinese patent CN 116346999 B discloses an image acquisition system and method. Based on a black and white camera and red, green and blue three basic color light sources, the patent acquires multiple groups of images under different light source states under an external signal to solve the problem of high-precision detection of defects, color types and product depth at the same time. The technical problems solved by the present application are different. SUMMARY

[0012] The present application provides an image processing method and system, a color image acquisition method and a storage medium, which at least solve one of the above technical problems.

[0013] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0014] An image processing method, comprising:

[0015] Acquiring current color channel gray scale images of a calibration board captured by a black and white camera under red, green and blue light sources, respectively, calculating initial gain coefficients of each channel gray scale image relative to target gray scale;

[0016] If the light source brightness and camera exposure time can be adjusted, then select any adjustment method as a first discrete adjustment, and then calculate the theoretical level of the first discrete adjustment of the channel based on the initial gain coefficient of the channel, quantize the theoretical level to an actual level, and realize the first discrete adjustment;

[0017] Based on the theoretical level and the actual level of the first-stage discrete adjustment, a first residual error of the first-stage discrete adjustment of the channel is calculated.

[0018] If the light source brightness and the camera exposure time can be adjusted, any adjustment mode is selected as the second-stage discrete adjustment, the theoretical level of the second-stage discrete adjustment of the channel is calculated based on the first residual error, the theoretical level is quantified as the actual level, and the second-stage discrete adjustment is realized; and a second residual error of the second-stage discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the second-stage discrete adjustment.

[0019] Based on the number of stages of the discrete adjustment, the initial gain coefficient or the residual error after the discrete adjustment is used as the digital gain, so that the black-and-white camera achieves white balance, and the corrected image of each channel is obtained; the number of stages of the discrete adjustment is one or two.

[0020] The residual error is the ratio of the theoretical level to the actual level; the residual error includes the first residual error and the second residual error.

[0021] Further, if the first-stage discrete adjustment is the light source brightness adjustment, the theoretical level of the light source brightness of the channel is calculated based on the initial gain coefficient and the initial light source brightness of any channel; the light source brightness actual level is adjusted based on the theoretical level and a plurality of actual levels of the light source brightness, each actual level corresponding to a different light source brightness, so that the absolute difference between the actual level and the theoretical level is minimized.

[0022] Further, if the first-stage discrete adjustment is the camera exposure time adjustment, the theoretical level of the exposure time of the channel is calculated based on the initial gain coefficient and the initial exposure time of any channel; the exposure time step value is adjusted based on the theoretical level and the minimum step unit of the camera exposure time, each actual level corresponding to a different exposure time, so that the absolute difference between the actual level and the theoretical level is minimized.

[0023] Further, the initial gain coefficient includes:

[0024] The ideal gain coefficient of the gray scale image of the channel with respect to the target gray scale is calculated based on the gray scale value of the image of any channel; any channel of the red, green, and blue channel gray scale images is selected as the reference channel, the ideal gain coefficient corresponding to the reference channel is normalized to 1, and the normalized initial gain coefficient of each channel is obtained.

[0025] Further, rounding or downward rounding is used to quantize the theoretical level to the actual level, so that the absolute difference between the actual level and the theoretical level is minimized.

[0026] Further, before the digital gain is obtained, the method further comprises: judging fitting accuracy of the linear model based on the corresponding linear model of the discrete adjustment, if the fitting accuracy is less than a preset threshold, then completing the digital gain adjustment based on the image after the discrete adjustment; otherwise, obtaining a new three-channel gray image according to the actual level determined by the discrete adjustment as the digital gain adjustment object.

[0027] Based on the same inventive concept, the application further provides an image processing method and system, comprising:

[0028] Three light sources of red, green and blue;

[0029] A black-and-white camera is used to collect channel gray images of a calibration board under the red, green and blue light sources.

[0030] A calculation module is used to calculate initial gain coefficients of the channel gray images relative to a target gray.

[0031] If either the light source brightness or the camera exposure time can be adjusted, then either adjustment mode is selected as a first discrete adjustment, and then the theoretical level of the first discrete adjustment of the channel is calculated based on the initial gain coefficient of the channel, and the theoretical level is quantified as an actual level; and the first residual error after the first discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the first discrete adjustment.

[0032] If both the light source brightness and the camera exposure time can be adjusted, then either adjustment mode is selected as a second discrete adjustment, and then the theoretical level of the second discrete adjustment of the channel is calculated based on the first residual error, and the theoretical level is quantified as an actual level; and the second residual error after the second discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the second discrete adjustment.

[0033] Based on the number of discrete adjustments, the initial gain coefficient or the residual error after the discrete adjustment is used as the digital gain, so that the black-and-white camera reaches white balance and the corrected images of the channels are obtained; and the number of discrete adjustments is one or two.

[0034] The residual error is the ratio of the theoretical level to the actual level; and the residual error includes the first residual error and the second residual error.

[0035] The method further comprises a control module that adjusts the exposure time of the black-and-white camera and / or the brightness of the three light sources of red, green and blue based on the actual level output by the calculation module, so as to realize the discrete adjustment.

[0036] On the other hand, the application further provides a color image collection method, comprising: obtaining the corrected images of the channels of a to-be-tested object when the black-and-white camera reaches white balance according to the image processing method described above, and synthesizing the three corrected images into a color image.

[0037] Based on the same inventive concept, the application further provides an electronic device comprising a processor and a memory, the memory being configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method as described above.

[0038] In another aspect, the application further provides a computer-readable storage medium having stored therein at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by a processor to implement the method as described above.

[0039] The application has the following advantages:

[0040] The embodiment provides an image processing system, which realizes effective color balance in an early stage (photoelectric conversion stage) of signal acquisition based on a black-and-white camera and red, green and blue light sources, so that the dynamic range of an image sensor is maximized, the noise of a rear end is minimized, and color restoration accuracy is improved.

[0041] The embodiment provides an image processing method, which realizes high-precision white balance processing based on an imaging system of a black-and-white camera and a color light source. In addition, the method can select different levels of adjustment strategies according to actual conditions, and is flexibly adapted to different hardware requirements to adapt to constraints of different levels of hardware (for example, whether the light source is adjustable, and whether the exposure time is independently controllable).

[0042] The embodiment provides an image processing method with a high signal-to-noise ratio, which adjusts the light source brightness and the exposure time as two physical quantities in priority, so that the signal intensity of three channels is basically balanced before entering the image sensor, the dynamic range of the camera is maximized, and significant noise introduced by single use of a large multiple digital gain is avoided from the root.

[0043] The embodiment provides an image processing method with high precision, which adopts a three-level serial adjustment strategy, decomposes total correction coefficients to three different levels, processes only the “residual error” of the previous level at each level, realizes smooth transition from coarse adjustment to fine adjustment, and finally guarantees the precision by the digital gain, so that high-precision imaging balance is achieved.

[0044] The embodiment provides an image processing method with strong adaptability and high flexibility, which fully considers the discreteness of hardware control. The unique degradable design is a key advantage, which can flexibly switch to a three-level, two-level or one-level adjustment mode according to the hardware support degree (for example, whether the light source is adjustable, and whether the exposure time is independently controllable), realizes “top-optimal, low-configurable” wide applicability, and greatly expands the application scenarios and product coverage of the method.

[0045] The embodiment provides an image processing method, and input signals of overstrong channels are preferentially reduced through physical adjustment, high light details are reserved, and image detail loss caused by saturation of single channel signals is effectively prevented.

[0046] The embodiment provides an image processing method, and the whole process does not need manual intervention, can automatically complete white balance calibration, is suitable for being integrated into an automatic visual system, and has intelligent and automatic characteristics. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a schematic diagram of an image processing system of the embodiment of the present application.

[0048] Figure 2 is a flowchart of an image processing method of the embodiment of the present application. DETAILED DESCRIPTION

[0049] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in combination with the drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiments of the present application, rather than all the embodiments of the present application.

[0050] As shown in Figure 1 the present application provides an image processing system, which comprises three light sources of red, green and blue, and a black-and-white camera.

[0051] In the embodiment, the three light sources of red, green and blue are not limited in specific forms, and can be RGB ring light or dome light source, or screen light source.

[0052] The black-and-white camera is used to collect a current color channel gray scale image of the calibration board under the three light sources of red, green and blue.

[0053] According to the current light source color, the black-and-white camera collects a current color channel gray scale image of the calibration board; if the current light source is blue, the black-and-white camera collects a blue channel gray scale image Ib of the calibration board; if the current light source is green, the black-and-white camera collects a green channel gray scale image Ig of the calibration board; and if the current light source is red, the black-and-white camera collects a red channel gray scale image Ir of the calibration board.

[0054] The calibration board has the characteristics of spectral neutrality (i.e. reflecting all colors of red, green and blue light in equal proportions) and constant reflectivity.

[0055] For example, an 18% gray card has a reflectivity of about 18%, which is close to the average brightness of scenes in nature; or a pure white calibration board (about 90% reflectivity); or a calibration board with other color combinations, but which has a certain proportion (generally more than 50%) of white or gray areas to ensure the accuracy of the calculation results.

[0056] The appropriate calibration board can be selected according to an actual scene, and the application does not limit this.

[0057] The image processing system further comprises a calculation module configured to calculate initial gain coefficients of the gray images of the channels relative to the target gray level.

[0058] According to the determined adjustment strategy, the adjustment value corresponding to the adjustment level is calculated, and specifically comprises:

[0059] If either the light source brightness or the camera exposure time can be adjusted, either adjustment mode is selected as the first discrete adjustment, and then the theoretical level of the first discrete adjustment of the channel is calculated based on the initial gain coefficient of the channel, and the theoretical level is quantified as an actual level; the first residual error after the first discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the first discrete adjustment.

[0060] If both the light source brightness and the camera exposure time can be adjusted, either adjustment mode is selected as the second discrete adjustment, and then the theoretical level of the second discrete adjustment of the channel is calculated based on the first residual error, and the theoretical level is quantified as an actual level; the second residual error after the second discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the second discrete adjustment.

[0061] Based on the number of discrete adjustments, the initial gain coefficient or the residual error after the discrete adjustment is used as the digital gain to make the black-and-white camera achieve white balance and obtain the corrected images of the channels; the number of discrete adjustments is one or two.

[0062] The residual error is the ratio of the theoretical level to the actual level; the residual error includes the first residual error and the second residual error.

[0063] In addition, if there is a discrete adjustment, the image processing system further comprises:

[0064] The control module adjusts the exposure time of the black-and-white camera and / or the brightness of the red, green and blue light sources based on the actual level output by the calculation module to realize the discrete adjustment.

[0065] Based on the above image processing system, the application provides an image processing method, as shown in Figure 2 The method comprises the following steps:

[0066] The current color channel gray images of the calibration board captured by the black-and-white camera under the red, green and blue light sources are obtained, and the initial gain coefficient of each channel gray image relative to the target gray level is calculated.

[0067] If either the light source brightness or the camera exposure time can be adjusted, either adjustment mode is selected as the first discrete adjustment, and then the theoretical level of the first discrete adjustment of the channel is calculated based on the initial gain coefficient of the channel, and the theoretical level is quantified as an actual level, and the first discrete adjustment is realized;

[0068] Based on the theoretical level and the actual level of the first discrete adjustment, the first residual error after the first discrete adjustment of the channel is calculated.

[0069] If the light source brightness and the camera exposure time can be adjusted, any adjustment method is selected as the second discrete adjustment, the theoretical level of the second discrete adjustment of the channel is calculated according to the first residual error, the theoretical level is quantified as the actual level, and the second discrete adjustment is realized.

[0070] Based on the theoretical level and the actual level of the second discrete adjustment, the second residual error after the second discrete adjustment of the channel is calculated.

[0071] Based on the number of discrete adjustments, the initial gain coefficient or the residual error after the discrete adjustment is used as the digital gain, so that the black and white camera achieves white balance, and the corrected image of each channel is obtained.

[0072] The residual error is the ratio of the theoretical level to the actual level; the residual error includes the first residual error and the second residual error.

[0073] The embodiment based on the black and white camera and the red, green and blue light sources proposes an image processing method to realize high-precision white balance processing; in addition, the method proposed in the embodiment can select different levels of adjustment strategies according to actual conditions, and is flexible and adaptive to different hardware requirements.

[0074] If the light source brightness and the camera exposure time in the current scene can be adjusted, the adjustment strategy is first discrete adjustment-second discrete adjustment-digital gain adjustment. If the first discrete adjustment is light source brightness adjustment, the second discrete adjustment is camera exposure time adjustment; or the first discrete adjustment is camera exposure time adjustment, and the second discrete adjustment is light source brightness adjustment.

[0075] If only the light source brightness or the camera exposure time in the current scene can be adjusted, the adjustment strategy is first discrete adjustment-digital gain adjustment.

[0076] If the discrete adjustment in the current scene cannot be adjusted, the digital gain adjustment is used as the white balance adjustment method to realize color restoration of the image.

[0077] Although only the digital gain adjustment method is used, the image color restoration accuracy cannot be improved, but the flexible adaptation strategy proposed in the embodiment can ensure that the image still realizes white balance and outputs high-quality images under the condition of limited hardware conditions.

[0078] White balance is an index for describing the accuracy of white color generated by mixing red, green and blue three primary colors in a display. Through it, a series of problems of color restoration and tone processing can be solved. It is to realize that the camera image can accurately reflect the color condition of the object being photographed.

[0079] Regardless of the choice of adjustment strategy, before adjustment, the current color channel gray scale images of the calibration board are acquired by the black and white camera under red, green and blue light sources respectively, and the initial gain coefficient of each channel gray scale image relative to the target gray scale is calculated.

[0080] According to the current light source color, the black and white camera acquires the current color channel gray scale image I of the calibration board; if the current light source is blue, the black and white camera acquires the blue channel gray scale image Ib of the calibration board; if the current light source is green, the black and white camera acquires the green channel gray scale image Ig of the calibration board; if the current light source is red, the black and white camera acquires the red channel gray scale image Ir of the calibration board.

[0081] Based on the gray scale value of any channel image, the ideal gain coefficient of the channel gray scale image relative to the target gray scale is calculated; any channel of the red, green and blue channel gray scale images is selected as the reference channel, and the ideal gain coefficient corresponding to the reference channel is normalized to 1 to obtain the initial gain coefficient of each channel (i.e. the initial gain coefficient of each channel gray scale image relative to the target gray scale).

[0082] Generally, a standard white board or a medium gray board is selected as the calibration board. Then the mean value of each channel gray scale image corresponding to the current calibration board is calculated.

[0083] Or other forms of calibration board are selected, and the mean value of the channel gray scale image corresponding to the white area in the calibration board is calculated as the mean value of the color channel gray scale image.

[0084] The mean value of the gray scale value corresponding to the blue channel gray scale image Ib is μ(Ib); the mean value of the gray scale value corresponding to the green channel gray scale image Ig is μ(Ig); and the mean value of the gray scale value corresponding to the red channel gray scale image Ir is μ(Ir).

[0085] Generally, the target gray scale can be set as a certain percentage of the full well capacity of the camera, such as 70% to 90%.

[0086] Based on the imaging principle of the black and white camera, each pixel of the image sensor of the black and white camera can receive all wavelengths of light and convert them into a single luminance value (gray scale value). Due to the quantum efficiency of the image sensor, the sensitivity to light of different wavelengths (colors) is different. Generally, the black and white camera has the highest response to green light and lower response to red and blue light. This is superimposed with the characteristics of the human eye and LED, resulting in a large difference in the intensity of the R, G and B color signals received by the camera. Therefore, in the white balance algorithm, the mean value of the gray scale value of the green channel gray scale image is often used as the target gray scale.

[0087] In this embodiment, the specific value of the target gray scale is not limited, and the setting method of the target gray scale is also not limited; the target gray scale G is set according to the actual scene demand and hardware condition. targetYes.

[0088] The average gray value of each channel gray image μ(Ib), μ(Ig), μ(Ir) and the target gray G are known target The ideal gain coefficient K of each channel is calculated, as follows:

[0089] Kr = G target / μ(Ir)

[0090] Kg = G target / μ(Ig)

[0091] Kb = G target / μ(Ib)

[0092] Wherein, Kr represents the ideal gain coefficient of the red channel gray image; Kg represents the ideal gain coefficient of the green channel gray image; Kb represents the ideal gain coefficient of the blue channel gray image.

[0093] Preferably, the ideal gain coefficient of the current channel can also be calculated with the maximum gray value in the current channel gray image.

[0094] Kr = G target / Ir_max

[0095] Kg = G target / Ig_max

[0096] Kb = G target / Ib_max

[0097] Wherein, Kr represents the ideal gain coefficient of the red channel gray image; Kg represents the ideal gain coefficient of the green channel gray image; Kb represents the ideal gain coefficient of the blue channel gray image; Ir_max represents the maximum gray value of the red channel gray image; Ig_max represents the maximum gray value of the green channel gray image; Ib_max represents the maximum gray value of the blue channel gray image.

[0098] The above method calculates the ideal gain coefficient of the channel gray image relative to the target gray with the maximum gray value. Directly taking a single maximum value point of any channel gray image is very sensitive to noise, so that the calculation result of the ideal gain coefficient is inaccurate.

[0099] Preferably, the pixel value of the brightest pixel point can be taken as 1% or 0.5% (or other proportion, set according to the actual situation), and the weighted calculation is performed with the average value of the gray image, so as to calculate the ideal gain coefficient of the channel gray image relative to the target gray with the weighted result. This can effectively avoid the misjudgment caused by a single noise point or highlight point.

[0100] Kr = G target / (a r × Ir_max + b r × μ(Ir))

[0101] Kg = G target / (a g × Ig_max + b g × μ(Ig))

[0102] Kb = G target / (a b × Ib_max + b b × μ(Ib))

[0103] wherein a r , b r are pixel weighting coefficients of the red channel gray scale image, a g , b g are pixel weighting coefficients of the green channel gray scale image, and a b , b b are pixel weighting coefficients of the blue channel gray scale image.

[0104] In consideration of the condition limitation of actual scene and the convenience of adjustment, in order to ensure that the gray scale value of each channel gray scale image in the finally realized white balance effect tends to the target gray scale, a reference channel is selected from the original data to overcome the response difference caused by different color wavelength light sources.

[0105] Generally, the channel with the strongest signal (the channel with the smallest ideal gain coefficient) is set as the reference channel; according to the aforementioned black-and-white camera imaging principle and spectral response relationship, generally, the green channel is selected as the reference channel.

[0106] The ideal gain coefficient corresponding to the green channel is normalized to 1 (Kg'=1), and the ideal gain coefficients of the blue and red channels are simultaneously normalized to obtain the initial gain coefficients Kb', Kr' of the normalized channels.

[0107] Preferably, the blue or red channel can also be set as the reference channel, the ideal gain coefficient corresponding to the blue or red channel is normalized to 1, and the ideal gain coefficients of the other channels are normalized to obtain the initial gain coefficients of the normalized channels.

[0108] According to the aforementioned adjustment strategy, if the current scene does not support discrete adjustment, the initial gain coefficient is used as the digital gain to realize a single-stage adjustment mode. The channel image I' finally realized by white balance correction is:

[0109] Ir'=Ir×Kr'; Ig'=Ig×Kg'; Ib'=Ib×Kb'

[0110] In the formula, Ir' represents the corrected image of the red channel; Ig' represents the corrected image of the green channel; Ib' represents the corrected image of the blue channel; Kr' represents the initial gain coefficient of the red channel; Kg' represents the initial gain coefficient of the green channel; and Kb' represents the initial gain coefficient of the blue channel.

[0111] After obtaining the three-channel corrected images of the calibration plate, RGB color images can be obtained by synthesizing based on corresponding algorithms, and the specific process is not described herein.

[0112] According to the foregoing adjustment strategy, if the current scene supports discrete adjustment, the residual error after discrete adjustment is determined according to the number of discrete adjustment, so as to determine the digital gain.

[0113] The residual error includes a first residual error and a second residual error. The discrete adjustment includes light source brightness adjustment and camera exposure time adjustment.

[0114] If either the light source brightness or the camera exposure time can be adjusted, either adjustment mode is selected as a first-level discrete adjustment, and the theoretical level of the first-level discrete adjustment of the channel is calculated based on the initial gain coefficient of the channel, the theoretical level is quantized to an actual level, and the first-level discrete adjustment is realized.

[0115] The first residual error after the first-level discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the first-level discrete adjustment.

[0116] If both the light source brightness and the camera exposure time can be adjusted, either adjustment mode is selected as a second-level discrete adjustment, and the theoretical level of the second-level discrete adjustment of the channel is calculated based on the first residual error, the theoretical level is quantized to an actual level, and the second-level discrete adjustment is realized.

[0117] The second residual error after the second-level discrete adjustment of the channel is calculated based on the theoretical level and the actual level of the second-level discrete adjustment.

[0118] According to the foregoing, the discrete adjustment strategy specifically includes first-level discrete adjustment and second-level discrete adjustment, and the number of discrete adjustment can be selected according to actual scene requirements and hardware conditions.

[0119] The discrete adjustment includes light source brightness adjustment and exposure time adjustment, that is, either adjustment mode can be selected to realize discrete adjustment, or the two discrete adjustment modes can be combined to realize discrete adjustment, and the order of adjustment of the exposure time and the light source brightness in the combined adjustment is not limited.

[0120] If the first discrete regulation is the light source brightness regulation, the theoretical level of the light source brightness of the channel is calculated based on the initial gain coefficient of any channel and the initial light source brightness; and the actual level of the light source brightness is regulated based on the theoretical level and a plurality of actual levels of the light source brightness, each actual level corresponding to different light source brightness, so that the absolute difference between the actual level and the theoretical level is minimum.

[0121] Generally, the color light source (red, green and blue light sources) supports a limited number of discrete brightness levels, and the set thereof is wherein L1 is the darkest and Ln is the brightest. The initial light source brightness is L start .

[0122] Generally, the initial light source brightness is the middle value of the set. Preferably, the initial light source brightness can also be set as any value in the set.

[0123] The embodiment takes the green channel as an example, i.e. the initial gain coefficient Kg' of the green channel is 1. The initial gain coefficients Kr' of the red channel and Kb' of the blue channel are adjusted so as to tend to 1.

[0124] The theoretical levels Level_theory of the light source brightness corresponding to the red and blue channels are calculated, and the theoretical levels are quantified into the closest actual levels.

[0125] The theoretical level Level_theory_r of the light source brightness of the red channel is L start × Kr';

[0126] The theoretical level Level_theory_b of the light source brightness of the blue channel is L start × Kb';

[0127] In the embodiment, the initial light source brightness of the color light source is consistent by default.

[0128] If the initial light source brightness of the color light source is different, the theoretical level of the light source brightness of the channel is calculated based on the initial gain coefficient of the current color channel and the initial light source brightness of the current color light source.

[0129] Since the brightness level of the light source is discrete, the light source brightness level is quantified into the closest actual level Level_actual.

[0130] In the embodiment, the rounding or the downward rounding is adopted to quantify the theoretical level into the actual level, so that the absolute difference between the actual level and the theoretical level is minimum.

[0131] The actual level of the light source brightness of the red channel is ;

[0132] Actual level of light source brightness of blue channel ;

[0133] In the formula, the actual level is determined by the function argmin(·), which means that the argument of the function (the brightness level of the light source) makes the function take the minimum value when input into the function. The absolute difference between any brightness level L and the theoretical level is used as the function.

[0134] Any brightness level L belongs to the set of light source brightness levels .

[0135] Based on the theoretical level and the actual level of the first-order discrete adjustment, the first residual error ε of the first-order discrete adjustment of the channel is calculated.

[0136] First residual error ε of first-order discrete adjustment of red channel r_L = Level_theory_r / Level_actual_r;

[0137] First residual error ε of first-order discrete adjustment of blue channel b_L = Level_theory_b / Level_actual_b.

[0138] If the first-order discrete adjustment is camera exposure time adjustment, based on the initial gain coefficient and the initial exposure time of any channel, the theoretical level of the exposure time of the channel is calculated; based on the theoretical level and the minimum step unit of the camera exposure time, the step value of the camera exposure time is adjusted, and each actual level corresponds to the light amount of different exposure times, so that the absolute difference between the actual level and the theoretical level is minimized.

[0139] In general, the exposure time of a black and white camera is set with a minimum step unit ΔT (such as 1 ms, 1 μs, etc.), and its specific value is set according to the camera model and actual application requirements, which is not limited in this application.

[0140] The initial exposure time T0 is known.

[0141] This embodiment takes the green channel as an example, i.e., the initial gain coefficient of the green channel Kg' = 1. By adjusting the exposure time of the black and white camera, the initial gain coefficients of the red channel Kr' and the blue channel Kb' tend to 1.

[0142] In this embodiment, the initial exposure times of different color channels are all set to T0 by default.

[0143] The theoretical level T_theory of the exposure time corresponding to the red and blue channels is calculated, and the theoretical level is quantized to the closest actual level.

[0144] The theoretical level of the exposure time of the red channel T_theory_r = T0 x Kr';

[0145] The theoretical level of the exposure time of the blue channel T_theory_b = T0 x Kb';

[0146] Preferably, the theoretical level of the exposure time required by the red and blue channels can also be determined based on the green channel exposure time Tg as the initial exposure time.

[0147] In this embodiment, the rounding (function round(·)) or the downward rounding is used to quantize the theoretical level to the actual level, so that the absolute difference between the actual level and the theoretical level is minimized.

[0148] The actual level of the exposure time of the red channel T_actual_r = round(T_theory_r / ΔT) x ΔT;

[0149] The actual level of the exposure time of the blue channel T_actual_b = round(T_theory_b / ΔT) x ΔT;

[0150] Similarly, according to the theoretical level and the actual level, the first residual error after the first discrete adjustment is calculated:

[0151] The first residual error after the first discrete adjustment of the red channel ε r_T =T_theory_r / T_actual_r;

[0152] The first residual error after the first discrete adjustment of the blue channel ε b_T =T_theory_b / T_actual_b.

[0153] If the second discrete adjustment is not set, the first residual error after the first discrete adjustment ε (including ε r , ε b , and ε g ) is used as the digital gain to realize the first discrete adjustment-digital gain (two-stage) adjustment mode.

[0154] The final image I' after white balance correction is: Ir'=Ir x ε r ; Ig'=Ig x ε g ; Ib'=Ib x ε b

[0155] In the formula, Ir' represents the corrected image of the red channel; Ig' represents the corrected image of the green channel; Ib' represents the corrected image of the blue channel; and ε r is the first residual error after the first discrete adjustment of the red channel (including εr_L , ε r_T ) ; ε b is the first residual error of the blue channel after the first discrete adjustment (including ε b_L , ε b_T ) ; the first residual error of the green channel after the first discrete adjustment ε g = 1.

[0156] After obtaining the corrected images of the three channels of the calibration plate, the RGB color image can be obtained by synthesizing based on the corresponding algorithm, and the specific process will not be described herein.

[0157] If the light source brightness and the camera exposure time can be adjusted, any adjustment method is selected as the second discrete adjustment, and the theoretical level of the second discrete adjustment of the channel is calculated based on the first residual error. The theoretical level is quantized to the actual level to realize the second discrete adjustment.

[0158] Based on the theoretical level and the actual level of the second discrete adjustment, the second residual error of the channel after the second discrete adjustment is calculated.

[0159] Preferably, according to the demand, if the first discrete adjustment is the light source brightness, only the light source brightness is coarsely adjusted, the first residual error obtained by the coarse adjustment is calculated to obtain the theoretical level of the second discrete adjustment. At this time, the second discrete adjustment is still selected as the light source brightness for further fine adjustment, and the second residual error of the second discrete adjustment is calculated as the digital gain. Similarly, the camera exposure time can also be set to two-level discrete adjustment of coarse adjustment and fine adjustment.

[0160] If the two-level discrete adjustment adopts the same kind of discrete adjustment, the specific adjustment calculation of the two-level discrete adjustment is consistent, and it can also be considered that the coarse adjustment and the fine adjustment in the same kind of discrete adjustment are the first discrete adjustment.

[0161] In this embodiment, the specific division of the discrete adjustment level is not limited, and the fine adjustment realized based on the first residual error after the first discrete adjustment is the second discrete adjustment.

[0162] According to the foregoing adjustment strategy, the two discrete methods can be combined to form a one-level discrete adjustment-two-level discrete adjustment-digital gain (three-level) adjustment mode. The adjustment order of the exposure time and the light source brightness can be arbitrarily selected. At this time, the first discrete adjustment has completed the minimum precision adjustment, that is, the first residual error obtained by the first discrete adjustment cannot be adjusted by the current discrete adjustment method.

[0163] If the first discrete adjustment is the light source brightness, the second discrete adjustment is the exposure time. At this time, the theoretical level of the exposure time in the second discrete adjustment is calculated based on the first residual error ε of the light source brightness adjustment and the initial exposure time T0.

[0164] Similarly, in this embodiment, the green channel is taken as the reference channel, and the theoretical level of the exposure time in the second adjustment is calculated as follows:

[0165] The theoretical level of the exposure time of the red channel T(II)_theory_r = T0 x ε r_L ;

[0166] The theoretical level of the exposure time of the blue channel T(II)_theory_b = T0 x ε b_L ;

[0167] In the formula, the setting of the initial exposure time T0 is the same as above, and will not be repeated here.

[0168] The theoretical level of the exposure time is quantified into the actual level, and the specific calculation method is the same as above.

[0169] According to the theoretical level T(II)_theory and the actual level T(II)_actual of the second discrete adjustment, the second residual error δ is calculated:

[0170] The second residual error δ of the red channel after the second discrete adjustment of the red channel r_T = T(II)_theory_r / T(II)_actual_r;

[0171] The second residual error δ of the blue channel after the second discrete adjustment of the blue channel b_T = T(II)_theory_b / T(II)_actual_b.

[0172] Similarly, if the first discrete adjustment is the exposure time, the second discrete adjustment is the light source brightness; at this time, according to the first residual error ε after the exposure time adjustment and the initial light source brightness L start , the theoretical level of the light source brightness in the second adjustment is calculated.

[0173] The theoretical level of the light source brightness of the red channel Level(II)_theory_r = L start x ε r_T ;

[0174] The theoretical level of the light source brightness of the blue channel Level(II)_theory_b = L start x ε b_T ;

[0175] In the formula, the setting of the initial light source brightness is the same as above, and will not be repeated here.

[0176] The theoretical level of the light source brightness is quantified into the actual level, and the specific calculation method is the same as above.

[0177] According to the second residual error δ of the second discrete adjustment, the second residual error δ is calculated based on the theoretical level Level(II)_theory and the actual level Level(II)_actual:

[0178] The second residual error δ of the red channel r_L =Level(II)_theory_r / Level(II)_actual_r.

[0179] The second residual error δ of the blue channel b_L =Level(II)_theory_b / Level(II)_actual_b.

[0180] The second residual error δ (including δ r , δ b , δ g ) after the second discrete adjustment is taken as the digital gain to realize the first discrete adjustment-the second discrete adjustment-the digital gain (three-stage) adjustment mode.

[0181] The image I' finally realizing the white balance correction is: Ir'=Ir×δ r ; Ig'=Ig×δ g ; Ib'=Ib×δ b

[0182] In the formula, Ir' represents the corrected image of the red channel; Ig' represents the corrected image of the green channel; Ib' represents the corrected image of the blue channel; δ r is the second residual error of the red channel after the second discrete adjustment (including δ r_L , δ r_T ); δ b is the second residual error of the blue channel after the second discrete adjustment (including δ b_L , δ b_T ); and δ g is the second residual error of the green channel after the second discrete adjustment =1.

[0183] After the three-channel corrected images of the calibration plate are obtained, the RGB color image can be obtained by synthesizing based on the corresponding algorithm, and the specific process is not described herein.

[0184] Preferably, based on the above adjustment strategy, the embodiment also proposes a process for verifying the accuracy of the discrete adjustment mode to improve the white balance correction accuracy.

[0185] Before the digital gain adjustment, further comprising: judging the fitting accuracy of the linear model based on the corresponding linear model of the discrete adjustment, if the fitting accuracy is less than a preset threshold, then completing the digital gain adjustment on the basis of the image after the discrete adjustment; otherwise, obtaining a new three-channel gray image according to the actual level determined by the discrete adjustment as the digital gain adjustment object.

[0186] In the embodiment, the preset threshold is determined according to the adjustment accuracy of the light source and the black-and-white camera and the calculation accuracy in the actual scene.

[0187] The color light source (red, blue and green light source) usually has a linear model between the light source brightness and the light source level, but in actual application, the accuracy of the linear correlation directly affects the color accuracy of the final image sensor imaging; therefore, the embodiment further needs to verify the accuracy of the linear correlation of the linear model of the color light source before the digital gain adjustment.

[0188] Knowing the linear model of each color light source, under different color light sources, according to the gray value change of the image collected by the black-and-white camera under different brightness levels, the relationship between the linear model of the current color light source and the imaging is judged.

[0189] If the difference between the degree of change of the image gray value of the current color light source with the brightness level and the linear model (fitting result) is less than the threshold, it is determined that the fitting accuracy of the light source linear model meets the requirements, then after the light source brightness adjustment, the image after the light source brightness discrete adjustment can be directly used as the digital gain processing object (or the processing object of the second discrete adjustment) to perform digital gain adjustment and output the final corrected image; otherwise, the light source brightness is adjusted according to the actual level determined by the discrete adjustment, and the channel gray image of the actual level light source brightness is collected again as the digital gain processing object (or the processing object of the second discrete adjustment).

[0190] Similarly, the fitting accuracy of the linear model of the camera exposure time and the exposure step value (step unit) is verified, the images corresponding to different exposure times are collected, and the relationship between the image gray value and the change of the camera exposure time and the linear model is analyzed.

[0191] If the difference between the degree of change of the gray value of the image corresponding to any exposure time and the linear model (fitting result) is less than a threshold value, it is determined that the fitting accuracy of the linear model of the camera exposure time meets the requirements, and after exposure time adjustment, the image after exposure time discrete adjustment can be directly used as the processing object of digital gain adjustment (or the processing object of secondary discrete adjustment) to output the final corrected image; otherwise, the actual level of the light source brightness is adjusted according to the discrete adjustment, and the channel gray value image of the actual level of the exposure time is re-acquired as the processing object of digital gain adjustment (or the processing object of secondary discrete adjustment).

[0192] According to the above linear model verification content, the embodiment further proposes a residual error calculation method, which comprises:

[0193] If the linear model of the camera exposure time or the light source brightness does not meet the fitting accuracy requirements, the adjustment model can also be recalibrated; according to the theoretical level of the current discrete adjustment mode (camera exposure time or light source brightness), the predicted value (as the actual level) corresponding to the adjustment model is calculated, so that the absolute difference between the predicted value and the theoretical level is minimized. The ratio of the theoretical level to the predicted value is used as the residual error.

[0194] If it is a primary discrete adjustment, the initial gain coefficient of any channel is used to calculate the theoretical level of the channel; if it is a secondary discrete adjustment, the first residual error of any channel is used to calculate the theoretical level of the channel. The specific calculation method is the same as above, which will not be described here.

[0195] Based on the same inventive concept, the present application further proposes a color image acquisition method, which acquires the corrected images of each channel of the object to be measured when the black-and-white camera reaches white balance according to the above-mentioned image processing method, and synthesizes three corrected images into a color image.

[0196] Based on the same inventive concept, the present application further proposes an electronic device comprising a processor and a memory, wherein the memory is used to store a computer program comprising program instructions, and the processor is configured to invoke the program instructions to execute the above-mentioned method.

[0197] On the other hand, the present application further proposes a computer readable storage medium, wherein at least one instruction or at least one program is stored in the computer readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to realize the above-mentioned method.

[0198] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0199] The above examples are only used to illustrate the technical solutions of the present application, but not to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that, include: Acquire grayscale images of the current color channel of the calibration board by a black and white camera under red, green and blue light sources respectively, and calculate the initial gain coefficient of each grayscale image relative to the target grayscale. If either the light source brightness or the camera exposure time can be adjusted, then either adjustment method is selected as the first-level discrete adjustment. Based on the initial gain coefficient of any channel, the theoretical level of the first-level discrete adjustment of that channel is calculated, the theoretical level is quantified as the actual level, and the first-level discrete adjustment is realized. Based on the theoretical and actual levels of the first-level discrete adjustment, calculate the first residual error of the channel after the first-level discrete adjustment; If both the light source brightness and the camera exposure time can be adjusted, then any adjustment method can be selected as the secondary discrete adjustment. The theoretical level of the secondary discrete adjustment of the channel is calculated using the first residual error, and the theoretical level is quantified as the actual level to realize the secondary discrete adjustment. Based on the theoretical and actual levels of the two-stage discrete adjustment, the second residual error after the two-stage discrete adjustment of this channel is calculated. Based on the discrete adjustment series, using the initial gain coefficient or the residual error after discrete adjustment as the digital gain, the monochrome camera achieves white balance, acquiring calibrated images for each channel, specifically including: If both the light source brightness and the camera exposure time are adjustable in the current scene, the adjustment strategy is: first-level discrete adjustment - second-level discrete adjustment - digital gain adjustment; if the first-level discrete adjustment is the light source brightness adjustment, then the second-level discrete adjustment is the camera exposure time adjustment; or if the first-level discrete adjustment is the camera exposure time adjustment, then the second-level discrete adjustment is the light source brightness adjustment. If the current scene only allows adjustment of light source brightness or camera exposure time, the adjustment strategy is first-level discrete adjustment—digital gain adjustment; If discrete adjustments are not possible in the current scene, digital gain adjustment will be used as the white balance adjustment method to achieve color reproduction of the image. The quantification of the theoretical level to the actual level is: adjusting the brightness of the light source or the exposure time of the camera to minimize the absolute difference between the actual level and the theoretical level; The residual error is the ratio of the theoretical level to the actual level; the residual error includes the first residual error and the second residual error.

2. The image processing method according to claim 1, characterized in that, If the first-level discrete adjustment is the light source brightness adjustment, then the theoretical level of the light source brightness of that channel is calculated based on the initial gain coefficient and initial light source brightness of any channel. Based on the theoretical level and several actual levels of light source brightness, each actual level corresponds to a different light source brightness. The actual level of light source brightness is adjusted to minimize the absolute difference between the actual level and the theoretical level.

3. The image processing method according to claim 1, characterized in that, If the first-level discrete adjustment is the camera exposure time adjustment, then the theoretical level of the exposure time of that channel is calculated based on the initial gain coefficient and initial exposure time of any channel. Based on the theoretical level and the smallest step unit of camera exposure time, the step value of camera exposure time is adjusted. Each actual level corresponds to a different amount of light, so that the absolute difference between the actual level and the theoretical level is minimized.

4. The image processing method according to any one of claims 1-3, characterized in that, Initial gain coefficients, including: Based on the grayscale value of any channel image, calculate the ideal gain coefficient of the grayscale image of that channel relative to the target grayscale; select any channel among the red, green and blue grayscale images as the reference channel, normalize the ideal gain coefficient corresponding to the reference channel to 1, and obtain the normalized initial gain coefficient of each channel.

5. The image processing method according to claim 1, characterized in that, By using rounding or rounding down, the theoretical level is quantified into the actual level, so as to minimize the absolute difference between the actual level and the theoretical level.

6. The image processing method according to claim 1, characterized in that, Before digital gain adjustment, the following steps are also included: based on the linear model corresponding to discrete adjustment, the fitting accuracy of the linear model is determined. If the fitting accuracy is less than a preset threshold, digital gain adjustment is performed on the image after discrete adjustment; otherwise, a new three-channel grayscale image is obtained based on the actual level determined by discrete adjustment, which is used as the object of digital gain adjustment.

7. An image processing system, characterized in that, include: Red, green, and blue light sources; A monochrome camera captures grayscale images of the calibration board in the channels of red, green, and blue light sources. The calculation module calculates the initial gain coefficient of each channel grayscale image relative to the target grayscale; If either the light source brightness or the camera exposure time can be adjusted, then either adjustment method is selected as the first-level discrete adjustment. Based on the initial gain coefficient of any channel, the theoretical level of the first-level discrete adjustment of that channel is calculated, and the theoretical level is quantified as the actual level. Based on the theoretical and actual levels of the first-level discrete adjustment, calculate the first residual error of the channel after the first-level discrete adjustment; If both the light source brightness and the camera exposure time can be adjusted, then any adjustment method can be selected as the secondary discrete adjustment. The theoretical level of the secondary discrete adjustment of this channel is calculated using the first residual error, and the theoretical level is quantified as the actual level. Based on the theoretical and actual levels of the two-stage discrete adjustment, the second residual error after the two-stage discrete adjustment of this channel is calculated. Based on the discrete adjustment series, using the initial gain coefficient or the residual error after discrete adjustment as the digital gain, the monochrome camera achieves white balance, acquiring calibrated images for each channel, specifically including: If both the light source brightness and the camera exposure time are adjustable in the current scene, the adjustment strategy is: first-level discrete adjustment - second-level discrete adjustment - digital gain adjustment; if the first-level discrete adjustment is the light source brightness adjustment, then the second-level discrete adjustment is the camera exposure time adjustment; or if the first-level discrete adjustment is the camera exposure time adjustment, then the second-level discrete adjustment is the light source brightness adjustment. If the current scene only allows adjustment of light source brightness or camera exposure time, the adjustment strategy is first-level discrete adjustment—digital gain adjustment; If discrete adjustments are not possible in the current scene, digital gain adjustment will be used as the white balance adjustment method to achieve color reproduction of the image. The quantification of the theoretical level to the actual level is: adjusting the brightness of the light source or the exposure time of the camera to minimize the absolute difference between the actual level and the theoretical level; The residual error is the ratio of the theoretical level to the actual level; the residual error includes the first residual error and the second residual error.

8. The image processing system according to claim 7, characterized in that, Also includes: The control module, based on the actual level output by the calculation module, adjusts the exposure time of the black and white camera and / or the brightness of the three light sources (red, green, and blue) to achieve discrete adjustment.

9. A method for acquiring color images, characterized in that, include: According to any one of claims 1-6, the image processing method acquires the corrected images of each channel of the object under test when the black and white camera reaches white balance, and synthesizes the three corrected images into a color image.

10. A computer-readable storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the image processing method as described in any one of claims 1-6 or the color image acquisition method as described in claim 9.

Images