Image processing method and device, storage medium, electronic equipment and chip

Abstract

The invention relates to an image processing method and device, a storage medium, electronic equipment and a chip. The method comprises the steps of firstly obtaining an original image; performing color space conversion on the original image to obtain a first image; determining auxiliary information corresponding to the first image; performing white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result; and performing white balance correction on the original image according to the white point estimation result. According to the technical scheme, color space conversion processing is carried out on the original image, the generalization ability of the sensor is improved, the accuracy of a white point estimation result after color space alignment is obviously improved, white balance correction is carried out according to the white point estimation result of the processed original image, and the image quality is improved. And meanwhile, the stability and convergence timeliness of the full-scene white balance time domain effect are considered, so that the white balance effect is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, storage medium, electronic device, and chip. Background Art

[0002] During the camera imaging process, since the color of the light source under different light conditions can affect the color of the photographed object, there is a deviation between the color of the photographed object and the inherent color of the actual object. In order to make the camera imaging have a similar human visual color constancy, that is, to restore the inherent color of the object under different light conditions, the camera white balance technology has emerged.

[0003] Currently, the corresponding relationship between the color of natural objects and the color temperature of light source illumination can be modeled based on the big data statistics of human a priori to solve the white balance problem. Common algorithms include the gray world method, the white patch method, the white point-color temperature calibration method, etc. By reasonably weighted summing the statistical points distributed in the white area of the image, the color temperature of the scene light source is estimated, and then the white balance effect is achieved.

[0004] However, the traditional statistical modeling method relies on the statistical analysis of specific regions (such as white) in the image. When fewer statistical points fall in the white area or are mis-statistically analyzed, it may not be possible to accurately find the statistical points, resulting in a poor white balance effect. Summary of the Invention

[0005] The present disclosure provides an image processing method, apparatus, storage medium, electronic device, and chip, mainly aiming to improve the technical problem that the statistical points cannot be accurately found, resulting in a poor white balance effect.

[0006] According to the first aspect of the embodiments of the present disclosure, an image processing method is provided, including:

[0007] Obtain an original image;

[0008] Perform color space conversion on the original image to obtain a first image;

[0009] Determine the auxiliary information corresponding to the first image;

[0010] Perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result;

[0011] Perform white balance correction on the original image according to the white point estimation result.

[0012] Optionally, the shooting parameters include: sensor size, actual focal length, and object height;

[0013] The determining the target shooting distance between the object and the shooting lens according to the shooting parameters includes:

[0014] Determine a corresponding sensor conversion coefficient according to the sensor size;

[0015] Based on the sensor conversion coefficient, convert the actual focal length into an equivalent focal length;

[0016] Based on the sensor size, the equivalent focal length, and the object height, calculate the target shooting distance.

[0017] Optionally, the obtaining of the first image by performing color space conversion on the original image includes:

[0018] Perform lens shading correction and black level subtraction processing on the original image to generate a second image;

[0019] Perform color space mapping processing on the second image to obtain the first image.

[0020] Optionally, the performing of color space mapping processing on the second image to obtain the first image includes:

[0021] Screen out a reference sensor from multiple different types of image sensors;

[0022] Based on the preset color card pixel values of the reference sensor and the current image sensor, map the second image to the color space of the reference sensor to obtain the first image.

[0023] Optionally, the determining of the auxiliary information corresponding to the first image includes:

[0024] Calculate the auxiliary information based on the scene brightness and the global image statistical information of the first image.

[0025] Optionally, before performing white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result, the method further includes:

[0026] Perform scene classification on the first image to obtain a scene classification result;

[0027] According to the scene classification result, select a white point estimation model that matches the current scene;

[0028] Performing white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result includes:

[0029] Input the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain the white point estimation result.

[0030] Optionally, inputting the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain the white point estimation result includes:

[0031] Determine whether the usage condition of the white point estimation model is satisfied according to the frame identifier of the current video stream and the frequency factor of model scheduling;

[0032] If the usage condition of the white point estimation model is satisfied, input the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain the white point estimation result.

[0033] Optionally, performing white balance correction on the original image according to the white point estimation result includes:

[0034] Perform mean fusion on the white point estimation result and convert the white point estimation result into an estimated white point corresponding to the current image sensor color space;

[0035] Perform time-domain smoothing processing on the estimated white point to obtain a processed estimated white point;

[0036] Perform white balance correction on the original image by using the processed estimated white point.

[0037] According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including:

[0038] An acquisition module configured to acquire an original image;

[0039] A conversion module configured to obtain a first image by performing color space conversion on the original image;

[0040] A determination module configured to determine auxiliary information corresponding to the first image;

[0041] An estimation module configured to perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result;

[0042] A processing module configured to perform white balance correction on the original image according to the white point estimation result.

[0043] According to a third aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0044] According to a fourth aspect of the embodiments of the present disclosure, a communication device is provided, including: a transceiver; a memory; and a processor, which are respectively connected to the transceiver and the memory, and configured to control the wireless signal transceiver of the transceiver by executing computer-executable instructions on the memory, and be capable of implementing the method described in the first aspect.

[0045] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0046] According to a sixth aspect of the embodiments of the present disclosure, a chip is provided, including one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal from a memory of an electronic device and send the signal to the processor, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the method described in the first aspect.

[0047] By means of the above technical solutions, the present disclosure provides an image processing method, apparatus, storage medium, electronic device and chip. Specifically, first, an original image is acquired; then, by performing color space conversion on the original image, a first image is obtained; auxiliary information corresponding to the first image is determined; then, white point estimation is performed based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result; and white balance correction is performed on the original image according to the white point estimation result. Compared with the current related technologies, the present application improves the generalization ability of the sensor by performing color space conversion processing on the original image. After color space alignment, the accuracy of the white point estimation result is significantly improved. White balance correction is performed according to the white point estimation result of the processed original image, taking into account both the stability and convergence timeliness of the white balance time domain effect in the full scene, thereby improving the white balance effect.

[0048] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0050] Figure 1 A flowchart showing an image processing method provided by an embodiment of the present disclosure is shown;

[0051] Figure 2 A flowchart showing an image processing method provided by an embodiment of the present disclosure is shown;

[0052] Figure 3Shows a schematic flowchart of an example provided by an embodiment of the present disclosure;

[0053] Figure 4 Shows a schematic flowchart of an example provided by an embodiment of the present disclosure;

[0054] Figure 5 Shows a schematic flowchart of an example provided by an embodiment of the present disclosure;

[0055] Figure 6 Shows a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. Detailed implementation manners

[0056] Here, some embodiments of the present disclosure will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of the operations described herein is merely an example and is not limited to those set forth herein, but may be changed as will be apparent after understanding the present disclosure, except for operations that must be performed in a specific order. Additionally, descriptions of features known in the art may be omitted for the sake of clarity and conciseness. It should be noted that, without conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0057] The implementation manners described in some embodiments of the present disclosure below do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0058] Figure 1 Is a flowchart of an image processing method shown according to some embodiments of the present disclosure, as Figure 1 Shown, and includes the following steps.

[0059] Step 101, obtain an original image.

[0060] In some examples, the optical signal in the physical world can be captured by an image sensor. First, the sensor is used to convert the light into an electrical signal to generate raw data in the bayer format. Among them, the bayer data is the raw image data obtained from an image sensor using a Bayer color filter (also known as a color filter array, CFA), ensuring that the scene information is restored as realistically as possible during the process from light signal capture to image generation.

[0061] Step 102: Obtain a first image by performing color space conversion on the original image.

[0062] In some examples, first preprocess the bayer data of the sensor to output a three-channel linear RGB image. For example, taking RGrGbBbayer (i.e., the Bayer filter array) as an example, since the Bayer filter only allows each pixel to receive one of the three colors red, green, and blue, first perform demosaicing, that is, interpolate and calculate the missing two color components of each pixel. This step generates a full-color RGB image where each pixel contains complete red, green, and blue information, and then convert the image to another color space by applying corresponding mathematical transformations. Other different SensorBayer types are also within the protection scope of this embodiment.

[0063] Exemplarily, this embodiment not only supports the data processing of multiple Sensor Bayer types, but also provides a high-quality input basis for subsequent image processing tasks, enhancing the generality and processing effect of the algorithm. This process ensures the accuracy and efficiency in the process from the original image data to the subsequent image conversion.

[0064] Step 103: Determine the auxiliary information corresponding to the first image.

[0065] For example, the auxiliary information may include the scene brightness and RGB color values calculated based on the first image, as well as the global statistical information of the image obtained through a series of formula processing. These statistical information specifically involve the brightest pixel, the darkest pixel, the maximum and mean values of the three channels in the image, and combine threshold truncation operations to remove the influence of overexposed or underexposed pixels. Such auxiliary information, as the input for the inference of the artificial intelligence-based automatic white balance (AI Auto White Balance, AIAWB) network, can help improve the accuracy of white point estimation, enabling the model to more accurately understand the light source characteristics and their influence on colors under different lighting conditions, thereby achieving a better automatic white balance adjustment effect.

[0066] Step 104: Perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result.

[0067] Exemplarily, performing white point estimation based on the first image and the auxiliary information corresponding to the first image enables the model to accurately estimate the white point of the light source according to the specific lighting conditions and color distribution of the current scene. It can not only improve the accuracy of white point estimation, but also enhance the adaptability and stability of the model in different environments, ensuring that the color performance of the final output image is more natural and real, effectively reducing color deviation and optimizing the visual experience.

[0068] Step 105: Perform white balance correction on the original image according to the white point estimation result.

[0069] Exemplarily, first perform mean fusion on the white point estimation result and convert it into the estimated white point corresponding to the current image sensor color space. Then perform time-domain smoothing processing on the estimated white point to ensure stability. Next, use the processed estimated white point to perform white balance correction on the original image data, adjusting the red, green, and blue channel values of each pixel so that white in the image can be accurately presented under various lighting conditions. Finally, output an RGB image with more realistic and natural colors, effectively improving the image quality and visual experience. It ensures high-quality conversion from the original image to the final product and enhances the accuracy of color reproduction.

[0070] Compared with the current related technologies, in this embodiment, first obtain the original image; then perform color space conversion on the original image to obtain the first image; determine the auxiliary information corresponding to the first image; then perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain the white point estimation result; perform white balance correction on the original image according to the white point estimation result. By applying the technical solution of this embodiment, performing color space conversion processing on the original image improves the generalization ability of the sensor. After color space alignment, the accuracy of the white point estimation result is significantly improved. Performing white balance correction according to the white point estimation result of the processed original image takes into account both the stability and convergence timeliness of the white balance time-domain effect in the full scene, thereby improving the white balance effect.

[0071] For further illustration of the specific implementation process of the method as Figure 1 shown, this embodiment provides the specific method as Figure 2 shown, and this method includes:

[0072] Step 201: Obtain the original image.

[0073] Step 202: Perform lens shading correction and black level subtraction processing on the original image to generate the second image.

[0074] Exemplarily, first perform lens shading correction and black level subtraction processing on the bayer data to reduce the influence of luminance shading, color shading, and dark current on the image quality of the model input image; use the bayer data to generate an RGB three-channel low-resolution image: the R and B channels remain unchanged, and the Gr and Gb are averaged to obtain the G channel, and perform local average downsampling on the image to obtain the second image RGB small (For example, the image size is 192*256*3).

[0075] In some examples, performing lens shading correction and black level reduction on Bayer data can not only improve image quality, making the image more accurate and realistic by reducing the effects of luminance shadows, color shadows, and dark current noise, but also optimize the data quality provided to subsequent image processing algorithms or machine learning models, ensuring that these systems can process based on more precise information. Additionally, converting to a low-resolution image can significantly reduce the computational complexity and the amount of data to be processed, facilitating real-time processing and applications on resource-constrained devices.

[0076] Step 203: Perform color space mapping processing on the second image to obtain the first image.

[0077] In some examples, first, a reference image sensor needs to be selected, and its pixels under each color block of the 24-color card are calculated. Then, the 24-color card pixel values of the current sensor are calculated using the same method. Next, a color conversion matrix is calculated based on the 24-color card pixel values of the current and reference sensors, and the RGB image obtained by the current sensor is converted to the color space of the reference sensor through this matrix to obtain the first image. This can effectively correct the color performance differences between different sensors and ensure the consistency and accuracy of image colors.

[0078] Optionally, step 203 may specifically include: screening out a reference sensor from multiple different types of image sensors; mapping the second image to the color space of the reference sensor based on the preset color card pixel values of the reference sensor and the current image sensor to obtain the first image.

[0079] In some examples, based on the preset color card pixel values between the selected reference sensor and the currently used image sensor, color space conversion calculations are performed to convert the image obtained by the second image sensor (i.e., the second image) to the color space of the reference sensor, thereby generating the first image that is consistent with the reference standard. This process achieves the consistency of color representation between different sensors and ensures the true reproduction of image colors.

[0080] Exemplarily, perform color space mapping processing on the RGB small image to obtain RGB ctm , and the specific process is as follows: Select a reference sensor from multiple different types of image sensors and calculate the 24-color card pixel values of this sensor:

[0081]

[0082] Calculate the 24-color card pixel values of the current sensor:

[0083]

[0084] Map the current sensor image to the reference sensor color space to obtain the first image RGB ctm :

[0085] RGB ctm = RGB · CTM (Equation 3)

[0086]

[0087] Where R d,λ,k represents the spectral reflectance of the Macbeth ColorChecker (Macbeth 24-color card) under E light; represents the spectral response sensitivity of the reference sensor; represents the spectral response sensitivity of the current sensor; L λ represents the spectral power density of the light source.

[0088] In some examples, the image preprocessing module performs color space conversion on the image to enable the AIAWB model to have cross-sensor generalization ability. Using the Color Transformation Matrix (CTM) for color space alignment, only the spectral response information of the sensor needs to be calibrated, and the AIAWB model can be quickly generalized to other cameras and sensors, greatly reducing the time and labor costs of data acquisition and model retraining for new sensors.

[0089] Step 204, determine the auxiliary information corresponding to the first image.

[0090] Optionally, step 204 may specifically include: calculating the auxiliary information based on the scene brightness and global image statistical information of the first image.

[0091] In some examples, the light intensity index (LuxIdx) and RGB ctm are sent to the prior parameter preprocessing module, and the auxiliary information is calculated using the channel brightest, darkest, channel maximum, and channel mean, etc. The following is one of the examples, and the following processing is performed within the module:

[0092]

[0093] RGB tmp = CLIP(RGB ctm , thr dark , thr sat ) (Equation 7)

[0094] Pt bright = Argmax(RGB tmp ) (Equation 8)

[0095] Ch max = Ch max (RGB tmp ) (Equation 9)

[0096] Ch mean = Chmean(RGB tmp ) (Equation 10)

[0097] Pt dark = Argmin(RGB tmp ) (Equation 11)

[0098]

[0099] where LuxIdx is a parameter representing the scene brightness, is a normalization factor; CLIP() performs threshold truncation on the image, and thr dark represents the dark pixel threshold, and thr sat represents the overexposure threshold; Argmax() represents obtaining the brightest pixel of the image, Argmin() represents obtaining the darkest pixel of the image; Chmax() represents obtaining the maximum value of the three channels of the image, and Chmean() represents obtaining the average value of the three channels of the image; Feature stat represents the finally obtained statistical information and serves as an auxiliary input for the AIAWB network inference.

[0100] Exemplarily, using the scene brightness LuxIdx and RGB as the auxiliary inputs of the network is beneficial to improving the white point estimation accuracy of the AIAWB network. Since the brightness and RGB color values directly reflect the current scene's lighting conditions and color distribution, this information provides additional context for the model, enabling it to better understand the actual light source characteristics in the image and their impact on color, thus making more accurate white balance adjustments. Under different lighting conditions, the color performance of the object surface will vary. By introducing brightness as an input, the model can learn how to adjust its white point estimation strategy according to different lighting intensities, improving its adaptability and robustness in various environments. The RGB values help the model identify the main hues in the image and infer the color temperature of the light source accordingly. This helps the model perform more accurate white balance adjustments, especially in the case of complex or mixed light sources, effectively reducing color deviations.

[0101] Step 205: Perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result.

[0102] Exemplarily, as shown in Figure 3 and Figure 4 , where Figure 3Distribution maps of the landing points of the white points of 4 different sensors (IMX586, sonyimx155, ov5640, smc509) in the white map space, where GT is the ground truth value; Figure 4 Distribution maps of the landing points of the white points GT of 4 different sensors in the whitemap space after color space conversion by CTM. It can be seen that after color space alignment, the white point consistency of cross-sensor training data is significantly improved.

[0103] Optionally, the method of this embodiment may further include: performing scene classification on the first image to obtain a scene classification result; selecting a white point estimation model matching the current scene according to the scene classification result; correspondingly, step 205 may specifically include: inputting the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain a white point estimation result.

[0104] In some examples, the scene classification module performs scene classification through a Convolutional Neural Network (CNN) to determine whether the current scene is a monochromatic light source, a pure color, or a conventional scene, etc., and selects a specific white point estimation model (AIAWB model) for network inference according to the classification result to obtain the white point estimation result corresponding to the AIAWB network. The scene classification here can be extended, such as there are also scenes like green plants, wood grains, multi-color temperatures, etc., but not limited to the three scenes listed above. By using the scene classification module and using different AIAWB model weights for different scenes, since the training data avoids the problem of semantic ambiguity samples, each model can obtain better white point estimation accuracy, thereby improving the accuracy performance of the entire scene.

[0105] Exemplarily, the scene classification result, the first image, and the auxiliary information corresponding to the first image are sent to the AIAWB network selection and inference module for AIAWB white point estimation to obtain a network inference result (wps).

[0106] In some examples, the AIAWB network can be designed using a Mixture of Experts (MoE) structure, which can improve the performance and efficiency of the model by combining the outputs of multiple "expert" networks. The network outputs multiple sets of white point estimation results (such as 3 sets, and the wps data dimension is [3, 2]). Using the MoE structure can improve the overall white point estimation accuracy and reduce the probability of errors in the white balance effect. Here, the MoE uses different structures to extract different features of interest, such as Mobile-like, AlexNet, etc. AIAWB adopts the MoE structure and multi-model strategy, and uses prior statistical information as the auxiliary input of the model to output multiple sets of white points, which is beneficial to reducing the probability of white balance errors and can obtain better white point estimation accuracy performance.

[0107] Optionally, step 205 may specifically further include: determining whether the usage condition of the white point estimation model is satisfied according to the frame identifier of the current video stream and the frequency factor of model scheduling; if the usage condition of the white point estimation model is satisfied, input the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain the white point estimation result.

[0108] Exemplarily, formula thirteen can be used to determine whether to perform AIAWB scheduling. If Model run is 0, the usage condition of the white point estimation model is satisfied; otherwise, the usage condition of the white point estimation model is not satisfied:

[0109]

[0110] where Frame id represents the frame identifier (frame ID) of the current video stream, and μ represents the frequency factor of model scheduling.

[0111] Step 206, perform white balance correction on the original image according to the white point estimation result.

[0112] The method of this embodiment can be used for automatic white balance calibration in the image signal processor (ISP) pipeline image processing link of camera photographing, video recording, etc. of intelligent mobile terminals, helping the colors of objects in the finally output image to be close to the colors observed by the human eye in the physical world.

[0113] Optionally, step 206 may specifically further include: performing mean fusion on the white point estimation result, and converting the white point estimation result into the estimated white point corresponding to the current image sensor color space; performing time-domain smoothing processing on the estimated white point to obtain the processed estimated white point; using the processed estimated white point to perform white balance correction on the original image.

[0114] Exemplarily, such as Figure 3As shown, the wps is passed into the MoE white point post - processing module, where the wps white points are mean - fused and the white point color space conversion is performed:

[0115] WP tmp =(∑wos) / n*Inv(CTM) (Formula XIV)

[0116] where n represents the number of MoE, Inv() represents the inverse of the CTM matrix, and the estimated white point WP of the current sensor color space is obtained through calculation tmp .

[0117] In some examples, the white point estimation result is sent to the result smoothing processing module for time - domain smoothing processing, which can avoid the problem of effect stability that may occur during model switching. An array with length L is used to store the latest WP in the time domain tmp , obtaining Tuple wp , and time - domain effect smoothing is performed through mean filtering:

[0118] WP=(∑Tuplewp) / L (Formula XV)

[0119] For example, WP = [rg, bg] is the final estimated white point of AIAWB, and the finally processed estimated white point is used to perform white balance correction on the bayer data input by the sensor:

[0120]

[0121] where S r 、S Gr 、S Gb 、S B respectively represent the bayer data obtained by the current sensor, and R, Gr, Gb, and B are the output results of the final white balance calibration.

[0122] Exemplarily, as Figure 5 shown, the RGB / GbB data obtained by the image sensor is first pre - processed by the image pre - processing module, and then the scene is classified by the scene classification module. According to the scene classification result and the information of the prior parameter pre - processing module, the corresponding AIAWB network inference is selected and executed, including the Base network, the solid color network, and the monochromatic light source network, etc. Next, the inference result (wps) is further processed by the MoE white point post - processing module, and finally the final white balance parameter (wp) is generated by the time - domain smoothing processing module.

[0123] In some examples, it is ensured that the color of the output image is closer to the true color observed by the human eye, while improving the stability and accuracy under different lighting conditions. This method not only reduces the effect fluctuations caused by model switching, but also ensures the quality of the final RGB output through precise processing of Bayer data, enabling the intelligent mobile terminal to provide a more real and stable color performance during photo-taking and video-recording, enhancing the user experience.

[0124] Compared with the current related technologies, in this embodiment, by performing color space conversion processing on the original image, the generalization ability of the sensor is improved, and the accuracy of the white point estimation result is significantly enhanced after color space alignment. Based on the white point estimation result of the processed original image, white balance correction is performed, taking into account both the stability and convergence timeliness of the white balance time domain effect in the full scene, thereby improving the white balance effect, and the AIAWB network has low computing power and can be deployed on intelligent terminal devices with limited computing power.

[0125] Figure 6 It is a block diagram of an image processing device shown according to some embodiments of the present disclosure. Referring to Figure 6 , the device includes: an acquisition module 31, a conversion module 32, a determination module 33, an estimation module 34, and a processing module 35.

[0126] The acquisition module 31 is configured to acquire the original image;

[0127] The conversion module 32 is configured to obtain a first image by performing color space conversion on the original image;

[0128] The determination module 33 is configured to determine the auxiliary information corresponding to the first image;

[0129] The estimation module 34 is configured to perform white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result;

[0130] The processing module 35 is configured to perform white balance correction on the original image according to the white point estimation result.

[0131] In some embodiments, the conversion module 32 is specifically configured to perform lens shading correction and black level reduction processing on the original image to generate a second image; and perform color space mapping processing on the second image to obtain the first image.

[0132] In some embodiments, the conversion module 32 is further specifically configured to screen out a reference sensor from multiple different types of image sensors; and map the second image to the color space of the reference sensor based on the preset color card pixel values of the reference sensor and the current image sensor to obtain the first image.

[0133] In some embodiments, the determining module 33 is specifically configured to calculate the auxiliary information based on the scene brightness of the first image and the global image statistics.

[0134] In some embodiments, the first image is subjected to scene classification to obtain a scene classification result; according to the scene classification result, a white point estimation model matching the current scene is selected; correspondingly, the estimating module 34 is specifically configured to input the first image and the auxiliary information corresponding to the first image into the white point estimation model for white point estimation to obtain the white point estimation result.

[0135] In some embodiments, the estimating module 34 is further specifically configured to determine whether the usage condition of the white point estimation model is satisfied according to the frame identifier of the current video stream and the frequency factor of model scheduling; if the usage condition of the white point estimation model is satisfied, the first image and the auxiliary information corresponding to the first image are input into the white point estimation model for white point estimation to obtain the white point estimation result.

[0136] In some embodiments, the processing module 35 is specifically configured to perform mean fusion on the white point estimation result, and convert the white point estimation result into an estimated white point corresponding to the color space of the current image sensor; perform time-domain smoothing processing on the estimated white point to obtain a processed estimated white point; use the processed estimated white point to perform white balance correction on the original image.

[0137] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0138] Those skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of the present disclosure can be implemented by electronic hardware, computer software, or a combination of the two. Whether such a function is implemented by hardware or software depends on the specific application and the design requirements of the entire system. For each specific application, those skilled in the art can use various methods to implement the described function, but such implementation should not be construed as exceeding the scope protected by the embodiments of the present disclosure.

[0139] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor of a computer, the functions of any one of the above method embodiments are implemented.

[0140] The present disclosure also provides a computer program product. When the computer program product is executed by a computer, it realizes the functions of any one of the above method embodiments. That is, a computer program is stored thereon, and when the computer program product is executed by a processor of the computer, it realizes the functions of any one of the above method embodiments.

[0141] Based on the method as Figures 1 to 2 described above, and Figure 6 the virtual device embodiment as Figures 1 to 2 described above, this embodiment further provides a chip, including one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal from a memory of an electronic device and send the signal to the processor, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the method as

[0142] described above.

[0143] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, the processes or functions according to the embodiments of the present disclosure are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer program can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server, data center, etc. that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a high-density digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0144] Those of ordinary skill in the art can understand that the various digital numbers such as the first, second, etc. involved in the present disclosure are only for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, nor do they represent the order of precedence.

[0145] At least one in the present disclosure can also be described as one or more. The plurality can be two, three, four, or more, and the present disclosure does not make any limitations. In the embodiments of the present disclosure, for a technical feature, the technical features in this technical feature are distinguished by "first", "second", "third", "A", "B", "C", and "D", etc. There is no order of precedence or size order among the technical features described by the "first", "second", "third", "A", "B", "C", and "D".

[0146] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (such as a disk, optical disc, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0147] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0148] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client - server relationship is created by computer programs running on respective computers and having a client - server relationship with each other.

[0149] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of this disclosure application can be achieved. There is no limitation herein.

[0150] In addition, it should be understood that the various embodiments described in this disclosure can be implemented separately or, where the solution permits, in combination with other embodiments.

[0151] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments claimed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this disclosure.

[0152] Those skilled in the art can clearly understand that for the sake of convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0153] As described above, this is only a specific embodiment of the present disclosure. However, the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Get the original image; Obtaining a first image by performing color space conversion on the original image; determining auxiliary information corresponding to the first image; Performing white point estimation based on the first image and auxiliary information corresponding to the first image to obtain a white point estimation result; White balance correction is performed on the original image according to the white point estimation result.

2. The method according to claim 1, characterized in that The step of converting the original image into a color space to obtain a first image includes: Performing lens shading correction and black level reduction processing on the original image to generate a second image; Perform color space mapping processing on the second image to obtain the first image.

3. The method according to claim 2, characterized in that The performing color space mapping processing on the second image to obtain the first image includes: Screening out a reference sensor from among a plurality of different types of image sensors; Based on the preset color card pixel values ​​of the reference sensor and the current image sensor, the second image is mapped to the color space of the reference sensor to obtain the first image.

4. The method according to claim 1, characterized in that: The determining the auxiliary information corresponding to the first image includes: The auxiliary information is calculated based on scene brightness and image global statistical information of the first image.

5. The method according to claim 1, characterized in that Before performing white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result, the method further includes: Performing scene classification on the first image to obtain a scene classification result; According to the scene classification result, selecting a white point estimation model matching the current scene; Performing white point estimation based on the first image and the auxiliary information corresponding to the first image to obtain a white point estimation result includes: The first image and the auxiliary information corresponding to the first image are input into the white point estimation model to perform white point estimation to obtain the white point estimation result.

6. The method according to claim 5, characterized in that The step of inputting the first image and the auxiliary information corresponding to the first image into the white point estimation model to perform white point estimation to obtain the white point estimation result includes: Determining whether a use condition of the white point estimation model is met according to a frame identifier of the current video stream and a frequency factor of model scheduling; If the use condition of the white point estimation model is met, the first image and the auxiliary information corresponding to the first image are input into the white point estimation model to perform white point estimation to obtain the white point estimation result.

7. The method according to any one of claims 1 to 6, characterized in that The performing white balance correction on the original image according to the white point estimation result comprises: Performing mean fusion on the white point estimation result, and converting the white point estimation result into an estimated white point corresponding to the current image sensor color space; Performing time domain smoothing processing on the estimated white point to obtain a processed estimated white point; The processed estimated white point is used to perform white balance correction on the original image.

8. An image processing device, characterized in that: include: An acquisition module is configured to acquire an original image; A conversion module, configured to obtain a first image by performing color space conversion on the original image; a determination module, configured to determine auxiliary information corresponding to the first image; an estimation module, configured to perform white point estimation based on the first image and auxiliary information corresponding to the first image to obtain a white point estimation result; The processing module is configured to perform white balance correction on the original image according to the white point estimation result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: Comprising the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

12. A chip, characterized in that: The invention comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 7.

Images