Image processing method and device, electronic equipment, chip and storage medium

Abstract

The application provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: in response to a shooting operation, a first shooting image is acquired; in response to the fact that a shooting scene corresponding to the first shooting image is the same as a shooting scene corresponding to a reference image, a known white point estimation result corresponding to the reference image is taken as a first target white point estimation result corresponding to the first shooting image; and the first shooting image is subjected to white balance (AWB) correction according to the first target white point estimation result, so as to obtain a first target image. Thus, in the case where the shooting scene of the first shooting image is the same as that of the reference image, the known white point estimation result corresponding to the reference image is directly taken as the final white point estimation result corresponding to the first shooting image, and it is not necessary to calculate the white point estimation result corresponding to the first shooting image in real time, so that the power consumption and the computing power of the equipment can be effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method and device, an electronic device, a chip and a storage medium. BACKGROUND

[0002] The human visual system has a complex color constancy, which means that even under different light source conditions, the human eye and brain can perceive the true color of an object. For example, a white piece of paper seen in daylight appears white even under the warm yellow light of a tungsten bulb. This ability enables humans to accurately identify the true color of an object under various lighting conditions.

[0003] However, the imaging system of a camera, including the camera and the sensor, does not have this natural color constancy. The sensor of the camera directly records the intensity and wavelength of light without the ability to intelligently adjust the ambient light. Therefore, under different color temperature light sources, the same object can be photographed in different colors. For example, a white piece of paper under an incandescent lamp can be photographed in a yellowish color, while the same piece of paper under a fluorescent lamp can appear bluish or greenish.

[0004] To enable the camera to simulate the color constancy of the human visual system, the industry has proposed the concept of automatic white balance (Auto White Balance, abbreviated as AWB). The AWB technology aims to automatically adjust the color response of the camera through an algorithm to ensure that photos or videos taken under different light source conditions can accurately restore the inherent color of the object. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems in the related art.

[0006] To this end, the present application proposes an image processing method, device, electronic device, chip and storage medium to directly use the known white point estimation result corresponding to the reference image as the final white point estimation result corresponding to the first photographed image without real-time calculation of the white point estimation result corresponding to the first photographed image, which can effectively reduce the power consumption and computing power of the device.

[0007] An embodiment of the present application proposes an image processing method, comprising:

[0008] In response to a shooting operation, a first photographed image is obtained;

[0009] in response to the first photographed image corresponding to a same shooting scene as a reference image corresponding to a shooting scene, the known white point estimation result corresponding to the reference image is taken as a first target white point estimation result corresponding to the first photographed image;

[0010] According to the first target white point estimation result, the first photographed image is subjected to white balance AWB correction to obtain a first target image.

[0011] Another aspect of the present application provides an image processing device, comprising:

[0012] The acquisition module is configured to acquire a first photographed image in response to a shooting operation.

[0013] The processing module is configured to, in response to the first photographed image corresponding to a same shooting scene as a reference image corresponding to a shooting scene, take a known white point estimation result corresponding to the reference image as a first target white point estimation result corresponding to the first photographed image.

[0014] The correction module is configured to, according to the first target white point estimation result, subject the first photographed image to white balance AWB correction to obtain a first target image.

[0015] Another aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image processing method according to the foregoing aspect.

[0016] Another aspect of the present application provides a chip, comprising an interface circuit and a processing circuit coupled with each other, the interface circuit is configured to input or output signals, and the processing circuit is configured to execute the image processing method according to the foregoing aspect.

[0017] Another aspect of the present application provides a non-transitory computer readable storage medium, having stored thereon computer program instructions, which are executed by a processor to implement the image processing method according to the foregoing aspect.

[0018] Another aspect of the present application provides a computer program product, having stored thereon a computer program, which is executed by a processor to implement the image processing method according to any of the foregoing aspects.

[0019] The image processing method, device, electronic device, chip and storage medium provided in the application can directly use the known white point estimation result corresponding to the reference image as the final white point estimation result corresponding to the first photographed image in the case that the shooting scene of the first photographed image is the same as that of the reference image, without the need to calculate the white point estimation result corresponding to the first photographed image in real time, which not only can effectively reduce the device power consumption and computing power, but also can guarantee the accuracy of white point estimation, so that the first photographed image is corrected based on the accurate white point estimation result, which can significantly improve the white balance effect and overall quality of the image and improve the shooting experience of the user.

[0020] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and / or additional aspects and advantages of the application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a first image processing method provided by an embodiment of the application;

[0023] Figure 2 A flowchart of a second image processing method provided by an embodiment of the application;

[0024] Figure 3 A flowchart of a third image processing method provided by an embodiment of the application;

[0025] Figure 4 A flowchart of a fourth image processing method provided by an embodiment of the application;

[0026] Figure 5 A schematic diagram of the implementation principle of any embodiment of the application;

[0027] Figure 6 A schematic diagram of the falling point distribution of different sensor white point GT (Ground Truth, i.e., white point true value) in a whitemap (white point mapping) space provided by an embodiment of the application;

[0028] Figure 7 A schematic diagram of the falling point distribution of different sensor white point GT after CTM color space conversion in a whitemap space provided by an embodiment of the application;

[0029] Figure 8 A structural schematic diagram of an image processing device provided by an embodiment of the application;

[0030] Figure 9 Fig. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0031] Figure 10 Fig. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application; DETAILED DESCRIPTION

[0032] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar notations used throughout the drawings denote the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0033] In the related art, the AWB method mainly includes the following two categories:

[0034] The first category: a big data statistical method based on human prior (hereinafter referred to as a statistical modeling AWB method based on prior), which models the corresponding relationship between natural object color and light source color temperature to solve the white balance problem. Common algorithms include gray world method, white block method, white point-color temperature calibration method, etc.

[0035] The second category: a deep learning method in the field of artificial intelligence (AI) (such as an AI-based scene color temperature estimation method), that is, through neural network learning on image data, realizing end-to-end image scene light source color temperature estimation.

[0036] With the continuous development and application of AI technology, a large number of studies have confirmed that compared with using a traditional prior statistical modeling AWB method, using a deep learning method in the field of AI can significantly improve the AWB effect and greatly reduce the color temperature misjudgment scene. Although the AI-based scene color temperature estimation method (hereinafter referred to as the AIAWB method) can obtain more accurate color restoration than the traditional prior statistical modeling AWB method, the time domain stability is significantly poorer than the traditional prior statistical modeling AWB method, therefore, how to ensure the time domain stability and convergence timeliness of the AIAWB method is a challenge.

[0037] In summary, the AWB algorithm in the related art mainly includes two categories: white point-color temperature calibration method and hybridAWB (hybridAWB), and the hybridAWB can be divided into two categories: traditional AWB+AISeg (AI segmentation) and traditional AWB+AIAWB.

[0038] Among them, the white point-color temperature calibration method: reasonably weighted sum of statistical points distributed in the white area of the image to estimate the scene light source color temperature. The advantages of this method are: in practical application, the parameters can be adapted according to the white balance effect of the specific scene, which has a strong parameter adjustment space, and because it uses a prior statistical analysis method, the time domain white balance effect of the conventional scene is relatively stable; The disadvantage is: in the case that the statistical points are less in the white area (such as large area of green plants / skin color scene) or the statistical points in the white area are miscounted (such as yellow table and chair), the white balance effect is incorrect.

[0039] Among them, hybridAWB: (1) traditional AWB+AISeg: using AI scene segmentation, its advantages are to segment out non-white / grey objects (such as yellow table, portrait skin, etc.), and exclude the corresponding statistical points to improve the accuracy of white balance; The disadvantage is that AI segmentation needs to consider additional performance / power consumption overhead, needs to ensure the stability of the timing segmentation result, is not friendly to video tasks, and cannot improve the white balance effect problem of less statistical points falling in the white area. (2) traditional AWB+AIAWB: using a convolutional neural network (CNN) model to estimate the white point and color temperature of the RGB image, its advantages are: it can effectively improve the accuracy of white balance in all scenes and reduce the probability of white balance error; The disadvantage is: since the CNN model inputs a semantic RGB image (different from the white point-color temperature calibration method, which uses statistical point distribution to estimate the white point), slight changes in image information (color, brightness, lens movement, etc.) may introduce video white balance flicker, that is, this method can be applied to the shooting scene, but the model does not have cross-sensor generalization ability, and does not consider the AIAWB video stream time stability problem, cannot be applied to video stream tasks, and uses a single weight model, which is difficult to achieve optimal performance in all scenes.

[0040] In view of the above problems, in the related art, the method for processing AWB time domain stability mainly includes: using adaptive mean filtering or trimmed-mean filtering algorithm for filtering processing. For traditional AWB methods similar to statistical analysis white point, the time domain stability performance in conventional scenes is acceptable, and using such filtering algorithm can realize the stability of video white balance effect, but cannot effectively solve the time domain stability and convergence timeliness problem of AIAWB.

[0041] Therefore, in view of at least one of the problems in the above related technologies, the present application proposes an image processing method, device, electronic equipment, chip and storage medium.

[0042] The image processing method, device, electronic device, chip and storage medium provided in the embodiments of the present application are described below with reference to the drawings. Before the embodiments of the present application are specifically described, for the convenience of understanding, first, the commonly used technical terms are introduced:

[0043] Bayer format is a color filter array format used for digital image sensors. It captures color images by placing a color filter, usually one of red (R), green (G), or blue (B), over each pixel.

[0044] RGB format is an additive color model that represents various colors by combining different intensities of red (R), green (G), and blue (B) colors.

[0045] Figure 1 The flowchart of the first image processing method provided in the embodiments of the present application.

[0046] It should be noted that the image processing method of the embodiments of the present application can be applied to an image processing device, which in some possible embodiments can be configured in an electronic device or a chip, so that the electronic device or the chip can perform image processing functions. In addition, in some possible embodiments, the image processing device can also be software in an electronic device, etc.

[0047] In any one of the embodiments of the present application, the chip can be integrated into an electronic device. The chip includes a graphics processing unit (GPU), a neural processing unit (NPU), a central processing unit (CPU), an image signal processing (ISP), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a system on chip (SOC), a reduced instruction set computer (RISC), etc., which are not listed one by one.

[0048] The electronic device includes a terminal, a camera, and the like having a photographing function and an image processing function. The terminal is an entity for receiving or transmitting a signal on the user side, such as a mobile phone. The terminal can also be referred to as a terminal device (terminal), a user equipment (UE), a mobile station (MS), a mobile terminal (MT), and the like. The terminal can be a car, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, and the like. Embodiments of the present application do not limit the specific technology and specific device form of the terminal.

[0049] As shown in Figure 1 The image processing method can include the following steps S101-S103:

[0050] Step S101, in response to a photographing operation, a first photographing image is acquired.

[0051] The photographing operation includes a photographing operation, a preview operation, and a video recording operation.

[0052] The first photographing image includes, but is not limited to, an original photographing image collected by an image sensor, an image processed after the original photographing image.

[0053] In the embodiments of the present application, in the case where it is monitored that the user triggers the photographing operation, the first photographing image can be acquired, wherein the first photographing image has a corresponding photographing scene.

[0054] Step S102, in response to the photographing scene corresponding to the first photographing image being the same as the photographing scene corresponding to the reference image, the known white point estimation result corresponding to the reference image is taken as the first target white point estimation result corresponding to the first photographing image.

[0055] The reference image refers to an image whose white point estimation result is known or has been explicitly determined.

[0056] Exemplarily, in a case that the first photographed image is a non-first photographed image, the reference image can be determined according to a previous photographed image of the first photographed image. For example, the previous photographed image can be converted to obtain the reference image. Exemplarily, the image formats of the first photographed image and the previous photographed image include a Bayer format, and the image format of the reference image includes an RGB format.

[0057] It can be understood that, in a case that the first photographed image is a non-first photographed image, the reference image is determined according to the previous photographed image, so as to determine the white point estimation result of the current photographed image based on the known white point estimation result corresponding to the reference image, which can ensure smooth transition of white balance adjustment and avoid visual jumping caused by frequent changes. The reason is that, if white point estimation is performed for each frame, the white balance may change frequently, which may further cause image flicker.

[0058] Exemplarily, in a case that the first photographed image is a first photographed image, the reference image can be determined according to a set image. For example, in a case that the image format of the set image is an RGB format, the set image can be directly used as the reference image, or the set image can be processed to obtain the reference image, which is not limited in the embodiments of the present application.

[0059] The set image is a pre-set image, for example, the set image can be a blank image, a default image, etc.

[0060] It should be noted that, in a case that the first photographed image is a first photographed image, the reference image is determined according to the set image, so as to determine the white point estimation result of the current photographed image based on the known white point estimation result corresponding to the reference image, which can provide a simple and effective initialization method, which ensures that even if there is no historical data for reference, there is a reasonable starting point for subsequent white balance adjustment, and the consistency and stability of the white balance are maintained.

[0061] In the embodiments of the present application, first, whether the shooting scene corresponding to the first shooting image and the shooting scene corresponding to the reference image are the same can be determined based on an image feature extraction method (such as a Scale-Invariant Feature Transform (SIFT), a Speeded-Up Robust Features (SURF), an Oriented FAST and Rotated BRIEF (Binary Robust Independent Elementary Features, ORB), or the like), a histogram statistics method, or the like.

[0062] As an example, taking the ORB algorithm to determine whether the shooting scene corresponding to the first shooting image and the shooting scene corresponding to the reference image are the same as an example, the implementation principle of the ORB algorithm can include the following steps a to d:

[0063] Step a: fast (FAST) corner detection: using the FAST algorithm to detect each first corner point in the second shooting image and each second corner point in the reference image.

[0064] The second shooting image is obtained by performing format conversion on the first shooting image, and the image format of the second shooting image is the same as that of the reference image. For example, the image format of the first shooting image includes a Bayer format, and the image format of the second shooting image includes an RGB format.

[0065] The number of the first corner points and the second corner points can be multiple.

[0066] Step b: feature direction estimation: in order to make the ORB have rotation invariance, the direction of each first corner point and the direction of each second corner point can be estimated by a gray centroid method.

[0067] Step c: BRIEF feature description: according to the direction of each first corner point, a first feature descriptor corresponding to the first shooting image is generated, and according to the direction of each second corner point, a second feature descriptor corresponding to the reference image is generated.

[0068] Step d: feature matching: the first feature descriptor and the second feature descriptor are matched to determine the similarity between the first feature descriptor and the second feature descriptor. If the similarity is higher than a threshold, it is determined that the shooting scene corresponding to the first shooting image and the shooting scene corresponding to the reference image are the same. If the similarity is less than or equal to the threshold, it is determined that the shooting scene corresponding to the first shooting image and the shooting scene corresponding to the reference image are different.

[0069] In the embodiment of the present application, in the case where it is judged that the shooting scene corresponding to the first shooting image is the same as the shooting scene corresponding to the reference image, the known white point estimation result corresponding to the reference image can be directly taken as the final white point estimation result corresponding to the first shooting image, which is recorded as the first target white point estimation result in the present application.

[0070] As an example, in the case where the first shooting image is a non-first frame image in shooting, the known white point estimation result corresponding to the previous frame shooting image of the first shooting image can be taken as the known white point estimation result corresponding to the reference image, and the known white point estimation result can be taken as the first target white point estimation result corresponding to the first shooting image.

[0071] As another example, in the case where the first shooting image is a first frame image in shooting, the known white point estimation result corresponding to the setting image can be taken as the known white point estimation result corresponding to the reference image, and the known white point estimation result can be taken as the first target white point estimation result corresponding to the first shooting image.

[0072] In step S103, the first target image is obtained by performing AWB correction on the first shooting image according to the first target white point estimation result.

[0073] In the embodiment of the present application, the first target image can be obtained by performing AWB correction on the first shooting image according to the first target white point estimation result.

[0074] As an example, taking the Bayer format image data collected by the image sensor as the first shooting image, the first target white point estimation result can be 2-dimensional data [rg, bg], wherein rg refers to R / G, i.e. the ratio of the intensity of the red (R) signal to the intensity of the green (G) signal; bg refers to B / G, i.e. the ratio of the intensity of the blue (B) signal to the intensity of the green (G) signal, and the first shooting image can be corrected by AWB by using the following formula in the present application:

[0075]

[0076] wherein S R , S Gr , S Gb and S BThe image data in Bayer format collected by the image sensor, R, Gr, Gb and B are the image data after final white balance calibration. In the standard Bayer format, the green (G) pixels are divided into two types: Gr and Gb, which are located between the red (R) and blue (B) pixels respectively, Gr refers to the green pixel adjacent to the red pixel, and Gb refers to the green pixel adjacent to the blue pixel.

[0077] The image processing method provided in the embodiments of the present application can directly use the known white point estimation result corresponding to the reference image as the final white point estimation result corresponding to the first captured image in the case that the shooting scene of the first captured image is the same as that of the reference image, without the need to calculate the white point estimation result corresponding to the first captured image in real time, which not only effectively reduces the power consumption and computing power of the device, but also guarantees the accuracy of the white point estimation, so that the AWB correction is performed on the first captured image based on the accurate white point estimation result, which can significantly improve the white balance effect and overall quality of the image and improve the user's shooting experience.

[0078] The embodiments of the present application provide another image processing method, Figure 2 The flowchart of the second image processing method provided in the embodiments of the present application is shown in FIG. 6.

[0079] It should be noted that the image processing method can be executed alone, or can be executed in combination with any one of the embodiments or the possible implementation manners in the embodiments, or can be executed in combination with any one of the related technologies, and the embodiments of the present application do not limit this.

[0080] As shown in FIG. 7, the image processing method can include the following steps S201 to S204: Figure 2

[0081] Step S201, in response to a shooting operation, a first captured image is acquired.

[0082] It should be noted that the explanation and description of step S201 can be referred to the related description in any one of the embodiments of the present application, which will not be repeated here.

[0083] Step S202, in response to the fact that the shooting scene corresponding to the first captured image is different from the shooting scene corresponding to the reference image, a scene classification is performed on a second captured image to obtain a scene category to which the second captured image belongs.

[0084] The second captured image is obtained by performing format conversion on the first captured image. Exemplarily, the image format of the first captured image includes a Bayer format, and the image format of the second captured image includes an RGB format.

[0085] ​The scene categories include, but are not limited to, a single-color light source, a pure color (such as a large-area green plant scene, a skin color scene), and a regular scene.

[0086] In the embodiments of the present application, in the case where the shooting scene corresponding to the first shooting image is different from the shooting scene corresponding to the reference image, the image classification technology can be used to classify the scene of the second shooting image, to obtain the scene category to which the second shooting image belongs.

[0087] For example, a trained image classification model can be used to classify the scene of the second shooting image, to obtain a classification result, wherein the classification result is used to indicate the scene category to which the second shooting image belongs.

[0088] The image classification model includes, but is not limited to, a convolutional neural network (CNN) model.

[0089] In any one of the embodiments of the present application, the second shooting image can be obtained by using steps A to C:

[0090] Step A: performing lens shading correction and black level reduction processing on the first shooting image to obtain a first intermediate image, to reduce luma shading (luminance non-uniformity), color shading (color non-uniformity), and reduce the influence of dark current on image quality.

[0091] Step B: generating a plurality of channel second intermediate images according to the first intermediate image; wherein the resolution of the second intermediate image is lower than that of the first intermediate image.

[0092] As an example, when the image format of the first shooting image is the Bayer format, and the image format of the second intermediate image is the RGB format, the R channel and the B channel of the first intermediate image can be maintained unchanged, the Gr and Gb channels are averaged to obtain the G channel, and then local average down-sampling is performed on the image to obtain the second intermediate image.

[0093] Step C: performing color space mapping processing on the second intermediate image to obtain the second shooting image.

[0094] In the embodiments of the present application, the second intermediate image can be mapped to the color space corresponding to the specified reference image sensor to obtain the second shooting image.

[0095] As an example, a color transformation matrix (CTM) associated with the target image sensor that captures the first captured image can be obtained; wherein the color transformation matrix is used to map the color space of the captured image of the target image sensor to the corresponding color space of the reference image sensor, so that in this application, the second intermediate image can be mapped to the corresponding color space of the reference image sensor based on the color transformation matrix, to obtain the second captured image.

[0096] In summary, the color space mapping or conversion processing of the image can realize the color space alignment of the captured images collected by different image sensors, that is, alignment to the color space corresponding to the same reference image sensor, so that the subsequent AIAWB model has the generalization ability across image sensors (sensors), and the white point consistency of the captured images across sensors is improved.

[0097] In any one embodiment of the present application, the color transformation matrix associated with the target image sensor is obtained in the following manner: the pixel values of a plurality of color cards of the reference image sensor are calculated according to the spectral response sensitivity of the reference image sensor, the spectral reflectance of the standard plurality of color cards under a constant light source, and the spectral power density of the light source; and the pixel values of a plurality of color cards of the target image sensor are calculated according to the spectral response sensitivity of the target image sensor, the spectral reflectance, and the spectral power density of the light source, so that in this application, the color transformation matrix can be calculated based on the pixel values of the plurality of color cards of the reference image sensor and the pixel values of the plurality of color cards of the target image sensor, wherein the color transformation matrix is used to map the color space of the captured image of the target image sensor to the corresponding color space of the reference image sensor.

[0098] Exemplarily, the image format of the second intermediate image is RGB format, and the plurality of color cards are 24 color cards, and the second intermediate image is marked as RGB small The following formula (2) can be used to calculate the 24 color card pixel values of the reference image sensor:

[0099]

[0100] The following formula (3) is used to calculate the 24 color card pixel values of the target image sensor:

[0101]

[0102] The following formulas (4) and (5) are used to map the second intermediate image RGB small to the corresponding color space of the reference image sensor to obtain the second captured image RGB ctm :

[0103] RGBctm = RGB small • CTM; (4)

[0104]

[0105] wherein, R λ,d is an open-source parameter, representing the spectral reflectance of a standard multiple color card (such as macbeth 24 color card) under a constant light source (such as E light); is a calibration parameter of the reference image sensor, representing the spectral response sensitivity of the reference image sensor in three channels; is a calibration parameter of the target image sensor currently photographed, representing the spectral response sensitivity of the target image sensor in three channels; L λ represents the spectral power density of the constant light source; λ represents the wavelength of the constant light source; CTM is a 3x3 color conversion matrix.

[0106] In summary, based on the multiple color card pixel values of the reference image sensor and the multiple color card pixel values of the target image sensor, the color conversion matrix associated with the target image sensor is calculated, which can improve the accuracy and effectiveness of the calculation result.

[0107] Step S203, based on the scene category, white point estimation is performed on the second photographed image and the third photographed image to obtain a plurality of groups of first white point estimation results.

[0108] The third photographed image is obtained by format conversion on the first photographed image, and the channel number of the third photographed image is less than that of the second photographed image.

[0109] Exemplarily, the resolution of the third image can be lower than that of the second image. For example, the second photographed image can be a 3-channel RGB image, and the third photographed image can be a 2-channel low-resolution image.

[0110] Optionally, in order to reduce luma shading (luminance unevenness), color shading (color unevenness), and reduce the influence of dark current on image quality, in the present application, lens shading correction and black level reduction can also be performed on the first photographed image, and format conversion is performed on the processed first photographed image to obtain the third photographed image.

[0111] As an example, taking the image format of the first captured image as the Bayer format as an example, the first captured image in the Bayer format can be used to generate a third captured image of two channels [R / G, B / G], for example, the R channel and the B channel can be maintained unchanged, the Gr and Gb channels are averaged to obtain the G channel, and then the RGB three channels are used to calculate R / G and B / G, where R / G refers to the ratio of the intensity of the red (R) signal to the green (G) signal; B / G refers to the ratio of the intensity of the blue (B) signal to the green (G) signal.

[0112] In the embodiments of the present application, the second captured image and the third captured image can be respectively subjected to white point estimation based on the scene category, to obtain a plurality of sets of first white point estimation results corresponding to the second captured image and a plurality of sets of white point estimation results corresponding to the third captured image.

[0113] In any one of the embodiments of the present application, the plurality of sets of first white point estimation results can be estimated by the following steps D to E:

[0114] Step D: determining a target AIAWB model adapted to the scene category from a plurality of candidate AIAWB models.

[0115] The candidate AIAWB model includes but is not limited to a base (Base) model adapted to a general scene (or referred to as a Base network), a pure color model adapted to a pure color scene (or referred to as a pure color network), and a monochromatic light source model adapted to a monochromatic light source scene (or referred to as a monochromatic light source network).

[0116] Each candidate AIAWB model is trained using sample images associated with the scene category to which the candidate AIAWB model is adapted.

[0117] Each candidate AIAWB model uses a Mixture of Expert (MoE) structure or MoE strategy design, that is, each candidate AIAWB model includes a plurality of expert networks, and each expert network is used to predict a set of white point estimation results corresponding to an input image.

[0118] Step E: using a plurality of expert networks in the target AIAWB model to respectively perform white point estimation on the second captured image and the third captured image, to obtain a plurality of sets of first white point estimation results corresponding to the second captured image and a plurality of sets of first white point estimation results corresponding to the third captured image.

[0119] The target AIAWB model includes a plurality of expert networks, and each expert network is used to predict a set of white point estimation results corresponding to an input image. For example, the number of expert networks contained in the target AIAWB model is marked as N.

[0120] In the embodiments of the present application, the N expert networks in the target AIAWB model can be used to perform white point estimation on the second and third captured images respectively, to obtain N groups of first white point estimation results corresponding to the second captured image and N groups of first white point estimation results corresponding to the third captured image.

[0121] In summary, using the MoE strategy for white point estimation can improve the overall accuracy of white point estimation and reduce the probability of white balance effect error, thereby improving image quality and improving the user's shooting experience.

[0122] In step S204, the first captured image is corrected for AWB according to the multiple groups of first white point estimation results, to obtain a second target image.

[0123] In the embodiments of the present application, first, the final white point estimation result corresponding to the first captured image can be generated according to the multiple groups of first white point estimation results, which is referred to as the second target white point estimation result in the present application.

[0124] As an example, the multiple groups of first white point estimation results can be fused to obtain the second target white point estimation result corresponding to the first captured image.

[0125] As another example, the multiple groups of first white point estimation results can be averaged and filtered to obtain the second target white point estimation result corresponding to the first captured image.

[0126] For example, the multiple groups of first white point estimation results can be averaged to obtain a second white point estimation result, and the second white point estimation result can be subjected to first time domain filtering to obtain the second target white point estimation result. In this way, the rationality and stability of white point estimation can be improved, and the stability of the white balance effect can be improved.

[0127] The first time domain filtering includes but is not limited to adaptive clustering time domain filtering.

[0128] Then, the first captured image can be corrected for AWB according to the second target white point estimation result, to obtain the first captured image after AWB correction, which is referred to as the second target image in the present application. The implementation principle is similar to that of step S103, and will not be described here.

[0129] The image processing method of the embodiments of the present application can perform white point estimation on the second and third captured images in a targeted manner based on the scene category to which the second captured image belongs, which can improve the accuracy of white point estimation. Furthermore, the final white point estimation result corresponding to the first captured image is generated based on the multiple groups of white point estimation results obtained by estimation, and the first captured image is corrected for AWB, which can improve the accuracy of white point estimation and reduce the probability of white balance effect error.

[0130] The embodiment of the present application provides another image processing method, Figure 3 The flowchart of the third image processing method provided by the embodiment of the present application is shown.

[0131] It should be noted that the image processing method can be executed alone, or can be executed in combination with any embodiment or possible implementation manner in the present application, or can be executed in combination with any technical solution in the related art, and the embodiment of the present application does not make any limitation in this regard.

[0132] As shown in the image processing method, Figure 3 may include the following steps S301-S308:

[0133] Step S301, in response to a shooting operation, a first shooting image is acquired.

[0134] Step S302, in response to the shooting scene corresponding to the first shooting image being different from the shooting scene corresponding to the reference image, the second shooting image is classified according to the scene, and the scene category to which the second shooting image belongs is obtained.

[0135] The second shooting image is obtained by performing format conversion on the first shooting image.

[0136] Step S303, based on the scene category, the second shooting image and the third shooting image are subjected to white point estimation, and a plurality of groups of first white point estimation results are obtained.

[0137] The third shooting image is obtained by performing format conversion on the first shooting image, and the channel number of the third shooting image is less than that of the second shooting image.

[0138] Step S304, the plurality of groups of first white point estimation results are averaged to obtain a second white point estimation result.

[0139] It should be noted that the explanation and description of steps S301-S304 can refer to the related description in any embodiment of the present application, and will not be repeated here.

[0140] Exemplarily, taking an example that the first white point estimation result is predicted by using an AIAWB model (denoted as a target AIAWB model in the present application), the second white point estimation result can be marked as the white point estimation result of the AIAWB corresponding to the first shooting image.

[0141] Step S305, the second white point estimation result is subjected to first time domain filtering processing to obtain a third white point estimation result.

[0142] The first time domain filtering processing includes but is not limited to adaptive clustering time domain filtering processing.

[0143] In any embodiment of this application, the first time-domain filtering process can be implemented using the following steps F to I:

[0144] Step F: Write the second white point estimation result into the first white point queue; wherein, the first white point queue records the white point estimation results corresponding to each frame of images captured continuously during this shooting.

[0145] For example, the white point estimation results corresponding to each frame of captured images recorded in the first white point queue can be the white point estimation results of AIAWB corresponding to each frame of captured images.

[0146] Step G: Obtain the first filtering factor associated with the first time-domain filtering process and the value of the target flag bit; wherein, the target flag bit is used to indicate whether the shooting scene corresponding to the first captured image is different from the shooting scene corresponding to the reference image.

[0147] Step H: Determine the first filter diameter based on the first filter factor and the length of the first white point queue.

[0148] For example, the product of the first filter factor and the length of the first white point queue can be used as the first filter diameter.

[0149] Step 1: Based on the first filter diameter, perform the first time-domain filtering on the first white point queue to obtain the third white point estimation result.

[0150] For example, the first white dot queue is marked as WP. queue The length of the first white dot queue is L. The following formulas (6) to (8) can be used to calculate the length of the first white dot queue WP. queue After performing the first time-domain filtering process, the third white point estimation result WP2 is obtained:

[0151]

[0152] WP q =Sort(WP) queue ,WP diff (7)

[0153]

[0154] Here, α1 and α2 represent filtering factors, which can adjust the number of white points that deviate from the cluster center to be filtered out; Sort() indicates filtering according to WP. diff Size, for WP queue Sort to get WP q For example, you can follow WP diff From small to large, WP queue Perform the sequence to obtain WP q .

[0155] wherein Sence change denotes a target flag, for example, Sence change ==True indicates that the shooting scene corresponding to the first shooting image is different from the shooting scene corresponding to the reference image, corresponding to a motion scene, Sence change ==False indicates that the shooting scene corresponding to the first shooting image is the same as the shooting scene corresponding to the reference image, corresponding to a static scene. Exemplarily, a relatively small filter diameter can be used for the motion scene In this way, the convergence speed of the white balance effect can be accelerated, and a relatively large filter diameter can be used for the static scene A more stable white balance effect can be obtained.

[0156] wherein, denotes the first K white points selected from the WP q , and denotes the first K white points selected from the WP q , and denotes the final white point estimation result of the AIAWB, which is recorded as a third white point estimation result in the present application.

[0157] In summary, the first time domain filtering processing on the second white point estimation result can improve the stability of white point estimation, and further improve the stability of the white balance effect.

[0158] In step S306, the third white point estimation result and a fourth white point estimation result are fused to obtain a fifth white point estimation result, wherein the fourth white point estimation result is obtained by performing white point estimation on the third shooting image.

[0159] In any one of the embodiments of the present application, the reference AWB algorithm can be used to perform white point estimation on the third shooting image to obtain the fourth white point estimation result, and the third white point estimation result and the fourth white point estimation result are fused to obtain the fifth white point estimation result. Exemplarily, taking the first white point estimation result obtained by using the AIAWB model (recorded as a target AIAWB model in the present application) for prediction as an example, the fifth white point estimation result can be marked as the AIAWB+reference AWB white point estimation result (or called fusion white point estimation result) corresponding to the first shooting image.

[0160] wherein the reference AWB algorithm refers to a pre-set AWB algorithm, and exemplarily, the reference AWB algorithm includes a traditional AWB algorithm, such as a white point-color temperature calibration method.

[0161] ​As an example, a fusion weight adapted to a scene category to which the second shot image belongs can be queried to fuse the third white point estimation result and the fourth white point estimation result based on the fusion weight to obtain a fifth white point estimation result. Exemplarily, the third white point estimation result and the fourth white point estimation result can be fused to obtain the fifth white point estimation result WP using the following formulas (9) to (10). fusion :

[0162] WP ictm = WP2·CTM -1 ; (9)

[0163] WP fusion = β*WP ictm + (1-β)*WP1; β∈[0,1] (10)

[0164] wherein WP1 represents the fourth white point estimation result, and β represents the fusion weight, which can be obtained based on the scene category to which the second shot image belongs, wherein the weight table records the correspondence between different scene categories and fusion weights. Optionally, the fusion weights corresponding to different scene categories can be different, for example, a relatively large fusion weight is given for a pure color, monochromatic light source scene, and a relatively small fusion weight is given for a regular scene.

[0165] In step S307, the fifth white point estimation result is subjected to second time domain filtering processing to obtain a second target white point estimation result.

[0166] wherein the second time domain filtering processing includes but is not limited to adaptive Trimmed-mean time domain filtering processing.

[0167] In any one of the embodiments of the present application, the second time domain filtering processing can be implemented using the following steps J to M:

[0168] Step J: write the fifth white point estimation result into a second white point queue; wherein the second white point queue records white point estimation results corresponding to each frame of shot image continuously collected at this time of shooting.

[0169] Exemplarily, the white point estimation results corresponding to each frame of shot image recorded in the second white point queue can be the white point estimation results (or referred to as fusion white point estimation results) of AIAWB+ reference AWB corresponding to each frame of shot image.

[0170] Step K: obtain a second filtering factor associated with the second time domain filtering processing and the value of a target flag; wherein the target flag is used to indicate whether the shooting scene corresponding to the first shot image is different from the shooting scene corresponding to the reference image.

[0171] Step L: Determine the second filter diameter based on the second filter factor and the length of the second white point queue.

[0172] For example, the product of the second filter factor and the length of the second white point queue can be used as the second filter diameter.

[0173] Step M: Based on the second filter diameter, perform a second time-domain filtering process on the second white point queue to obtain the second target white point estimation result.

[0174] For example, the second white dot queue is marked as WP. queue2 The length of the second white dot queue is L2, and the following formula (11) can be used to calculate the length of the second white dot queue WP. queue2 A second time-domain filtering process is performed to obtain the second target white point estimation result WP:

[0175]

[0176] Where δ1 and δ2 represent the filtering factors; Indicates to WP queue2 Filter White point data after the maximum and minimum values, Indicates to WP queue2 Filter White point data after the maximum and minimum values; the final white point estimation result obtained after filtering is WP = [rg, bg].

[0177] In summary, performing a second time-domain filter on the fifth white point estimation result can improve the stability of the white point estimation, thereby improving the stability of the white balance effect.

[0178] Step S308: Based on the estimation result of the white point of the second target, perform AWB correction on the first captured image to obtain the second target image.

[0179] It should be noted that the explanation of step S308 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0180] The image processing method of this application embodiment combines multiple sets of white point estimation results calculated based on the scene category to which the second captured image belongs, and the white point estimation results calculated based on the reference AWB algorithm, to determine the final white point estimation result corresponding to the first captured image. This can improve the white point estimation accuracy and thus obtain a more accurate white balance effect. Furthermore, by using two-stage temporal filtering, the stability and convergence timeliness issues of the white balance effect of the AIAWB + traditional AWB algorithm on the video stream can be effectively solved.

[0181] This application provides another image processing method. Figure 4A flowchart of a fourth image processing method provided in the embodiments of the present application is shown.

[0182] It should be noted that the image processing method can be executed alone, or can be executed in combination with any of the embodiments or possible implementation manners in the embodiments of the present application, or can be executed in combination with any of the technical solutions in the related art, and the embodiments of the present application do not make any limitation in this regard.

[0183] As shown in the fourth image processing method provided in the embodiments of the present application, the image processing method can include the following steps S401 to S406. Figure 4

[0184] Step S401, in response to a shooting operation, acquiring a first shooting image.

[0185] It should be noted that the explanation and description of step S401 can refer to the related description in any of the embodiments of the present application, and will not be repeated here.

[0186] Step S402, acquiring first statistical information obtained by histogram statistics on a second shooting image, and second statistical information obtained by histogram statistics on a reference image.

[0187] The second shooting image is obtained by format conversion on the first shooting image, and the image format of the second shooting image is the same as that of the reference image.

[0188] It should be noted that the explanation and description of the second shooting image and the reference image in the foregoing embodiments are also applicable to this embodiment, and will not be repeated here.

[0189] In the embodiments of the present application, the second shooting image can be subjected to histogram statistics to obtain the first statistical information, and the reference image can be subjected to histogram statistics to obtain the second statistical information.

[0190] As an example, taking the first shooting image as a non-first frame image, such as the nth (n is a positive integer greater than 1) frame image, the first statistical information can be marked as Hist n , and the second statistical information can be marked as Hist n-1 .

[0191] Step S403, acquiring the difference between the first statistical information and the second statistical information.

[0192] The difference between the first statistical information and the second statistical information can include the difference value between the first statistical information and the second statistical information, and the absolute value of the difference value.

[0193] As an example, the difference between the first statistical information and the second statistical information = ABS(Hist n -Hist​n-1 ), where ABS() is an absolute value.

[0194] In step S404, it is judged whether the difference is greater than the scene change threshold parameter. If yes, steps S405 to S407 are executed. If no, steps S408 to S409 are executed.

[0195] where the scene change threshold parameter is a pre-set parameter. Exemplarily, the scene change threshold parameter is marked as Thr diff .

[0196] It should be noted that steps S405 to S407 and steps S408 to S409 are two parallel implementation manners. In actual application, only one of them is executed.

[0197] In step S405, it is determined that the shooting scene corresponding to the first shooting image is different from the shooting scene corresponding to the reference image, and the second shooting image is subjected to scene classification to obtain a scene category to which the second shooting image belongs.

[0198] where the second shooting image is obtained by performing format conversion on the first shooting image.

[0199] In the embodiment of the present application, in the case where the difference between the first statistical information and the second statistical information is greater than the scene change threshold parameter, it can be determined that the shooting scene corresponding to the first shooting image is different from the shooting scene corresponding to the reference image. At this time, the second shooting image can be subjected to scene classification based on image classification technology to obtain a scene category to which the second shooting image belongs.

[0200] As an example, it can be judged whether the following formula is true. If yes, Sence change is True, it indicates that the shooting scene corresponding to the first shooting image is different from the shooting scene corresponding to the reference image (i.e., the shooting scene changes):

[0201] Sence change =∑ABS(Hist n -Hist n-1 )>Thr diff ; (12)

[0202] In step S406, white point estimation is performed on the second shooting image and the third shooting image based on the scene category to obtain a plurality of groups of first white point estimation results.

[0203] where the third shooting image is obtained by performing format conversion on the first shooting image, and the channel number of the third shooting image is less than the channel number of the second shooting image.

[0204] Step S407: performing AWB correction on the first captured image according to the multiple sets of first white point estimation results to obtain a second target image.

[0205] It should be noted that the explanation and description of steps S405 to S407 can refer to the related description in any embodiment of the present application, and will not be repeated here.

[0206] In any embodiment of the present application, the first target image can also be rendered for display. For example, if the image format of the first target image is Bayer format, the image format of the first target image can be converted to RGB format, and then image rendering is performed for display. In this way, the actual shooting and display requirements of the user can be met.

[0207] Step S408: determining that the shooting scene corresponding to the first captured image is the same as the shooting scene corresponding to the reference image, and taking the known white point estimation result corresponding to the reference image as the first target white point estimation result corresponding to the first captured image.

[0208] In the embodiments of the present application, in the case where the difference between the first statistical information and the second statistical information is less than or equal to the scene change threshold parameter, it can be determined that the shooting scene corresponding to the first captured image is the same as the shooting scene corresponding to the reference image. At this time, the known white point estimation result corresponding to the reference image can be taken as the first target white point estimation result corresponding to the first captured image.

[0209] As an example, it can be judged whether the following formula is established or not. If not, i.e., Sence change is False (false), it indicates that the shooting scene corresponding to the first captured image is the same as the shooting scene corresponding to the reference image (i.e., the shooting scene does not change):

[0210] Sence change =∑ABS(Hist n -Hist n-1 )>Thr diff ;

[0211] Step S409: performing AWB correction on the first captured image according to the first target white point estimation result to obtain a first target image.

[0212] It should be noted that the explanation and description of steps S408 to S409 can refer to the related description in any embodiment of the present application, and will not be repeated here.

[0213] In any one of the embodiments of the present application, the second target image can also be rendered based on the second target image for display, for example, in the case of a Bayer format as the image format of the second target image, the image format of the second target image can be converted into an RGB format, and then image rendering is performed for display. Thus, the actual shooting requirements and display requirements of the user can be met.

[0214] The image processing method of the embodiments of the present application can quickly detect whether the shooting scene of two frames of images changes without complex image analysis, improve the detection efficiency, and the calculation of the histogram statistical information has lower requirements for computing resources compared to other complex image feature extraction methods, and is suitable for real-time applications.

[0215] In any one of the embodiments of the present application, the technical solutions provided by the present application can be used for automatic white balance calibration in the ISP pipeline (image processing link) of terminal photographing, video recording, etc., to help the object color in the final output image to be close to the color observed by the human eye in the physical world. Compared with the traditional AWB algorithm, more accurate white balance effect can be obtained in the present application; compared with the hybridAWB algorithm, the accuracy of white point estimation of the AWB algorithm can be greatly improved in the present application, and the stability and convergence timeliness of the full-scene white balance time domain effect are considered, the precision of white point estimation is improved, and the computing power is reduced, which can be deployed in terminals with limited computing power.

[0216] As an example, the image format of the first shooting image includes a Bayer format, and the implementation principle of the AWB method provided by the present application can be as shown in Figure 5 , mainly including the following steps:

[0217] Step 1: Preprocess the Bayer image (referred to as the first shooting image in the present application) collected by the target sensor through the image preprocessing module, output a three-channel linear RGB image, and the preprocessing mainly includes the following sub-steps:

[0218] Step 1.1: Correct the lens shadow and reduce the black level of the Bayer image to obtain a preprocessed Bayer image (referred to as the first intermediate image in the present application), so as to reduce luma shading (luminance non-uniformity), color shading (color non-uniformity), and reduce the influence of dark current on the image quality of the input image of the subsequent model;

[0219] Step 1.2: Generate a two-channel low-resolution statistical image UV static (referred to as the third shooting image in the present application) using the preprocessed Bayer image;

[0220] Exemplarily, the size of the UV static may be: 48*64*2; wherein 48 refers to a width component (W), 64 refers to a height component (H), and 2 refers to a channel component (C).

[0221] Step 1.3: generating a low-resolution image RGB small (in this application, referred to as the second intermediate image) in RGB format using the preprocessed Bayer image: the R and B channels remain unchanged, the Gr and Gb channels are averaged to obtain the G channel, and the image is locally averaged and down-sampled to obtain RGB small .

[0222] Exemplarily, the size of the RGB small may be: 192*256*3; wherein 192 refers to a width component (W), 256 refers to a height component (H), and 3 refers to a channel component (C).

[0223] Step 1.4: performing color space mapping processing on the RGB small to obtain an RGB ctm (in this application, referred to as the second captured image), and the main process is as follows:

[0224] (1) selecting a reference sensor from multiple different types of image sensors (sensor), and calculating the 24-color card pixel value of the reference sensor using formula (2).

[0225] (2) calculating the 24-color card pixel value of the target sensor of the current captured image using formula (3).

[0226] (3) using formulas (4) and (5) to map the captured image of the target sensor to the color space corresponding to the reference sensor to obtain RGB ctm .

[0227] Step 2: sending the RGB ctm to a scene change detection module, performing histogram statistics on the RGB ctm in the scene change detection module to obtain Hist n , and determining whether the shooting scene has changed through the following formula:

[0228] Sence change =∑ABS(Hist n -Hist n-1 )>Thr diff ;

[0229] wherein Hist n-1This refers to the histogram statistics of the reference image, in Sense. change When True, it indicates that the shooting scene has changed.

[0230] Step 3: The shooting scene remains unchanged (i.e., the scene remains unchanged). change If the value is False, the known white point estimation result corresponding to the reference image can be used as the white point estimation result corresponding to the Bayer image acquired by the current target sensor.

[0231] Step 4: When the shooting scene changes (i.e., Scene) change If the value is True, the UV values ​​in step 1 can be... static The data is then passed to a traditional AWB processing module for white point estimation to obtain WP1. For example, the traditional AWB processing module can use the white point-color temperature calibration method to perform UV calibration. static Perform white point estimation.

[0232] Step 5: Classify RGB using the scene classification module ctm Scenes are classified to obtain scene categories, including but not limited to: monochromatic light source, solid color, and conventional scene.

[0233] Step 6: Combine the three-channel RGB ctm Two-channel UV static The classification results from step 5 are sent to the AIAWB model selection and inference module to perform AIAWB white point estimation and obtain the network inference result wps.

[0234] The AIAWB model uses the MoE structure design, which can output multiple sets of white point estimation results (e.g., 3 sets, the data dimension of WPS is [3,2]). Using the MoE structure can effectively improve the overall accuracy of white point estimation and reduce the probability of white balance errors.

[0235] In this application, a target AIAWB model that matches the scene category indicated by the classification results can be selected from multiple candidate AIAWB models based on the classification results. Then, multiple expert networks within the target AIAWB model are used to analyze the RGB data. ctm and UV static Perform white point estimation to obtain multiple sets of white point estimation results (wps).

[0236] Step 7: Pass the WPS data to the timing white point queue recording module, calculate the average of multiple WPS data sets, and obtain the WP data. mean The WP mean This is the white point estimation result of AIAWB, and a queue of length L (referred to as the first white point queue in this application) is used to record the AIAWB white point estimation result of each video frame in the video stream, denoted as WP. queue .

[0237] Step 8: using the adaptive clustering time domain filtering processing module to WP queue Smooth processing, combined with Sence change State to do the effect of scene change after fast convergence.

[0238] Since the AIAWB model training uses a photograph image, the time domain stability of its white point estimation is usually inferior to that of the white point statistical analysis method in the traditional AWB algorithm. For example, for the AIAWB Base model, in order to obtain relatively optimal white point estimation accuracy in a regular scene, the training data will remove sample images of pure color and single color light source scenes, and such removal will make the Base model have poor time domain white point estimation stability in pure color and single color light source scenes. To solve the above problem, in the present application, multiple model weights can be used to cope with white point estimation in different scenes, so as to balance the high precision and time domain stability of the model.

[0239] In the present application, the stability of the white point estimation result can also be improved by using an adaptive clustering time domain filtering processing method:

[0240]

[0241] WP q = Sort(WP queue , WP diff );

[0242]

[0243] Step 9: the white point estimation result WP1 output by the traditional AWB processing module and the white point estimation result WP2 output by the adaptive clustering time domain filtering processing module are transmitted to the white point fusion processing module for fusion:

[0244] WP ictm = WP2·CTM -1 ;

[0245] WP fusion = β*WP ictm + (1-β)*WP1; β∈[0,1]

[0246] Step 10: the fusion result WP fusion is transmitted to the adaptive Trimmed-mean time domain filtering processing module for secondary time domain filtering:

[0247] Wherein, the Trimmed-mean filtering refers to calculating the average value of the remaining data after removing a certain proportion of the maximum and minimum data from the data.

[0248] First, the WP of each video frame in the video stream is stored using a second white point queue with a length of L2 fusion , to obtain WP queue2 , and then, the WP queue2 may be subjected to Trimmed-mean filtering according to Sence change :

[0249]

[0250] wherein WP = [rg, bg].

[0251] Step 11: using the calculated white point estimation result WP, the Bayer image input by the target sensor is subjected to white balance correction:

[0252]

[0253] wherein S R , S Gr , S Gb and S B represent the Bayer image collected by the target sensor, and R, Gr, Gb and B are the Bayer image after final white balance calibration.

[0254] In summary, the technical solution provided in the present application has at least the following advantages over the related art:

[0255] 1. The image pre-processing module performs color space conversion on the image, so that the AIAWB model has cross-sensor generalization capability, that is, color space alignment is performed using a CTM matrix, and only the spectral response information of the sensor needs to be calibrated, so that the AIAWB model can be quickly generalized to other cameras and sensors, greatly reducing the time and labor cost of image data collection and model retraining for new sensors.

[0256] As an example, the applicant tested the captured images of multiple sensors, and the test results are shown in Figure 6 and Figure 7 . After color space alignment, the white point consistency of the cross-sensor image data is significantly improved. Among them, Figure 6 and Figure 7 the horizontal axis represents R / G, and the vertical axis represents B / G.

[0257] 2. Compared with the traditional AWB algorithm, the technical solution provided in the present application can obtain more accurate white balance effect.

[0258] 3、The AIAWB model adopts MoE structure and a multi-model strategy, outputs multiple sets of white point estimation results, is beneficial to reduce the probability of white balance error, and obtains better white point estimation accuracy.

[0259] 4、Different AIAWB models and fusion weights are used for different scenes, the semantic ambiguous sample problem is avoided by training data, and each model can obtain better white point estimation accuracy.

[0260] 5、An adaptive clustering time domain filtering processing method is proposed, and adaptive Trimmed-mean time domain filtering is used for the fused white point estimation result, through the two-stage time domain filtering, the white balance effect stability and convergence timeliness problems of the AIAWB+ traditional AWB algorithm in the video stream can be effectively solved.

[0261] 6、A scene change detection module is used to monitor whether the shooting scene changes or not, so as to reduce the scheduling frequency of the AIAWB model and the traditional AWB algorithm, and the algorithm deployment power consumption can be effectively reduced.

[0262] In order to realize the above-mentioned embodiments, an image processing device is further provided by the embodiments of the present application.

[0263] Figure 8 A structural schematic diagram of an image processing device provided by the embodiments of the present application.

[0264] As Figure 8 shown, the image processing device 800 can include an acquisition module 810, a processing module 820, and a correction module 830.

[0265] The acquisition module 810 is configured to acquire a first shooting image in response to a shooting operation.

[0266] The processing module 820 is configured to, in response to the shooting scene corresponding to the first shooting image being the same as the shooting scene corresponding to the reference image, take the known white point estimation result corresponding to the reference image as a first target white point estimation result corresponding to the first shooting image.

[0267] The correction module 830 is configured to perform white balance AWB correction on the first shooting image according to the first target white point estimation result, to obtain a first target image.

[0268] Further, in an implementation form of the embodiment of the application, the obtaining module 810 is further configured to: in response to the first photographed image being a non-first photographed image, taking the known white point estimation result corresponding to a previous frame photographed image of the first photographed image as the known white point estimation result corresponding to the reference image; wherein the reference image is determined according to the previous frame photographed image; in response to the first photographed image being a first photographed image, taking the known white point estimation result corresponding to the set image as the known white point estimation result corresponding to the reference image; wherein the reference image is determined according to the set image.

[0269] In an implementation form of the embodiment of the application, the image processing apparatus 800 can further include:

[0270] The classification module is configured to: in response to the photographed scene corresponding to the first photographed image being different from the photographed scene corresponding to the reference image, perform scene classification on the second photographed image to obtain a scene category to which the second photographed image belongs; wherein the second photographed image is obtained by performing format conversion on the first photographed image.

[0271] The estimation module is configured to: based on the scene category, perform white point estimation on the second photographed image and a third photographed image to obtain a plurality of groups of first white point estimation results; wherein the third photographed image is obtained by performing format conversion on the first photographed image, and the number of channels of the third photographed image is less than the number of channels of the second photographed image.

[0272] The correction module 830 is further configured to: based on the plurality of groups of first white point estimation results, perform AWB correction on the first photographed image to obtain a second target image.

[0273] In an implementation form of the embodiment of the application, the estimation module is configured to: from a plurality of candidate artificial intelligence (AI) AWB models, determine a target AI AWB model adapted to the scene category; and use a plurality of expert networks in the target AI AWB model to respectively perform white point estimation on the second photographed image and the third photographed image to obtain a plurality of groups of first white point estimation results corresponding to the second photographed image and a plurality of groups of first white point estimation results corresponding to the third photographed image.

[0274] In an implementation form of the embodiment of the application, the correction module 830 is configured to: average the plurality of groups of first white point estimation results to obtain a second white point estimation result; perform first time domain filtering processing on the second white point estimation result to obtain a second target white point estimation result corresponding to the first photographed image; and perform AWB correction on the first photographed image according to the second target white point estimation result to obtain a second target image.

[0275] In an implementation form of the embodiment of the application, the correction module 830 is configured to: perform first time domain filtering processing on the second white point estimation result to obtain a third white point estimation result; perform fusion on the third white point estimation result and a fourth white point estimation result to obtain a fifth white point estimation result, wherein the fourth white point estimation result is obtained by performing white point estimation on the third captured image; and perform second time domain filtering processing on the fifth white point estimation result to obtain the second target white point estimation result.

[0276] In an implementation form of the embodiment of the application, the correction module 830 is configured to: write the second white point estimation result into a first white point queue, wherein the first white point queue records white point estimation results corresponding to each frame of captured images continuously captured at this time of shooting; obtain a first filtering factor associated with the first time domain filtering processing and a value of a target flag, wherein the target flag is used to indicate whether a shooting scene corresponding to the first captured image is different from a shooting scene corresponding to the reference image; determine a first filtering diameter according to the first filtering factor and a length of the first white point queue; and perform the first time domain filtering processing on the first white point queue based on the first filtering diameter to obtain the third white point estimation result.

[0277] In an implementation form of the embodiment of the application, the correction module 830 is configured to: write the fifth white point estimation result into a second white point queue, wherein the second white point queue records white point estimation results corresponding to each frame of captured images continuously captured at this time of shooting; obtain a second filtering factor associated with the second time domain filtering processing and a value of the target flag, wherein the target flag is used to indicate whether the shooting scene corresponding to the first captured image is different from the shooting scene corresponding to the reference image; determine a second filtering diameter according to the second filtering factor and a length of the second white point queue; and perform the second time domain filtering processing on the second white point queue based on the second filtering diameter to obtain the second target white point estimation result.

[0278] In an implementation form of the embodiment of the application, the image processing apparatus 800 can further include:

[0279] The determination module is configured to: obtain first statistical information obtained by performing histogram statistics on the second captured image, and second statistical information obtained by performing histogram statistics on the reference image; obtain a difference between the first statistical information and the second statistical information; in response to the difference being greater than a scene change threshold parameter, determine that the shooting scene corresponding to the first captured image is different from the shooting scene corresponding to the reference image; and in response to the difference being less than or equal to the scene change threshold parameter, determine that the shooting scene corresponding to the first captured image is the same as the shooting scene corresponding to the reference image.

[0280] In an implementation form of the embodiment of the application, the obtaining module 810 is further configured to: perform lens shading correction and black level reduction processing on the first captured image to obtain a first intermediate image; generate a plurality of channel second intermediate images according to the first intermediate image; wherein the resolution of the second intermediate image is lower than that of the first intermediate image; and perform color space mapping processing on the second intermediate image to obtain the second captured image.

[0281] In an implementation form of the embodiment of the application, the obtaining module 810 is configured to: obtain a color conversion matrix associated with a target image sensor that captures the first captured image; wherein the color conversion matrix is used to map the color space of the captured image of the target image sensor to the color space corresponding to the reference image sensor; and map the second intermediate image to the color space corresponding to the reference image sensor based on the color conversion matrix to obtain the second captured image.

[0282] In an implementation form of the embodiment of the application, the obtaining module 810 is configured to: obtain a plurality of color card pixel values of the reference image sensor according to the spectral response sensitivity of the reference image sensor, the spectral reflectance of the standard plurality of color cards under the constant light source, and the spectral power density of the light source; obtain a plurality of color card pixel values of the target image sensor according to the spectral response sensitivity of the target image sensor, the spectral reflectance, and the spectral power density of the light source; and determine the color conversion matrix based on the plurality of color card pixel values of the reference image sensor and the plurality of color card pixel values of the target image sensor.

[0283] In an implementation form of the embodiment of the application, the image processing apparatus 800 can further include:

[0284] The rendering module is configured to perform any one of the following: rendering based on the first target image for display; and rendering based on the second target image for display.

[0285] It should be noted that the foregoing explanation of the embodiment of the image processing method is also applicable to the image processing apparatus of the embodiment, which will not be described here.

[0286] In the image processing apparatus of the embodiment of the application, in the case where the shooting scene of the first captured image is the same as that of the reference image, the known white point estimation result corresponding to the reference image can be directly used as the final white point estimation result corresponding to the first captured image, without the need for real-time calculation of the white point estimation result corresponding to the first captured image. This not only effectively reduces the power consumption and computing power of the device, but also ensures the accuracy of the white point estimation. Therefore, based on the accurate white point estimation result, the AWB correction is performed on the first captured image, which can significantly improve the white balance effect and overall quality of the image and improve the user's shooting experience.

[0287] To achieve the above-mentioned embodiments, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image processing method according to any one of the preceding embodiments.

[0288] Figure 9 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 9. For example, the electronic device 900 can be a vehicle, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0289] Referring to Figure 9 , the electronic device 900 can include one or more of the following components: a processing component 902, a memory 904, a power component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0290] The processing component 902 usually controls overall operations of the electronic device 900, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 902 can include one or more processors 920 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 902 can include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0291] The memory 904 is configured to store various types of data to support operations of the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0292] The power component 906 provides power to various components of the electronic device 900. The power component 906 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 900.

[0293] The multimedia component 908 includes a screen providing an output interface between the electronic device 900 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 908 includes a front camera and / or a back camera. When the electronic device 900 is in an operation mode, such as a photographing mode or a video mode, the front camera and / or the back camera can receive an external multimedia data. Each of the front and back cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0294] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC) that is configured to receive an external audio signal when the electronic device 900 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0295] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0296] The sensor component 914 includes one or more sensors for providing status assessments of various aspects of the electronic device 900. For example, the sensor component 914 can detect an open / closed position of the electronic device 900, relative positioning of components, such as a display and a keypad of the electronic device 900, a change in position of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and a temperature change of the electronic device 900. The sensor component 914 can include a proximity sensor configured to detect the presence of a nearby object without any physical touch. The sensor component 914 can also include a light sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, utilized in an imaging application. In some embodiments, the sensor component 914 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0297] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 916 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 916 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0298] In an example embodiment, the electronic device 900 can be implemented with one or more Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described methods.

[0299] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 904 including instructions, is also provided, which can be executed by the processor 920 of the electronic device 900 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0300] To achieve the above-mentioned embodiments, the present application further provides a chip, wherein the chip comprises an interface circuit and a processing circuit coupled with each other, the interface circuit is configured to input or output a signal, and the processing circuit is configured to perform the image processing method provided in any one of the above-mentioned embodiments.

[0301] Figure 10 FIG. 1 is a structural schematic diagram of a chip according to an embodiment of the present application. As shown in FIG. 1, the chip 1000 can include a processing circuit 1001 and one or more interface circuits 1002. Figure 10 The chip 1000 can include a processing circuit 1001 and one or more interface circuits 1002.

[0302] The chip 1000 can include a processing circuit 1001 and one or more interface circuits 1002.

[0303] In some embodiments, the chip 1000 further includes one or more interface circuits 1002. Optionally, the interface circuit 1002 is connected with the memory 1003, and the interface circuit 1002 can be configured to receive a signal from the memory 1003 or other devices, and the interface circuit 1002 can be configured to send a signal to the memory 1003 or other devices. For example, the interface circuit 1002 can read an instruction stored in the memory 1003 and send the instruction to the processing circuit 1001.

[0304] In some embodiments, the interface circuit 1002 performs at least one of the communication steps such as sending and / or receiving in the above-mentioned method, and the processing circuit 1001 performs other steps.

[0305] In some embodiments, the terms such as interface circuit, interface, transceiver pin, and transceiver can be replaced with each other.

[0306] In some embodiments, the chip 1000 further includes one or more memories 1003 for storing instructions. Optionally, all or part of the memory 1003 can be outside the chip 1000.

[0307] To achieve the above-mentioned embodiments, the present application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image processing method according to any one of the above-mentioned method embodiments.

[0308] To achieve the above-mentioned embodiments, the present application further provides a computer program product, which stores a computer program, and the computer program is executed by a processor to implement the image processing method according to any one of the above-mentioned method embodiments.

[0309] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. Moreover, the usage of the terms "first", "second" or "third" does not limit the quantity or order of the specific features, structures, materials or characteristics, but rather the term "first", "second" or "third" can be used to distinguish different features, structures, materials or characteristics, which can be combined in any suitable manner. Furthermore, the singular forms "a", "an" and "the" include plural references unless the context clearly dictates otherwise.

[0310] Furthermore, the terms "first", "second", or the like, merely denote different instances of a similar feature, structure, material or characteristic, without necessarily implying any relative importance or any particular order. Thus, a feature defined with "first" or "second" can implicitly or explicitly include at least one of the features. The meaning of "a", "an" and "the" includes plural references unless the context clearly dictates otherwise.

[0311] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more steps for implementing specific logic functions or steps, and the terms in the description are used for causing or carrying out or upgrading of an action between other hardware under their control. The description of processes and methods of operations should be considered as merely illustrative of the principles of the application.

[0312] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (e.g., magnetic, optical or other) a machine-readable storage diskette (e.g., floppy disk, optical disk, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), optical fibers, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium can even be paper or other suitable medium upon which the program is printed, as the program can be electronically captured, for example via the optical scanner of a device or other electronic capture device, and then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0313] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in some embodiments, the steps or methods can be implemented in a combination of hardware and software. If implemented in hardware, as in another embodiment, any of the above techniques can be implemented with or without the use of a programmable digital signal processor (DSP) or other programmable device. In some embodiments, the steps or methods can be implemented using a combination of different hardware devices.

[0314] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0315] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0316] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: in response to a shooting operation, acquiring a first shooting image; in response to the shooting scene corresponding to the first shooting image being the same as the shooting scene corresponding to a reference image, estimating a known white point corresponding to the reference image as a first target white point estimation result corresponding to the first shooting image; performing white balance (AWB) correction on the first shooting image according to the first target white point estimation result to obtain a first target image. The method further comprises: in response to the shooting scene corresponding to the first shooting image being different from the shooting scene corresponding to the reference image, performing scene classification on a second shooting image to obtain a scene category to which the second shooting image belongs; wherein the second shooting image is obtained by performing format conversion on the first shooting image; based on the scene category, performing white point estimation on the second shooting image and a third shooting image to obtain a plurality of groups of first white point estimation results; wherein the third shooting image is obtained by performing format conversion on the first shooting image, and the number of channels of the third shooting image is less than that of the second shooting image; performing AWB correction on the first shooting image according to the plurality of groups of first white point estimation results to obtain a second target image.

2. The method of claim 1, wherein, The reference image and the known white point estimation result are determined by the following steps: in response to the first shooting image being a non-first frame image, estimating a known white point corresponding to a previous frame shooting image of the first shooting image as a known white point estimation result corresponding to the reference image; wherein the reference image is determined according to the previous frame shooting image; in response to the first shooting image being a first frame image, estimating a known white point corresponding to a set image as a known white point estimation result corresponding to the reference image; wherein the reference image is determined according to the set image.

3. The method of claim 1, wherein, The method further comprises: from a plurality of candidate artificial intelligence (AI) AWB models, determining a target AI AWB model suitable for the scene category; using a plurality of expert networks in the target AI AWB model, performing white point estimation on the second shooting image and the third shooting image respectively to obtain a plurality of groups of first white point estimation results corresponding to the second shooting image and a plurality of groups of first white point estimation results corresponding to the third shooting image.

4. The method of claim 1, wherein, The method further comprises: averaging the plurality of groups of first white point estimation results to obtain a second white point estimation result; performing first time domain filtering processing on the second white point estimation result to obtain a second target white point estimation result corresponding to the first shooting image; performing AWB correction on the first shooting image according to the second target white point estimation result to obtain a second target image.

5. The method of claim 4, wherein, The method further comprises: performing first time domain filtering processing on the second white point estimation result to obtain a second target white point estimation result corresponding to the first shooting image. performing first time domain filtering on the second white point estimation result to obtain a third white point estimation result; fusing the third white point estimation result and a fourth white point estimation result to obtain a fifth white point estimation result, wherein the fourth white point estimation result is obtained by performing white point estimation on the third captured image; performing second time domain filtering on the fifth white point estimation result to obtain the second target white point estimation result.

6. The method of claim 5, wherein, The first time domain filtering on the second white point estimation result to obtain a third white point estimation result comprises: writing the second white point estimation result into a first white point queue, wherein the first white point queue records white point estimation results corresponding to each frame of captured image continuously captured at this time of capturing; obtaining a first filtering factor associated with the first time domain filtering and a value of a target flag, wherein the target flag is used to indicate whether a capturing scene corresponding to the first captured image is different from a capturing scene corresponding to the reference image; determining a first filtering diameter according to the first filtering factor and a length of the first white point queue; performing first time domain filtering on the first white point queue based on the first filtering diameter to obtain the third white point estimation result.

7. The method of claim 5, wherein, The second time domain filtering on the fifth white point estimation result to obtain the second target white point estimation result comprises: writing the fifth white point estimation result into a second white point queue, wherein the second white point queue records white point estimation results corresponding to each frame of captured image continuously captured at this time of capturing; obtaining a second filtering factor associated with the second time domain filtering and a value of a target flag, wherein the target flag is used to indicate whether a capturing scene corresponding to the first captured image is different from a capturing scene corresponding to the reference image; determining a second filtering diameter according to the second filtering factor and a length of the second white point queue; performing the second time domain filtering on the second white point queue based on the second filtering diameter to obtain the second target white point estimation result.

8. The method according to any one of claims 1-7, characterized in that, After the first captured image is obtained in response to the capturing operation, the method further comprises: obtaining first statistical information obtained by performing histogram statistics on a second captured image, and second statistical information obtained by performing histogram statistics on the reference image, wherein the second captured image is obtained by performing format conversion on the first captured image; obtaining a difference between the first statistical information and the second statistical information; in response to the difference being greater than a scene change threshold parameter, determining that a capturing scene corresponding to the first captured image is different from a capturing scene corresponding to the reference image; in response to the difference being less than or equal to a scene change threshold parameter, determining that a capturing scene corresponding to the first captured image is the same as a capturing scene corresponding to the reference image.

9. The method of claim 8, wherein, The second captured image is obtained by the following steps: performing lens shading correction and black level reduction on the first captured image to obtain a first intermediate image; According to the first intermediate image, a plurality of channels of a second intermediate image are generated; wherein the resolution of the second intermediate image is lower than that of the first intermediate image; The second intermediate image is subjected to color space mapping processing to obtain the second shooting image.

10. The method of claim 9, wherein, The color space mapping processing of the second intermediate image to obtain the second shooting image comprises: Obtaining a color conversion matrix associated with a target image sensor that shoots the first shooting image; wherein the color conversion matrix is used to map the color space of the shooting image of the target image sensor to the color space corresponding to a reference image sensor; Based on the color conversion matrix, the second intermediate image is mapped to the color space corresponding to the reference image sensor to obtain the second shooting image.

11. The method of claim 10, wherein, The color conversion matrix associated with the target image sensor that shoots the first shooting image is obtained, comprising: According to the spectral response sensitivity of the reference image sensor, the spectral reflectance of a plurality of standard color cards under a constant light source, and the spectral power density of the light source, a plurality of color card pixel values of the reference image sensor are obtained; According to the spectral response sensitivity of the target image sensor, the spectral reflectance, and the spectral power density of the light source, a plurality of color card pixel values of the target image sensor are obtained; Based on the plurality of color card pixel values of the reference image sensor and the plurality of color card pixel values of the target image sensor, the color conversion matrix is determined.

12. The method according to any one of claims 1-7, characterized in that, The method further comprises any of the following: Based on the first target image, rendering is performed to display; Based on the second target image, rendering is performed to display.

13. An image processing apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire a first shooting image in response to a shooting operation; A processing module is configured to, in response to the shooting scene corresponding to the first shooting image being the same as the shooting scene corresponding to a reference image, estimate a known white point of the reference image as a first target white point estimation result corresponding to the first shooting image; A correction module is configured to perform AWB correction on the first shooting image based on the first target white point estimation result to obtain a first target image. The device further comprises: A classification module is configured to, in response to the shooting scene corresponding to the first shooting image being different from the shooting scene corresponding to the reference image, perform scene classification on a second shooting image to obtain a scene category to which the second shooting image belongs; wherein the second shooting image is obtained by format conversion on the first shooting image; An estimation module is configured to perform white point estimation on the second shooting image and a third shooting image based on the scene category to obtain a plurality of first white point estimation results; wherein the third shooting image is obtained by format conversion on the first shooting image, and the number of channels of the third shooting image is less than that of the second shooting image; A correction module is further configured to perform AWB correction on the first shooting image based on the plurality of first white point estimation results to obtain a second target image.

14. An electronic device, comprising: A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 12 when executing the program.

15. A chip, characterized by The chip comprises an interface circuit and a processing circuit coupled to each other, the interface circuit is used for inputting or outputting signals, and the processing circuit is used for implementing the method according to any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 12.

Images