Image processing method, image processing device, electronic device, medium, and chip

Abstract

This disclosure relates to an image processing method, image processing apparatus, electronic device, medium, and chip. The method includes: acquiring an image to be processed, the image to be processed including at least one region, each region having a corresponding gray confidence level, the gray confidence level being used to characterize the probability that the region is a gray region; estimating the white point of the image to be processed using at least two reference white balance algorithms based on the gray confidence level of the at least one region, obtaining at least two sets of first white point estimation results; determining a target white point estimation result based on the at least two sets of first white point estimation results; and performing white balance correction on the image to be processed based on the target white point estimation result to obtain a target image. This disclosure can improve the correction effect, thereby improving the accuracy of restoring the true colors of the image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing apparatus, an electronic device, a medium, and a chip. Background Technology

[0002] In real life, lighting environments are extremely complex and variable. Indoor lighting, due to the characteristics of the lamps and the usage scenario, has a narrow color temperature distribution and is often warm; outdoor sunlight, however, varies greatly in color temperature over time and with changing weather conditions, generally tending towards cool. This significant difference in color temperature between indoor and outdoor lighting can cause image sensors to misperceive colors, resulting in color casts in captured images and an inability to accurately represent the colors of the scene. In the field of image processing, white balance correction serves to eliminate the interference of different lighting conditions on image colors, ensuring natural and realistic color reproduction.

[0003] Therefore, how to effectively improve its correction effect in order to improve the accuracy of restoring the true colors of the image has become a key problem that urgently needs to be solved in the current image field. Summary of the Invention

[0004] This disclosure provides an image processing method, an image processing apparatus, an electronic device, a medium, and a chip. This disclosure can improve the correction effect and increase the accuracy of restoring the true colors of the image.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0006] Obtain an image to be processed; wherein the image to be processed includes at least one region, each region has a corresponding gray confidence level, and the gray confidence level is used to characterize the probability that the region is a gray region;

[0007] Based on the gray confidence of the at least one region, at least two reference white balance algorithms are used to estimate the white point of the image to be processed, and at least two sets of first white point estimation results are obtained.

[0008] Based on the at least two sets of first white point estimation results, determine the target white point estimation result;

[0009] Based on the target white point estimation result, white balance correction is performed on the image to be processed to obtain the target image.

[0010] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0011] An acquisition module is used to acquire an image to be processed; wherein the image to be processed includes at least one region, each region has a corresponding gray confidence level, and the gray confidence level is used to characterize the probability that the region is a gray region;

[0012] The first determining module is used to perform white point estimation on the image to be processed using at least two reference white balance algorithms based on the gray confidence of the at least one region, and obtain at least two sets of first white point estimation results.

[0013] The second determining module is used to determine the target white point estimation result based on the at least two sets of first white point estimation results;

[0014] The processing module is used to perform white balance correction on the image to be processed based on the target white point estimation result, so as to obtain the target image.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the image processing method of the first aspect embodiment.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the image processing method of the first aspect embodiment.

[0017] According to a fifth aspect of the present disclosure, a chip is provided, the chip including a processing circuit for implementing the image processing method described in any one of the first aspects. Exemplarily, the chip includes an image processing chip (ISP).

[0018] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0019] The image processing method, image processing apparatus, electronic device, medium, and chip of this disclosure first acquire an image to be processed, which includes at least one region. Each region has a corresponding gray confidence level, which is used to characterize the probability that the region is a gray region. Based on the gray confidence level of the at least one region, at least two reference white balance algorithms are used to estimate the white point of the image to be processed, resulting in at least two sets of first white point estimation results. Then, based on the at least two sets of first white point estimation results, a target white point estimation result is determined. Based on the target white point estimation result, white balance correction is performed on the image to be processed to obtain a target image, thereby improving the correction effect and enhancing the accuracy of restoring the true colors of the image.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0022] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0023] Figure 2 This is a schematic diagram of gray confidence levels in different regions according to an embodiment of the present disclosure;

[0024] Figure 3 This is a schematic diagram of gray confidence and region weight under a gray-scale world algorithm according to an embodiment of the present disclosure;

[0025] Figure 4 This is a flowchart illustrating multi-round white balance correction of an image to be processed according to an embodiment of the present disclosure;

[0026] Figure 5 This is a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] The image processing method, image processing apparatus, electronic device, medium, and chip of the present disclosure are described below with reference to the accompanying drawings.

[0031] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0032] It should be noted that the image processing method of this disclosure can be applied to an image processing device. In some possible embodiments, the image processing device can be configured in an electronic device or a chip so that the electronic device or chip can perform image processing functions.

[0033] Electronic devices include, but are not limited to, vehicles, servers, and terminals. A terminal is a user-side entity used to receive or transmit signals, such as a mobile phone. Terminals can also be called terminal equipment (UE), user equipment (UE), mobile station (MS), mobile terminal (MT), etc. Terminals can be vehicles with communication capabilities, intelligent vehicles, mobile phones, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminals, augmented reality (AR) terminals, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0034] like Figure 1 As shown, the image processing method of this disclosure includes the following steps:

[0035] Step S101: Obtain the image to be processed; wherein the image to be processed includes at least one region, and each region has a corresponding gray confidence level, which is used to characterize the probability that the region is a gray region.

[0036] In this embodiment, the method for acquiring the image to be processed includes:

[0037] In response to the shooting operation, the original captured image is acquired;

[0038] Use the original captured image as the image to be processed; or,

[0039] The original captured image is processed to obtain the image to be processed.

[0040] For example, when a user triggers the shooting function of a camera device (such as a smartphone camera or a professional digital camera), the device's image sensor captures the raw image data of the current scene, thus generating the raw image. This raw image contains complete visual information of the scene at the moment of shooting, and is directly used as the image to be processed. For instance, in an outdoor landscape shooting scenario, when a user presses the shutter button on their phone's camera, the camera immediately acquires the raw image of the current landscape and uses it as the image to be processed. This method can quickly acquire real-time scene image data and is suitable for scenarios with high real-time requirements and where preprocessing of the raw image is not necessary, such as on-site monitoring and instant snapshots.

[0041] After acquiring the original captured images, basic processing is performed on them, including black level correction, dead pixel correction, and lens distortion correction.

[0042] To adapt to images output from different sensors, the versatility and adaptability of this image processing method are improved. After acquiring the image to be processed (either the original captured image or a pre-processed image), the image is converted from a first color space (such as the color space of the RAW image) to a second color space (such as the sRGB color space) to obtain the converted image. This conversion process is implemented through a color space conversion algorithm, ensuring accurate mapping of the image between different color spaces and providing suitable image data for subsequent operations such as region detection and white balance correction.

[0043] Region detection is performed on the transformed image to obtain at least one region and its corresponding gray confidence score. Region detection algorithms can employ image detection algorithms or AI image segmentation algorithms from related technologies. For example, edge-detection-based algorithms can be used to divide the image into different regions by detecting edge information. Simultaneously, a machine learning model is used to analyze each region and calculate the probability that it is a gray region, i.e., the gray confidence score. Detection algorithms can also detect common memory colors such as blue sky, green plants, skin tone, and wood grain, which are equivalent to the probability of non-gray areas. For example, for the blue sky region in the image, the algorithm can identify its features and determine that the probability of this region being non-gray is relatively high. The detection results are presented in image form, with different regions using different labels or colors to represent their gray confidence scores, ultimately resulting in an image containing the gray confidence scores of different regions, such as... Figure 2 As shown.

[0044] To improve computational efficiency, this disclosure proposes two optimization measures after acquiring the image to be processed and before converting it from a first color space to a second color space. First, the image to be processed is downsampled. By reducing the image resolution, the amount of data in subsequent processing is reduced, thereby lowering computational complexity. Second, the originally complex image processing modules (including black level correction, bad pixel correction, and lens distortion correction) are replaced with functionally equivalent but simpler modules. These simplified modules reduce computational steps and resource consumption while ensuring basic processing functions, thus effectively improving overall computational efficiency.

[0045] Step S102: Based on the gray confidence of at least one region, at least two reference white balance algorithms are used to estimate the white point of the image to be processed, and at least two sets of first white point estimation results are obtained.

[0046] At least two white balance algorithms include the Grey World (GW) white balance algorithm, and may also include the Max RGB algorithm, the Grey Index algorithm, and the Deep Learning-Based Auto White Balance algorithm, etc.

[0047] In the application of the Gray World algorithm, it is necessary to determine the corresponding region weight, denoted as GW weight, based on the gray confidence level of at least one region. i Among them, at least one region's regional weight has a positive relationship with the gray confidence level, such as... Figure 3 As shown. For example, for a certain area, if the gray confidence level obtained by the area detection algorithm is 0.8 (indicating that the probability of the area being a gray area is relatively high), then the gray world algorithm has a relatively high weight in that area; if the gray confidence level is 0.3 (indicating that the probability of the area being a gray area is relatively low), then the gray world white balance algorithm has a relatively low weight.

[0048] Then, R / G is calculated separately for different regions. i B / G i The value, and according to the corresponding region weight GWweight i A weighted summation operation is performed. Through this series of calculations, a set of first white point estimation results based on the gray-scale world white balance algorithm can be obtained, denoted as R / G(GW) and B / G(GW), respectively. The calculation formulas for the first white point estimation results are as follows (1) and (2).

[0049]

[0050] Examples of calculations for the first white point estimation results of other reference white balance algorithms are as follows:

[0051] The basic idea of ​​the maximum RGB value algorithm is that the point corresponding to the maximum value of the three RGB channels in an image is close to white. The calculation steps for the first white point estimation result based on the maximum RGB value algorithm are as follows:

[0052] Iterate through all pixels of the image to be processed and find the maximum values ​​in the red (R), green (G), and blue (B) channels, denoted as Rmax, Gmax, and Bmax, respectively.

[0053] Using the green channel as a reference (other channels can also be selected), calculate the gains of the red and blue channels relative to the green channel to obtain the first white point estimation results R / G(Max) and B / G(Max), and the calculation formulas are as follows (3) and (4).

[0054] R / G(Max)=Rmax / Gmax (3)

[0055] B / G(Max)=Bmax / Gmax (4)

[0056] The grayscale index algorithm performs white balance processing based on the grayscale information of an image. Assuming a specific relationship exists between the grayscale index and the RGB channels, the calculation steps for the first white point estimation result based on the grayscale index algorithm are as follows:

[0057] For each pixel in the image, its gray value is calculated according to the common gray value calculation formula as follows (5).

[0058] Gray=0.299R+0.587G+0.114B (5)

[0059] The grayscale values ​​of the entire image are statistically analyzed to obtain a grayscale histogram.

[0060] Based on the grayscale histogram, select a suitable grayscale value as a reference point. Let's assume the selected grayscale reference value is Grayref.

[0061] By establishing a relationship model between grayscale values ​​and RGB channels (for example, assuming a linear relationship between grayscale values ​​and RGB channels), a corresponding set of first white point estimation results R / G(GI) and B / G(GI) are calculated.

[0062] A simple linear model can be represented as:

[0063] Assuming R = a1*Gray + b1, G = a2*Gray + b2, B = a3*Gray + b3, the values ​​of a1, a2, a3, b1, b2 and b3 can be solved by using the known grayscale reference point Grayref and some assumptions (such as the RGB values ​​of white points being approximately equal), and then a set of first white point estimation results based on the grayscale index algorithm can be obtained, as shown in formulas (6) and (7) below.

[0064] R / G(GI)=(a1*Grayref+b1) / (a2*Grayref+b2) (6)

[0065] B / G(GI)=(a3*Grayref+b3) / (a2*Grayref+b2) (7)

[0066] Deep learning-based automatic white balance algorithms typically use neural networks to learn the relationship between image features and white balance parameters. The calculation steps for the first white point estimation result in a deep learning-based automatic white balance algorithm are as follows:

[0067] A large dataset of images with labeled white balance parameters was collected to train the neural network.

[0068] Construct a suitable neural network structure (such as a convolutional neural network CNN), take the image as input, and the white balance parameters (such as the gain values ​​of R / G and B / G) as output. Train the network using a training dataset and adjust the network parameters to minimize the prediction error.

[0069] For the image to be processed, it is input into a trained neural network, which outputs the predicted white balance parameters, namely the white point estimation results R / G(DL) and B / G(DL). The specific network output format may vary depending on the network structure and training method, but it can ultimately be converted into the ratio of R / G and B / G, that is, the first white point estimation result.

[0070] Step S103: Determine the target white point estimation result based on at least two sets of first white point estimation results.

[0071] In this embodiment, the algorithm weights of at least two reference white balance algorithms are first obtained, and based on the algorithm weights of at least two reference white balance algorithms, multiple sets of first white point estimation results are weighted and fused to obtain the target white point estimation results R / G(final) and B / G(final). The calculation formulas for the target white point estimation results R / G(final) and B / G(final) are as follows (8) and (9).

[0072] R / G(final)=R / G(GW)*algoweight(GW)+R / G(algo1)*algoweight(algo1)+

[0073] R / G(algo2)*algoweight(algo2)(8)

[0074] B / G(final)=B / G(GW)*algoweight(GW)+B / G(algo1)*algoweight(algo1)+

[0075] B / G(algo2)*algoweight(algo2)(9)

[0076] Where algoweight(GW) is the algorithm weight of the first reference white balance algorithm, such as the gray world white balance algorithm; algoweight(algo1) is the algorithm weight of the second reference white balance algorithm; and algoweight(algo2) is the algorithm weight of the third reference white balance algorithm. algoweight(GW) + algoweight(algo1) + algoweight(algo2) = 1.

[0077] It should be noted that the above formulas (8) and (9) are only examples of three reference white balance algorithms. The above formulas (8) and (9) may also include two reference white balance algorithms, or more, that is, four or more reference white balance algorithms. The specifics are not shown here.

[0078] The formulas (8) and (9) above are explained below using two reference white balance algorithms as examples.

[0079] Suppose that for a certain region, the gray-world white balance algorithm estimates a first set of white point values ​​as R / G(GW) = 1.2 and B / G(GW) = 1.1, with an algorithm weight (algoweight(GW)) of 0.6; the perfect reflection algorithm estimates a first set of white point values ​​as R / G(algo1) = 1.5 and B / G(algo1) = 1.3, with an algorithm weight (algoweight(algo1)) of 0.4. Then the target white point estimation results for this region are: R / G = 1.2 * 0.6 + 1.5 * 0.4 = 1.32 and B / G = 1.1 * 0.6 + 1.3 * 0.4 = 1.18.

[0080] Step S104: Based on the target white point estimation result, perform white balance correction on the image to be processed to obtain the target image.

[0081] After obtaining the target white point estimation results R / G(final) and B / G(final), the R / G and B / G ratios in the target white point estimation results R / G(final) and B / G(final) are used as adjustment parameters. For each pixel in the image to be processed, the color is adjusted according to its original R, G, and B values ​​according to the following formulas (10), (11), and (12).

[0082] Assuming the original pixel values ​​are (Rorig, Gorig, Borig) and the adjusted pixel values ​​are (Radj, Gadj, Badj), then:

[0083] Radj=Rorig*Gtarget / (R / G(final) (10)

[0084] Gadj = Gorig (11)

[0085] Badj=Borig*Gtarget / (B / G(final)) (12)

[0086] Gtarget can be set to a standard green channel value (such as 255 or dynamically determined based on image characteristics). By traversing all pixels in the image and performing the above calculations, the target image after white balance correction is finally obtained.

[0087] Therefore, this disclosure first acquires the image to be processed, and based on the gray confidence of at least one region in the image to be processed, uses at least two reference white balance algorithms to estimate the white point of the image to be processed, obtaining at least two sets of first white point estimation results. Then, based on the at least two sets of first white point estimation results, the target white point estimation result is determined, and based on the target white point estimation result, white balance correction is performed on the image to be processed to obtain the target image. This disclosure uses multiple white balance algorithms for fusion processing. During the fusion process, even if the weight of one white balance algorithm decreases, the other algorithms can still work together to effectively ensure the accuracy of restoring the true colors of the image.

[0088] To further improve the correction effect and enhance the accuracy of restoring the true colors of the image, multiple rounds of white balance correction can be performed on the image to be processed based on the target white point estimation results to obtain the target image.

[0089] Figure 4 This is a flowchart illustrating multi-round white balance correction of an image to be processed according to an embodiment of the present disclosure.

[0090] like Figure 4 As shown, the embodiment of this disclosure performs multi-round white balance correction on the image to be processed, including:

[0091] Step S401: For the first round of white balance correction, the target white point estimation result is used as the intermediate white point estimation result of the image to be processed in the first round of white balance correction. Based on the intermediate white point estimation result of the image to be processed in the first round of white balance correction, white balance correction is performed on the image to be processed to obtain the intermediate image output by the first round of white balance correction.

[0092] In the first round of white balance correction, the target white point estimation result obtained in step S104 is directly used as the intermediate white point estimation result of the image to be processed in the first round of white balance correction. Then, based on the intermediate white point estimation result of the image to be processed in the first round of white balance correction, a white balance correction operation is performed on the image to be processed, thereby obtaining the intermediate image output by the first round of white balance correction.

[0093] The white balance correction process mainly involves adjusting the gain of each color channel in the image to restore the image's colors to a standard state. For example, based on the midpoint white point estimation result, the gain of the red, green, and blue channels of the image is adjusted separately to eliminate color deviations caused by different lighting conditions.

[0094] Step S402: For the i-th round of white balance correction, re-estimate the white point of the image to be processed to obtain at least two sets of second white point estimation results, and determine the intermediate white point estimation result of the image to be processed in the i-th round of white balance correction based on the at least two sets of second white point estimation results; where i is a positive integer greater than 1.

[0095] In the process of re-estimating the white point of the image to be processed to obtain at least two sets of second white point estimation results, the same method as in step S102 can be used. After obtaining at least two sets of second white point estimation results, the intermediate white point estimation result of the image to be processed in the i-th round of white balance correction is determined based on the at least two sets of second white point estimation results. For example, the white point estimation results of different algorithms can be weighted and fused again to obtain a new intermediate white point estimation result.

[0096] Step S403: Determine whether the difference between the estimated intermediate white point of the image to be processed in the i-th round of white balance correction and the estimated intermediate white point of the (i-1)-th round of white balance correction is less than or equal to a set difference threshold.

[0097] The difference threshold can be flexibly set according to the actual application scenario and needs.

[0098] Step S404: If yes, then end the white balance correction process and use the intermediate image output from the (i-1)th round of white balance correction as the target image.

[0099] If the difference between the estimated intermediate white point of the image under test in the i-th round of white balance correction and the estimated intermediate white point of the (i-1)-th round of white balance correction is less than or equal to a set difference threshold, this indicates that the white balance correction process has converged and achieved a satisfactory correction effect. At this point, the white balance correction process ends, and the intermediate image output from the (i-1)-th round of white balance correction is used as the target image.

[0100] For example, the difference threshold is set to 0.05. If the difference between R / G and B / G in the white point estimation results between the i-th round and the (i-1)-th round is 0.03 and 0.02 respectively, the correction process ends and the target image is output because both the difference between R / G and B / G are less than 0.05.

[0101] Step S405: If not, then perform white balance correction on the image to be processed based on the intermediate white point estimation result of the i-th round of white balance correction to obtain the intermediate image output by the i-th round of white balance correction.

[0102] If the difference between the estimated intermediate white point of the image to be processed in the i-th round of white balance correction and the estimated intermediate white point of the (i-1)-th round of white balance correction is greater than a set difference threshold, it indicates that the white balance correction process has not yet reached a stable state. At this point, based on the estimated intermediate white point of the i-th round of white balance correction, white balance correction is performed on the image to be processed, resulting in the intermediate image output from the i-th round of white balance correction. Then, the process proceeds to the next round of judgment and correction; that is, after step S405, it returns to step S402 until the termination condition is met.

[0103] For example, the difference threshold is set to 0.05. If the difference between R / G and B / G in the white point estimation results between the i-th round and the (i-1)-th round is 0.06 and 0.04 respectively, the termination condition is not met because the difference between R / G is greater than 0.05 and the difference between B / G is less than 0.05. The correction process continues until the termination condition is met and the target image is output.

[0104] Therefore, this disclosure adopts a method of performing multiple rounds of white balance correction on the image to be processed, continuously optimizing and adjusting the final output result to gradually achieve a better state.

[0105] In one embodiment of this disclosure, the image processing method of this application further includes:

[0106] Render the target image for display.

[0107] For example, after acquiring the target image through multiple rounds of white balance correction, it is transmitted to the graphics rendering engine. The engine first decodes the target image, converting the image data from its storage format into an internal data structure that the engine can process. Next, based on preset rendering parameters such as lighting models and material properties, it performs lighting calculations and color mixing for each pixel in the image. For instance, for the surface of an object in the image, it calculates the appropriate lighting intensity and color for each pixel by combining the position and intensity of the virtual light source with the normal direction of the object's surface. Finally, the processed image data is output to the frame buffer of the display device, which then displays it on the screen at a certain refresh rate, allowing the user to intuitively see the high-quality image after white balance correction and rendering.

[0108] For example, in game development, after obtaining the target image through multiple rounds of white balance correction, it is imported into the game engine. The game engine first performs resource management on the target image, storing it in an appropriate resource pool and dynamically loading and unloading it according to the needs of the game scene. During the rendering phase, the engine performs perspective and projection transformations on the target image based on factors such as the camera viewpoint and scene layout in the game, mapping it from three-dimensional space to the two-dimensional screen. Simultaneously, various special effects, such as shadows, reflections, and blurring, are applied to enhance the realism and visual effects of the image. For instance, a real-time shadow algorithm adds realistic shadows to objects in the image, allowing them to better integrate into the game scene. Finally, the rendered target image is sent to the graphics card for final rendering and displayed on the player's screen, providing an immersive gaming experience.

[0109] For example, in the field of medical image processing, after multiple rounds of white balance correction are performed on medical images (such as X-rays, CT scans, and MRIs) to obtain the target image, specialized medical image rendering software is used for display. This software performs special rendering processing on the target image based on the characteristics of the medical image and clinical needs. For instance, for CT images, different color mappings are set according to different tissue densities, converting grayscale images into color images so that doctors can more clearly observe and distinguish different tissue structures. Simultaneously, the software also supports image scaling, rotation, and translation, allowing doctors to view image details from different angles. Furthermore, to meet the needs of telemedicine and consultations, the rendered target image can be transmitted and shared in real time via the network, allowing doctors in different regions to simultaneously view and discuss image data, improving diagnostic efficiency and accuracy.

[0110] For example, in virtual reality (VR) applications, after obtaining a target image through multiple rounds of white balance correction, it is rendered as part of the virtual scene. The VR rendering engine combines data from the head-tracking device to adjust the viewpoint and position of the target image in real time to achieve an immersive visual experience. For instance, when the user turns their head, the engine quickly calculates the new viewpoint and re-renders the target image, making the user feel as if they are in a real virtual scene. Simultaneously, to improve rendering efficiency and reduce latency, the engine employs optimization techniques such as Level of Detail (LOD) and frustum clipping, rendering only the image content within the user's current field of view. Finally, the rendered target image is displayed in front of the user through a VR headset, allowing the user to experience the charm of the virtual world firsthand.

[0111] In summary, the image processing method of this disclosure first acquires an image to be processed, which includes at least one region. Each region has a corresponding gray confidence level, which is used to characterize the probability that the region is a gray region. Based on the gray confidence level of the at least one region, at least two reference white balance algorithms are used to estimate the white point of the image to be processed, resulting in at least two sets of first white point estimation results. Then, based on the at least two sets of first white point estimation results, a target white point estimation result is determined. Based on the target white point estimation result, white balance correction is performed on the image to be processed to obtain the target image, thereby improving the correction effect and enhancing the accuracy of restoring the true colors of the image.

[0112] Figure 5 This is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0113] like Figure 5 As shown, the image processing apparatus 500 of this embodiment includes: an acquisition module 510, a first determination module 520, a second determination module 530, and a processing module 540.

[0114] The acquisition module 510 is used to acquire the image to be processed; wherein the image to be processed includes at least one region, each region has a corresponding gray confidence level, and the gray confidence level is used to characterize the probability that the region is a gray region;

[0115] The first determining module 520 is used to estimate the white point of the image to be processed by using at least two reference white balance algorithms based on the gray confidence of at least one region, and to obtain at least two sets of first white point estimation results.

[0116] The second determining module 530 is used to determine the target white point estimation result based on at least two sets of first white point estimation results;

[0117] The processing module 540 is used to perform white balance correction on the image to be processed based on the target white point estimation result, so as to obtain the target image.

[0118] In one embodiment of this disclosure, at least two reference white balance algorithms include a gray-world white balance algorithm;

[0119] The first determining module 520 includes:

[0120] The first determining unit is used to determine the corresponding region weight for the gray world algorithm based on the gray confidence of at least one region, wherein the region weight corresponding to at least one region has a positive relationship with the gray confidence.

[0121] The second determining unit is used to determine a set of first white point estimation results based on the gray-scale world white balance algorithm according to the region weights corresponding to at least one region.

[0122] In one embodiment of this disclosure, the second determining module 530 includes:

[0123] The acquisition unit is used to acquire the algorithm weights of at least two reference white balance algorithms;

[0124] The third determining unit is used to perform weighted fusion of multiple sets of first white point estimation results based on the fusion weights of at least two reference white balance algorithms to obtain the target white point estimation result.

[0125] In one embodiment of this disclosure, the processing module 540 includes:

[0126] The processing unit is used to perform multiple rounds of white balance correction on the image to be processed based on the target white point estimation results, so as to obtain the target image.

[0127] In one embodiment of this disclosure, the processing unit is used to perform multiple rounds of white balance correction on the image to be processed based on the target white point estimation result to obtain the target image, including:

[0128] For the first round of white balance correction, the target white point estimation result is used as the intermediate white point estimation result of the image to be processed in the first round of white balance correction. Based on the intermediate white point estimation result of the image to be processed in the first round of white balance correction, the image to be processed is white balance corrected to obtain the intermediate image output by the first round of white balance correction.

[0129] For the i-th round of white balance correction, the white point is re-estimated for the image to be processed, resulting in at least two sets of second white point estimation results. Based on the at least two sets of second white point estimation results, the intermediate white point estimation result of the image to be processed in the i-th round of white balance correction is determined; where i is a positive integer greater than 1.

[0130] Determine whether the difference between the estimated midpoint white point of the image to be processed in the i-th round of white balance correction and the estimated midpoint white point of the (i-1)-th round of white balance correction is less than or equal to a set difference threshold.

[0131] If so, the white balance correction process ends, and the intermediate image output from the (i-1)th round of white balance correction is used as the target image;

[0132] If not, then based on the intermediate white point estimation result of the i-th round of white balance correction, perform white balance correction on the image to be processed to obtain the intermediate image output by the i-th round of white balance correction.

[0133] In one embodiment of this disclosure, the acquisition module 510 is further configured to:

[0134] The image to be processed is converted from the first color space to the second color space to obtain the converted image. Region detection is then performed on the converted image to obtain at least one region and the gray confidence level corresponding to the region.

[0135] In one embodiment of this disclosure, the acquisition module 510 acquires the image to be processed in the following ways:

[0136] In response to the shooting operation, the original captured image is acquired;

[0137] Use the original captured image as the image to be processed; or,

[0138] The original captured image is processed to obtain the image to be processed.

[0139] In one embodiment of this disclosure, the apparatus further includes:

[0140] The display module is used to render and display the target image.

[0141] It should be noted that for details not disclosed in the image processing apparatus of this disclosure, please refer to the details described in the image processing method of this disclosure, which will not be repeated here.

[0142] The image processing apparatus of this disclosure acquires an image to be processed through an acquisition module. The image to be processed includes at least one region, and each region has a corresponding gray confidence level. The gray confidence level is used to characterize the probability that a region is a gray region. A first determination module estimates the white point of the image to be processed using at least two reference white balance algorithms based on the gray confidence levels of the at least one region, obtaining at least two sets of first white point estimation results. A second determination module determines a target white point estimation result based on the at least two sets of first white point estimation results. A processing module performs white balance correction on the image to be processed based on the target white point estimation result to obtain a target image. This disclosure can improve the correction effect and enhance the accuracy of restoring the true colors of the image.

[0143] To implement the above embodiments, this disclosure also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the above-described image processing method.

[0144] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0145] Reference Figure 6 The electronic device 600 may include one or more of the following components: processing component 602, memory 604, power component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0146] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0147] Memory 604 is configured to store various types of data to support the operation of electronic device 600. Examples of such data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0148] Power component 606 provides power to various components of electronic device 600. Power component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0149] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0150] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0151] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0152] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0153] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 4G (Fourth Generation), or 5G (Fifth Generation), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0154] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0155] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0156] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described image processing method.

[0157] To implement the above embodiments, this disclosure also provides a chip, which includes a processing circuit for implementing the image processing method described above. Exemplarily, the chip includes an image processing chip (ISP).

[0158] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0159] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0160] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0161] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0162] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0163] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0164] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0165] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that, The method includes: Obtain an image to be processed; wherein the image to be processed includes at least one region, each region has a corresponding gray confidence level, and the gray confidence level is used to characterize the probability that the region is a gray region; Based on the gray confidence of the at least one region, at least two reference white balance algorithms are used to estimate the white point of the image to be processed, and at least two sets of first white point estimation results are obtained. Based on the at least two sets of first white point estimation results, determine the target white point estimation result; Based on the target white point estimation result, white balance correction is performed on the image to be processed to obtain the target image.

2. The method according to claim 1, characterized in that, The at least two reference white balance algorithms include the gray-world white balance algorithm; The process involves estimating the white point of the image to be processed using at least two reference white balance algorithms based on the gray confidence level of the at least one region, resulting in at least two sets of first white point estimation results, including: For the gray world algorithm, the corresponding region weight is determined based on the gray confidence of the at least one region, and the region weight corresponding to the at least one region has a positive relationship with the gray confidence. Based on the region weights corresponding to the at least one region, a set of first white point estimation results based on the gray-scale world white balance algorithm is determined.

3. The method according to claim 1, characterized in that, The step of determining the target white point estimation result based on the at least two sets of first white point estimation results includes: Obtain the algorithm weights of at least two reference white balance algorithms; Based on the algorithm weights of the at least two reference white balance algorithms, the multiple sets of first white point estimation results are weighted and fused to obtain the target white point estimation result.

4. The method according to claim 1, characterized in that, The step of performing white balance correction on the image to be processed based on the target white point estimation result to obtain the target image includes: Based on the target white point estimation results, the image to be processed is subjected to multiple rounds of white balance correction to obtain the target image.

5. The method according to claim 4, characterized in that, The step of performing multiple rounds of white balance correction on the image to be processed based on the target white point estimation result to obtain the target image includes: For the first round of white balance correction, the target white point estimation result is used as the intermediate white point estimation result of the image to be processed in the first round of white balance correction. Based on the intermediate white point estimation result of the image to be processed in the first round of white balance correction, the image to be processed is white balance corrected to obtain the intermediate image output by the first round of white balance correction. For the i-th round of white balance correction, the white point is re-estimated on the image to be processed to obtain at least two sets of second white point estimation results, and based on the at least two sets of second white point estimation results, the intermediate white point estimation result of the image to be processed in the i-th round of white balance correction is determined; where i is a positive integer greater than 1. Determine whether the difference between the estimated midpoint white point of the image to be processed in the i-th round of white balance correction and the estimated midpoint white point of the (i-1)-th round of white balance correction is less than or equal to a set difference threshold. If so, the white balance correction process ends, and the intermediate image output by the (i-1)th round of white balance correction is taken as the target image; If not, then based on the intermediate white point estimation result of the i-th round of white balance correction, the image to be processed is subjected to white balance correction to obtain the intermediate image output by the i-th round of white balance correction.

6. The method according to claim 1, characterized in that, After acquiring the image to be processed, the process further includes: The image to be processed is converted from the first color space to the second color space to obtain the converted image; Region detection is performed on the transformed image to obtain at least one region and the corresponding gray confidence level of the region.

7. The method according to claim 1, characterized in that, The method further includes: The target image is rendered for display.

8. An image processing apparatus, characterized in that, include: An acquisition module is used to acquire an image to be processed; wherein the image to be processed includes at least one region, each region has a corresponding gray confidence level, and the gray confidence level is used to characterize the probability that the region is a gray region; The first determining module is used to perform white point estimation on the image to be processed using at least two reference white balance algorithms based on the gray confidence of the at least one region, and obtain at least two sets of first white point estimation results. The second determining module is used to determine the target white point estimation result based on the at least two sets of first white point estimation results; The processing module is used to perform white balance correction on the image to be processed based on the target white point estimation result, so as to obtain the target image.

9. The apparatus according to claim 8, characterized in that, The at least two reference white balance algorithms include the gray-world white balance algorithm; The first determining module includes: The first determining unit is used to determine the corresponding region weight for the gray world algorithm based on the gray confidence of the at least one region, wherein the region weight corresponding to the at least one region has a positive relationship with the gray confidence. The second determining unit is used to determine a set of first white point estimation results based on the gray-scale world white balance algorithm according to the region weights corresponding to the at least one region.

10. The apparatus according to claim 8, characterized in that, The second determining module includes: The acquisition unit is used to acquire the algorithm weights of at least two reference white balance algorithms; The third determining unit is used to perform weighted fusion of the multiple sets of first white point estimation results based on the fusion weights of the at least two reference white balance algorithms to obtain the target white point estimation result.

11. The apparatus according to claim 8, characterized in that, The processing module includes: The processing unit is used to perform multiple rounds of white balance correction on the image to be processed based on the target white point estimation result to obtain the target image.

12. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the image processing method as described in any one of claims 1-7.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the image processing method according to any one of claims 1-7.

14. A chip comprising a processing circuit for implementing the image processing method according to any one of claims 1-7.

Images