A color adaptation-based image processing method, electronic device and related medium

Abstract

This application provides an image processing method, electronic device, and related medium based on color adaptation. The method includes: acquiring a RAW image in response to a user's shooting command; performing local white balance processing on the RAW image to generate a first processed image corresponding to the RAW image; dividing the first processed image into i image blocks; and performing color adaptation processing on each of the i image blocks in parallel to generate a second processed image corresponding to each image block, and then stitching them together to generate a target image. By implementing the method of this application, white balance processing can be performed on the RAW image to generate a first processed image, and then the first processed image can be divided into multiple image blocks and color adaptation processing can be performed on each block separately. This helps to achieve full-pixel-level color adaptation processing on the first processed image without affecting the image color adaptation processing rate, which helps to make the image color adaptation effect more significant, thereby improving the user experience.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to an image processing method, electronic device and related medium based on color adaptation. Background Technology

[0002] With the increasing sophistication of camera functions on electronic devices, the scenarios in which users take photos have become more diverse, leading to a growing emphasis on the image quality of these devices. The image quality of an electronic device depends partly on the camera model and partly on its image processing capabilities. To provide users with images that more closely resemble human vision, existing technologies can perform color adaptation processing based on the difference between the ambient and screen color temperatures. However, this method is unsuitable for images captured in mixed lighting scenarios. Because the lighting conditions in mixed lighting scenarios are complex, relying solely on the difference between the ambient and screen color temperatures for color adaptation processing can easily result in inaccurate colors for some objects in the image (e.g., some objects appear bluish in mixed color temperature lighting scenarios), thus degrading the user's shooting experience. Therefore, improving image color adaptation is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] In a first aspect, this application provides a color-adaptive-based image processing method, an electronic device, and a related medium, the method including:

[0004] In response to the user's shooting command, acquire RAW images;

[0005] Perform white balance processing on the RAW image to generate the first processed image corresponding to the RAW image;

[0006] The first processed image is cut into i image blocks, where i ≥ 2 and i is a positive integer;

[0007] In a multi-process manner, color adaptation processing is performed on each image block in i image blocks simultaneously to generate a second processed image corresponding to each image block;

[0008] The target image is generated by stitching together the second-processed images corresponding to each image block.

[0009] Implementing the method provided in the first aspect, the electronic device can acquire a RAW image in response to the user's shooting command and perform white balance processing on the RAW image to generate a first processed image. This helps to balance the color temperature and brightness of the image, preventing severe local overexposure or color cast (such as a bluish or yellowish tint) in the image presented to the user, thus ensuring a better shooting experience. Furthermore, the electronic device can also divide the first processed image into multiple image blocks and perform color adaptation processing on each image block in parallel in a multi-process manner to obtain multiple second processed images. It can be seen that the electronic device can perform corresponding color adaptation processing based on the lighting conditions of each part (or pixel) of the first processed image, which helps to improve the effect of image color adaptation processing, thereby providing the user with images with better colors and a better shooting experience. However, existing technologies do not consider the light source distribution of the shooting scene or the complexity of the lighting on objects in the image. They only perform color adaptation processing on the image based on the difference between the ambient color temperature and the screen color temperature. This easily leads to inaccurate object colors in the color-adapted image, failing to provide a good shooting experience for the user. Unlike existing technologies, the method of this application fully considers the lighting conditions of each object in the image, and can perform color adaptation processing on each image block of the first processed image separately, which helps to ensure the color adaptation effect of each image block, thereby ensuring the imaging effect of each object in the image. Moreover, the method of this application embodiment can adopt a multi-process approach, performing color adaptation processing on each image block simultaneously, which helps to improve the color adaptation processing effect while ensuring the processing speed, thereby further improving the user's shooting experience.

[0010] Implementing the method provided in the first aspect, in some embodiments, performing color adaptation processing on each image block among i image blocks to generate a second processed image corresponding to each image block may include:

[0011] Based on the color information of the pixels contained in each image block, the color adaptation gain matrix corresponding to each image block is determined. The color information may include color temperature value, luminance coefficient and chromaticity value.

[0012] Based on the color adaptation gain matrix corresponding to each pixel block, a second processed image corresponding to each image block is generated.

[0013] By implementing the method provided in the above embodiments, an electronic device can determine a color adaptation gain matrix based on the color information of the pixels contained in each image block, thereby generating a second processed image corresponding to each image block based on each image block and its corresponding color adaptation matrix, achieving the purpose of color adaptation processing for each image block. The color information of a pixel may include the color temperature value, the luminance coefficient, and the chromaticity value of that pixel.

[0014] Implementing the method provided in the first aspect, in some embodiments, performing white balance processing on a RAW image to generate a first processed image corresponding to the RAW image may include:

[0015] The RAW image is input into the local white balance model, and the first processed image is output.

[0016] By implementing the method provided in the above embodiments, this application can use a local white balance model to perform white balance processing on RAW images, thereby achieving the purpose of balancing the color temperature and brightness of the image. This helps to prevent the images presented to users from having severe local overexposure or local color casts (such as a bluish or yellowish tint), ensuring the user's shooting experience. The local white balance model is a neural network model.

[0017] In implementing the method provided in the first aspect, in some embodiments, the local white balance model is a model trained using mixed light source images as training samples and the standard output image corresponding to the mixed light source images as labels.

[0018] The scene in which the image is captured contains n light sources, where n ≥ 2 and n is a positive integer.

[0019] The standard output image corresponding to the mixed light source image is generated based on the mixed light source image and the target chromaticity value.

[0020] By implementing the methods provided in the above embodiments, the local white balance model in this application uses mixed light source images as training samples, which helps the local white balance model cover more image shooting scenarios. This ensures good color balance capabilities for images collected or captured in various complex light source scenarios, laying the foundation for subsequent color adaptation processing and thus guaranteeing the user's shooting experience. Furthermore, the method in this application embodiment can generate training labels for the local white balance model based on mixed light sources and target chromaticity values. By using self-made training data pairs, it helps ensure the iterative effect of the local white balance model and improve its color balance capability.

[0021] In implementing the method provided in the first aspect, in some embodiments, the target chromaticity value is generated by fusing the chromaticity value corresponding to each of the n calibration images and the fusion ratio corresponding to each of the n calibration images, and there is a one-to-one correspondence between the n calibration images and the n light sources.

[0022] Implementing the method provided in the above embodiments, the mixed light source image in this application corresponds to n light sources and also to n calibration images. Furthermore, it can be considered that there is a one-to-one correspondence between the n calibration images and the n light sources. The method in this application embodiment can determine the target chromaticity value based on the chromaticity value corresponding to each of the n calibration images and the fusion ratio corresponding to each calibration image. Determining the target chromaticity value helps generate a suitable standard output image for the mixed light source image, thereby ensuring the training effect of the local white balance model.

[0023] In implementing the method provided in the first aspect, in some embodiments, any one of the n calibration images is generated by preprocessing the single-light source image corresponding to any one calibration image. The preprocessing may include lens shading correction processing, image format conversion, and white balance calibration processing.

[0024] The shooting scene corresponding to any of the above calibration images contains a light source that corresponds to any of the above calibration images.

[0025] Implementing the method provided in the above embodiments, the mixed light source image in this application corresponds to n single light source images, and there is a one-to-one correspondence between the n single light source images and the aforementioned n light sources. After lens shading correction processing, image format conversion processing, and white balance calibration processing, the single light source images can generate corresponding calibration images. Among them, lens shading correction processing can correct the uneven brightness distribution of the image caused by the optical characteristics of the lens and image sensor, so that the corrected image has the same brightness level throughout the entire field of view; image format conversion mainly refers to converting the RAW image into a three-channel PNG image, so that the two G channel values ​​in the RAW image are merged into one G channel value, which helps in the subsequent calculation of the target chromaticity value; white balance calibration processing is to further equalize the color of the image, reduce the color cast of the image, and make the colors of each object in the image more consistent with the actual situation. Through the above preprocessing, it helps to generate a suitable standard output image, thereby ensuring the training effect of the local white balance model.

[0026] In implementing the method provided in the first aspect, in some embodiments, the image is calibrated as a PNG image;

[0027] The fusion ratio corresponding to each calibration image is generated based on the G channel value of each calibration image.

[0028] Implementing the method provided in the above embodiments, the calibration image in this application is a three-channel PNG image. The PNG image can merge the two G channel values ​​in the RAW image into one G channel value, which helps to calculate the target chromaticity value in the subsequent process, thereby achieving the purpose of creating a standard output image corresponding to the mixed light source image, thus ensuring the training effect of the local white balance model.

[0029] Secondly, embodiments of this application provide an electronic device, which may include: an interaction module, an image acquisition module, and an image processing module;

[0030] The interaction module can be used to receive shooting instructions from the user;

[0031] The image acquisition module can be used to acquire RAW images in response to the user's shooting command;

[0032] The image processing module can be used to perform white balance processing on RAW images and generate a first processed image corresponding to the RAW image;

[0033] The image processing module can also be used to cut the first processed image into i image blocks, where i ≥ 2 and i is a positive integer;

[0034] The image processing module can also be used to perform color adaptation processing on each image block in i image blocks simultaneously in a multi-process manner, generating a second processed image corresponding to each image block;

[0035] The image processing module can also be used to stitch together the second processed images corresponding to each image block to generate the target image.

[0036] In implementing the method provided in the second aspect, in some embodiments, the electronic device may further include:

[0037] The image processing module can also be used to determine the color adaptation gain matrix corresponding to each image block based on the color information of the pixels contained in each image block. The color information may include color temperature value, brightness coefficient and chromaticity value.

[0038] The image processing module can also be used to generate a second processed image corresponding to each image block based on the color adaptation gain matrix corresponding to each pixel block.

[0039] In implementing the method provided in the second aspect, in some embodiments, the electronic device may further include:

[0040] The image processing module can also be used to input RAW images into a local white balance model and output a first processed image.

[0041] In implementing the method provided in the second aspect, in some embodiments, the local white balance model is a model trained using mixed light source images as training samples and the standard output image corresponding to the mixed light source images as labels.

[0042] The scene corresponding to the mixed light source image contains n light sources, where n ≥ 2 and n is a positive integer.

[0043] The standard output image corresponding to the mixed light source image is generated based on the mixed light source image and the target chromaticity value.

[0044] In implementing the method provided in the second aspect, in some embodiments, the target chromaticity value is generated by fusing the chromaticity value corresponding to each of the n calibration images and the fusion ratio corresponding to each of the n calibration images, and there is a one-to-one correspondence between the n calibration images and the n light sources.

[0045] In implementing the method provided in the second aspect, in some embodiments, any one of the n calibration images is generated by preprocessing the single-light source image corresponding to any one calibration image. The preprocessing may include lens shading correction processing, image format conversion, and white balance calibration processing.

[0046] The shooting scene corresponding to any of the above calibration images contains a light source that corresponds to any of the above calibration images.

[0047] In implementing the method provided in the second aspect, in some embodiments, the image is calibrated as a PNG image;

[0048] The fusion ratio corresponding to each calibration image is generated based on the G channel value of each calibration image.

[0049] Thirdly, embodiments of this application provide an electronic device including one or more processors and one or more memories; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by one or more processors, cause the execution of the method described in the first aspect and any possible implementation thereof.

[0050] Fourthly, embodiments of this application provide a chip including logic circuitry and an interface, wherein the logic circuitry and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuitry is used to execute the code instructions to cause the method in the first aspect or any possible implementation thereof to be executed.

[0051] Fifthly, this application provides a computer-readable storage medium including instructions that, when executed on a target terminal, cause the execution of the method described in the first aspect and any possible implementation thereof.

[0052] In a sixth aspect, this application provides a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0053] Understandably, the electronic devices provided in the second and third aspects, the chip provided in the fourth aspect, the computer-readable storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect are all related to the color adaptation-based image processing method provided in the first aspect, and can be used to execute the method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of the color adaptation-based image processing method in the corresponding first aspect, and will not be repeated here. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0055] Figure 1 This is a flowchart illustrating an image color restoration method provided by existing technology;

[0056] Figure 2 This is a schematic flowchart of an image processing method based on color adaptation provided in an embodiment of this application;

[0057] Figure 3 This is a schematic diagram of a scene for cutting a first processed image according to an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of a scene for generating a target image provided in an embodiment of this application;

[0059] Figures 5a-5c This is a schematic diagram of an image processing scene based on color adaptation provided in an embodiment of this application;

[0060] Figure 6 This is a schematic flowchart of another color adaptation-based image processing method provided in an embodiment of this application;

[0061] Figure 7 This is a schematic diagram of a process for training a local white balance model according to an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of a process for creating training data for training a local white balance model, provided in an embodiment of this application.

[0063] Figure 9 This is a schematic diagram of a scene for acquiring a single-light source image provided in an embodiment of this application;

[0064] Figure 10This is a schematic diagram of another process for creating training data for training a local white balance model, provided in an embodiment of this application.

[0065] Figure 11 This is a schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application;

[0066] Figure 12 This is a schematic diagram of the hardware structure of an electronic device 100 according to an embodiment of this application;

[0067] Figure 13 This is a software structure block diagram provided for an embodiment of this application. Detailed Implementation

[0068] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. The terminology used in the following embodiments of this application is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that in the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0069] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0070] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0071] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first:

[0072] (1) RAW image

[0073] RAW images are the raw data captured by an image sensor and converted into digital signals; they are the data format of the image output by the image sensor. A sensor, also known as a photosensitive element, is a device that converts optical images into electronic signals, such as a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In this application, the image sensor can be located within the image acquisition module. In the field of image technology, RAW format is an unprocessed, uncompressed format, i.e., raw image encoded data (digital negative). Common RAW format suffixes include .ARW, .SRF, .SR2, .crw, .cr2, and .cr3.

[0074] (2) PNG image

[0075] PNG is a lossless image compression format, short for "Portable Network Graphics." As a standard format for transmitting images over the internet, PNG is recommended by the World Wide Web Consortium (W3C) and is widely used on the web. PNG supports indexed, grayscale, and RGB color schemes, as well as features such as an alpha channel, enabling transparent image effects.

[0076] (3) RGB color mode

[0077] The RGB color model is an industry-standard color system that uses variations in the three color channels—red (R), green (G), and blue (B)—and their superposition to create a wide variety of colors. RGB represents the colors of the red, green, and blue channels. This standard covers almost all colors that can be perceived by human vision and is one of the most widely used color systems.

[0078] (4) Automatic white balance

[0079] Automatic white balance (AWB) refers to the algorithm used to restore white to true white (typically the white seen by humans under natural sunlight) in images taken under different color temperatures. Because sensors cannot change their photosensitivity according to changes in ambient light like the human eye, an additional module is needed to simulate human perception and restore white to its true white under different ambient light conditions. AWB algorithms are mainly divided into two types: local white balance and global white balance.

[0080] (5) GT image

[0081] In machine learning, Ground Truth is generally used to represent the classification accuracy of the training set in supervised learning, and is used to prove or disprove a hypothesis. Supervised machine learning labels the training data, and the correctly labeled data constitutes the Ground Truth. In the embodiments of this application, the GT image can be understood as the standard output image.

[0082] Please see Figure 1 , Figure 1 A flowchart illustrating an image color restoration method provided by existing technology.

[0083] like Figure 1 As shown, the method includes:

[0084] S101: Receives RGB color gamut data generated by the shooting device.

[0085] S102: Obtain the ambient color temperature of the shooting environment when the shooting device captures the image, the ambient color temperature of the display device, and the color coordinates of white in the color gamut information of the display device's screen.

[0086] S103: Based on the color temperature of the shooting environment, the color temperature of the display environment, and the color coordinates of white in the color gamut information of the display device's screen, perform color adaptation transformation on the RGB data of the screen color gamut to obtain the RGB data of the display environment.

[0087] Furthermore, the imaging device converts the sensor RGB data obtained when capturing the image to the color gamut of the display device based on the screen color gamut information, thus obtaining the screen color gamut RGB data.

[0088] Among them, the color adaptation transformation of the screen color gamut RGB data based on the shooting environment color temperature, the display environment color temperature, the color coordinates of white in the display device's screen color gamut information includes:

[0089] The color temperature of the shooting environment and the color temperature of the display environment are blended to obtain the blended color temperature;

[0090] Map the blended color temperature to the target color temperature;

[0091] Calculate the color coordinates corresponding to the target color temperature;

[0092] Calculate the ratio matrix of the color coordinates corresponding to the target color temperature and the color coordinates of white in the screen color gamut information;

[0093] Convert the screen's RGB color gamut data to the XYZ color space to obtain the first XYZ color data;

[0094] The first XYZ color data is converted to the LMS color space to obtain the first LMS color data;

[0095] Multiply the first LMS color data by the color coordinate ratio matrix to obtain the second LMS color data;

[0096] The second LMS color data is converted to the XYZ color space to obtain the second XYZ color data;

[0097] The second XYZ data is converted to the RGB color space to obtain the RGB data of the display environment.

[0098] More specifically, converting the sensor RGB data obtained when the imaging device captures an image to the color gamut of the display device, based on the screen's color gamut information, includes:

[0099] Calculate the color gamut mapping matrix based on the screen color gamut information;

[0100] The sensor's RGB data is converted to the display device's color gamut based on the color gamut mapping matrix.

[0101] As can be seen, existing technologies, in order to reproduce the human eye's viewing effect when capturing images, refer to the ambient color temperature of the shooting environment, the display environment color temperature, and the screen color temperature (the white coordinates in the screen's color gamut information represent the screen color temperature) to achieve accurate color reproduction. Existing technologies perform color adaptation transformation based on the ambient color temperature, display environment color temperature, and the white coordinates in the screen's color gamut information, changing the image's color under different color temperature conditions and improving the accuracy of image color reproduction to achieve the visual effect captured in different viewing environments. However, existing technologies do not consider the light source distribution of the shooting scene, nor the complexity of the lighting on objects in the image. They only perform color adaptation processing based on the difference between the ambient color temperature and the screen color temperature. This easily leads to problems such as blurry object edges in the color-adapted image, reducing image quality and failing to provide a good shooting experience for users.

[0102] Unlike existing technologies, the method in this application fully considers the lighting conditions of each object in the image, and can perform color adaptation processing on each image block of the first processed image separately, which helps to ensure the color adaptation effect of each image block, thereby ensuring the imaging effect of each object in the image. Moreover, the method in this application can adopt a multi-process approach, performing color adaptation processing on each image block simultaneously, which helps to improve the color adaptation processing effect while ensuring the processing speed, thereby further improving the user's shooting experience.

[0103] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image processing method based on color adaptation, provided in an embodiment of this application.

[0104] like Figure 2 As shown, the method may include:

[0105] S201: Electronic device 100 acquires a RAW image in response to the user's shooting command.

[0106] It should be noted that electronic device 100 can also be called user equipment (UE). It can be handheld or wearable. It can also be called user terminal, terminal equipment, access terminal equipment, UE unit, UE station, mobile device, UE terminal equipment, mobile terminal, wireless communication equipment, UE agent, or UE device, etc. Its specific form can be a mobile phone, tablet computer, computer with wireless transceiver capabilities, wearable terminal device, etc., and it is an entity on the user side used to receive or transmit signals. The specific structure of electronic device 100 can be found in [reference needed]. Figure 11 and Figure 12The relevant details will not be elaborated upon here.

[0107] S202: Electronic device 100 performs white balance processing on the RAW image to generate a first processed image corresponding to the RAW image.

[0108] In some possible implementations, the electronic device 100 can input a RAW image into a local white balance model to output a first processed image.

[0109] In this application, the local white balance model in the embodiment method is a model trained using mixed light source images as training samples and the standard output image corresponding to the mixed light source images as labels. This helps to solve the problem of color imbalance in images captured in mixed light source scenes, and avoids the final image presented to the user from having a serious color cast (such as some objects in the image appearing bluish or yellowish), thereby improving the user's shooting experience.

[0110] It should be noted that the format of the first processed image can be determined based on the image presentation requirements of the electronic device 100 (or the user's requirements). The first processed image can be any one of JPEG, BMP, PNG or TITF, and this application does not limit it.

[0111] Furthermore, the image after white balance processing (such as the first processed image mentioned above) will carry gain values ​​corresponding to the R, G, and B channels. For example, the RGB data arrangement of the RAW image acquired by the electronic device 100 mainly includes four types: GRBG, RGGB, BGGR, and GBRG. It can be seen that all four arrangements result in one pixel corresponding to two G channel values. To facilitate subsequent image optimization, the electronic device 100 can merge the two G channel values ​​corresponding to each pixel in the RAW image into one G value. Specifically, the merging method can be to calculate the average of the two G channel values. For example, if pixel A corresponds to G1 and G2 in the RAW image, after the electronic device 100 converts the RAW image to a PNG image, the G channel corresponding to pixel A in the PNG image can be (G1+G2) / 2.

[0112] S203: The electronic device 100 cuts the first processed image into i image blocks.

[0113] Where i ≥ 2, and i is a positive integer. To preserve the sharpness of the edges of objects in the first processed image and achieve better color adaptation processing, the method in this embodiment selects to perform full-pixel-level color adaptation processing on the first processed image. Since performing full-pixel-level color adaptation processing on a single image increases the computational load of the electronic device 100, resulting in excessively long image processing time, to solve this problem, the method in this embodiment selects to cut the first processed image into multiple image blocks and simultaneously perform color adaptation processing on each image block, which helps to improve the efficiency of color adaptation processing.

[0114] Specifically, the electronic device 100 can segment the first processed image based on a pre-set value i by a technician. The segmentation can be horizontal or vertical. Further, the electronic device 100 can determine the number of pixels in the width of the first processed image as the value i, and also determine the number of pixels in the height of the first processed image as the value i, and then segment the first processed image based on these values ​​i. Even further, the electronic device 100 can segment the first processed image into i image blocks of size a×b, where the values ​​a and b can be determined based on the size of the first processed image and the value i. It should be noted that the units for a and b can be image size units such as inches, pixels, and centimeters.

[0115] For example, please see Figure 3 , Figure 3 This is a schematic diagram illustrating a scenario of cutting a first processed image, as provided in an embodiment of this application. Figure 3 As shown, the electronic device 100 in the method of this application embodiment can segment the first processed image based on the size of the first processed image (the number of pixels corresponding to the image width and the number of pixels corresponding to the image height). Figure 3 If the first processed image has a size of 8 pixels × 6 pixels, it can be horizontally cut into 6 image blocks of size 8 pixels × 1 pixel according to the number of pixels corresponding to the image height, or vertically cut into 8 image blocks of size 1 pixel × 6 pixels according to the number of pixels corresponding to the image width, thereby achieving the purpose of improving the efficiency of color adaptation processing.

[0116] In some possible implementations, the technician can pre-set the value of i, that is, it can be assumed that the electronic device 10 selects to cut the first processed image of various sizes into i image blocks. For example, let i be 24. If the electronic device 100 can capture images of various sizes (e.g., it can capture images of 1080 pixels × 1080 pixels and 3024 pixels × 4032 pixels), when the image size corresponding to the user's shooting instruction is 1080 × 1080 pixels, the electronic device 100 can horizontally cut the first processed image (corresponding to a size of 1080 pixels × 1080 pixels) into 24 image blocks of 1080 pixels × 45 pixels, or vertically cut it into 24 image blocks of 45 pixels × 1080 pixels. When the image size corresponding to the user's shooting instruction is 3024 × 4032 pixels, the electronic device 100 can horizontally cut the first processed image (corresponding to a size of 3024 × 4032 pixels) into 24 image blocks of 4032 × 126 pixels, or vertically cut it into 24 image blocks of 168 pixels × 3024 pixels.

[0117] Optionally, during the process of cutting the first processed image into multiple image blocks, the electronic device 100 can set a serial number label for each image block to indicate the position of each image block in the first processed image, so as to ensure that the second image block can be correctly spliced ​​in the subsequent process.

[0118] It should be noted that the above examples of the method for cutting the first processed image and the specifications of the corresponding pixel matrix of the first processed image are only for illustrating the method of the embodiments of this application in more detail, and should not be construed as limiting this application. Those skilled in the art can choose other cutting schemes based on actual conditions.

[0119] S204: The electronic device 100 performs color adaptation processing on each image block in parallel based on the color information of each image block in the i image blocks, and generates a second processed image corresponding to each image block.

[0120] The electronic device 100 performs color adaptation processing on each of the i image blocks in parallel in a multi-process manner to generate a second processed image corresponding to each image block.

[0121] Specifically, based on the color information of each image block, a color adaptation scheme corresponding to each image block is determined, and color adaptation processing is performed on each image block in parallel. For example, the electronic device 100 can look up the corresponding target color value in the color adaptation table based on the color information corresponding to the image block, and determine the color gain matrix based on the image color value corresponding to each pixel in the image block, thereby achieving the purpose of color adaptation processing on the image block.

[0122] In some possible implementations, color adaptation processing is performed on each image block in the i image blocks to generate a second processed image corresponding to each image block, which may include:

[0123] Based on the color information of the pixels contained in each image block, determine the color adaptation gain matrix corresponding to each image block;

[0124] Based on the color adaptation gain matrix corresponding to each pixel block, a second processed image corresponding to each image block is generated.

[0125] Color information may include color temperature, luminance coefficient, and chromaticity value.

[0126] Specifically, the electronic device 100 can directly query the chromaticity value of the color block corresponding to each image block, thereby determining at least one of the target color temperature value, target luminance coefficient, or target chromaticity value corresponding to the image block. Further, based on at least one of the target color temperature value, target luminance coefficient, or target chromaticity value, the color adaptation gain value corresponding to each pixel in the image block is determined, thereby determining the color adaptation gain matrix corresponding to the image block. Even further, by multiplying the pixel matrix corresponding to each image block with the color adaptation gain matrix, a second processed image corresponding to each image block can be generated.

[0127] S205: The electronic device 100 stitches together the second processed images corresponding to each image block to generate a target image.

[0128] For example, please see Figure 4 , Figure 4 This is a schematic diagram of a scene for generating a target image, provided in an embodiment of this application.

[0129] like Figure 4 As shown, each image block, after color adaptation processing, can generate a corresponding second processed image. The electronic device 100 can stitch the second processed images based on the sequence labels corresponding to each second image block (which can also be understood as the sequence labels corresponding to each image block). Optionally, the sequence labels can be in the form of numbers, and the target image can be generated by correctly stitching the second processed images according to an ascending order. The target image can be understood as the image finally presented to the user.

[0130] Please see Figures 5a-5c , Figures 5a-5c This is a schematic diagram of an image processing scene based on color adaptation, provided as an embodiment of this application.

[0131] like Figure 5a As shown, after receiving the user's shooting instruction, the electronic device 100 acquires the corresponding RAW image, performs white balance and color adaptation processing on the RAW image, and finally generates the target image.

[0132] like Figure 5b As shown, the electronic device 100 can display the target image in the preview frame of the shooting interface. Furthermore, the electronic device 100 can respond to the user's viewing command (such as clicking the preview frame) and display the target image in the relevant interface of the photo album application, which helps the user to view the target image more clearly and in detail.

[0133] Specifically, the processing procedure of the electronic device 100 for RAW images can be referred to... Figure 5c .like Figure 5c As shown, a local white balance model is used to perform white balance operation on the RAW image, and a first processed image is output. Further, based on the size of the first processed image, it is horizontally divided into i image blocks, and color adaptation processing is performed on each image block using a multi-process synchronous execution method, thereby obtaining a second processed image corresponding to each image block. Furthermore, the electronic device 100 can correctly stitch the various second processed images into a single target image based on the location tags corresponding to each second processed image.

[0134] As can be seen, the electronic device 100 in this embodiment can acquire a RAW image in response to a user's shooting command and perform white balance processing on the RAW image to generate a first processed image. This helps to balance the color temperature and brightness of the image, preventing the image presented to the user from having severe local overexposure or local color cast (such as a bluish or yellowish tint), thus helping to ensure the user's shooting experience. Furthermore, the electronic device 100 can perform corresponding color adaptation processing based on the lighting conditions of each part (or pixel) of the first processed image, which helps to improve the effect of image color adaptation processing, thereby providing the user with an image of better quality and giving the user a better shooting experience.

[0135] For example, please see Figure 6 , Figure 6 This is a schematic flowchart illustrating another color-adaptive-based image processing method provided in an embodiment of this application. Figure 6 As shown, the method in this application embodiment first utilizes a mixed light source image (a RAW format image, and the shooting scene corresponding to the mixed light source image contains n light sources) and the standard output image corresponding to the mixed light source image (i.e., Figure 6 Training a local white balance model using a hybrid light source (GT) (i.e., ...) Figure 6 The LocalAWB model (as described in the text) can be referenced for its specific training process. Figures 7-9 The relevant content will not be elaborated here.

[0136] Furthermore, after the local white balance model is trained, it can be loaded into the electronic device 100. After the electronic device 100 acquires a RAW image in response to the user's shooting command, it can input the RAW image into the local white balance model. Correspondingly, the local white balance model will output a LocalAWB inference result image (i.e., the first processed image mentioned above).

[0137] Furthermore, the electronic device 100 performs Color Adaptation (CA) processing on the LocalAWB inference result image. Specifically, the electronic device 100 performs CA processing synchronously on each image patch corresponding to the LocalAWB inference result image in a multi-process manner. Optionally, the electronic device 100 can refer to the CA processing method corresponding to Global White Balance (Global AWB) to perform CA processing on each image patch. Specifically, the CA processing method corresponding to Global White Balance (Global AWB) can be understood as the electronic device 100 determining a color adaptation gain matrix (CA Gain Map) based on an image and performing CA processing on the image based on this color adaptation gain matrix. In this embodiment, the method, by determining the color adaptation gain matrix corresponding to each image patch and performing CA processing on each image patch based on the color adaptation matrix corresponding to each image patch, helps maintain the edge sharpness of each object contained in the RAW image, ensuring the imaging effect of each object in the image, thereby improving the color adaptation effect.

[0138] Furthermore, the electronic device 100 processes the second processed image (i.e., the image obtained by color adaptation of each image patch) through color adaptation. Figure 6 The target image is generated by stitching together the results after color adaptation, and finally, the target image can be presented to the user in response to the user's viewing command.

[0139] Please see Figure 7 , Figure 7 This is a flowchart illustrating a method for training a local white balance model, as provided in an embodiment of this application.

[0140] like Figure 7 The method includes:

[0141] S701: Obtain the training dataset, which includes m mixed light source images and n single light source images corresponding to each of the m mixed light source images.

[0142] In the mixed light source image, there are n light sources in the shooting scene. There is a one-to-one correspondence between the single light source image and the light source. n≥2, where n is a positive integer, and m≥2, where m is a positive integer.

[0143] S702: Determine the target chromaticity value corresponding to each mixed light source image based on the chromaticity values ​​of the n single light source images corresponding to each mixed light source image.

[0144] In some possible implementations, determining the target chromaticity value corresponding to each mixed light source image based on the chromaticity values ​​of n single light source images corresponding to each mixed light source image may include:

[0145] Preprocess the n single-light source images corresponding to each mixed light source image to generate n calibration images corresponding to each mixed light source image. There is a one-to-one correspondence between the calibration images and the single-light source images. The preprocessing includes lens shadow correction, image format conversion and white balance calibration.

[0146] Based on the chromaticity values ​​of each of the n calibration images corresponding to each mixed light source image, and the fusion ratio of each calibration image, the target chromaticity value corresponding to each mixed light source image is determined. Optionally, the calibration image can be a PNG image.

[0147] In this application, the method can determine the fusion ratio of each calibration image corresponding to each mixed light source image based on the G channel value of each calibration image corresponding to each mixed light source image.

[0148] S703: Generate a standard output image corresponding to each mixed light source image based on each mixed light source image and the target chromaticity value corresponding to each mixed light source image.

[0149] Please see below. Figure 8 , Figure 8 This is a schematic diagram illustrating a process for creating training data for training a local white balance model, as provided in an embodiment of this application.

[0150] The method in this application embodiment can use a mixed light source image and the standard output image corresponding to the mixed light source image as training data pairs (which can be understood as a local white balance model being a model trained with the mixed light source image as the training sample and the standard output image corresponding to the mixed light source image as the label). The method in this application embodiment can create the standard output image corresponding to the mixed light source image based on each single light source image corresponding to any mixed light source image.

[0151] like Figure 8 As shown, the process of training a local white balance model may include:

[0152] First, for any mixed-light source image, obtain the individual light source images corresponding to that mixed-light source image. The scene corresponding to the mixed-light source image contains n light sources, where n ≥ 2, and n is a positive integer. It should be noted that both the mixed-light source image and its corresponding individual light source images are RAW images.

[0153] For example, please see Figure 9 , Figure 9 This is a schematic diagram illustrating a scene for acquiring a single-light source image, as provided in an embodiment of this application. Figure 9 As shown, suppose there exists a shooting scene corresponding to the first mixed light source image (e.g., Figure 9 In the first mixed light source scene 91, there are 3 light sources (such as...) Figure 9 If the first light source 92, the second light source 93, and the third light source 94 are used, then the method of this application embodiment can acquire corresponding single-light source images for each of these three light sources (e.g., ...). Figure 9 As can be seen from the first single-light source image 95, the second single-light source image 96, and the third single-light source image 97, there is a one-to-one correspondence between the above single-light source images and the multiple shooting scenes. The method of this application, by obtaining the individual single-light source images corresponding to the mixed light source image, helps to generate a standard output image that is more in line with the color adjustment scheme of the mixed light source image, thereby helping to improve the training effect of the local white balance model.

[0154] Furthermore, the method in this application embodiment can preprocess each single-light source image corresponding to the mixed-light source image to achieve the purpose of correcting the color imbalance problem existing in the single-light source image. Optionally, the preprocessing may include lens shading correction processing, image format conversion processing, and white balance calibration processing.

[0155] Specifically, in order to solve the problem of uneven image brightness distribution caused by the optical characteristics of the correction lens and image sensor, the method of this application embodiment can perform lens shading correction (LSC) processing on each single light source image and generate a corrected image corresponding to each single light source image, which helps to make the corrected image (which can be understood as the second processed image mentioned above) have the same brightness level in the entire field of view.

[0156] Furthermore, the method in this embodiment can perform format conversion on the corrected image, which helps determine the RGB channel values ​​corresponding to each pixel, facilitating subsequent calculation of the target luminance value. Optionally, the method in this embodiment can convert the RAW format corrected image into a corresponding PNG image. By converting the RAW image into a three-channel PNG image, the two G channel values ​​in the RAW image are merged into one G channel value, which helps in subsequent calculation of the target chromaticity value. The specific image format conversion process can be referred to the above. Figure 2 The details of step S202 and its related embodiments are not elaborated here.

[0157] Furthermore, the method in this application embodiment can perform white balance calibration processing on the PNG images corresponding to each calibrated image, which helps to further balance the colors of the image, reduce the color cast of the image, and make the colors of each object in the image more in line with the actual situation.

[0158] Furthermore, the method in this embodiment can generate a standard output image corresponding to the mixed light source image based on the mixed light source image and the target brightness value corresponding to the mixed light source image. The target chromaticity value can be generated by fusing the chromaticity values ​​corresponding to each calibration image and the fusion ratio corresponding to each calibration image. Specifically, the fusion ratio corresponding to each calibration image is generated based on the G channel value of each calibration image.

[0159] For example, combined Figure 9 The three single-light source images (i.e., the first single-light source image 95, the second single-light source image 96, and the third single-light source image 97) are preprocessed to generate the first calibration image 951 (assuming the corresponding G channel value is G). 951 The corresponding chromaticity value is L. 951 ), second calibration image 961 (assuming the corresponding G channel value is G) 961 The corresponding chromaticity value is L. 961 ) and the third calibration image 971 (assuming the corresponding G channel value is G) 971 The corresponding chromaticity value is L. 971 ).

[0160] Furthermore, the fusion weight corresponding to the first calibration image 951 can be determined to be W. 951 =G 951 / (G 951 +G 961 +G971, the fusion weight corresponding to the second calibration image 961 is W961=G961 / G951+G961+G971, and the fusion weight corresponding to the third calibration image 971 is W 971 =G 971 / (G 951 +G 961 +G 971 Therefore, the target chromaticity value can be determined to be L = L. 951 ×W 951 +L 961 ×W 961 +L 971 ×W 971 Furthermore, the method in this application embodiment can generate a standard output image corresponding to the mixed light source image based on the mixed light source image and the target chromaticity value corresponding to the mixed light source image.

[0161] It should be noted that during the training of the local white balance model using the method in this application embodiment, there are several mixed light source images. This application embodiment is only used to illustrate the main training steps of training the local white balance model in detail. Therefore, only a single mixed light source image is used as an example for illustration. Thus, the above example should not be construed as limiting this application.

[0162] As can be seen, the local white balance model in this application uses mixed light source images as training samples, which helps the local white balance model cover more image shooting scenarios and ensures good color balance capabilities for images collected or shot in various complex light source scenarios. This lays the foundation for subsequent color adaptation processing, thereby ensuring the user's shooting experience. Furthermore, the method in this application embodiment can generate training labels for the local white balance model based on mixed light sources and target chromaticity values. By using self-made training data pairs, it helps ensure the iterative effect of the local white balance model and improve its color balance capability.

[0163] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating another process for creating training data for training a local white balance model, provided as an embodiment of this application.

[0164] like Figure 10 As shown, based on a single mixed-light source image, the method of this application embodiment can perform lens shading correction processing, conversion to three-channel PNG (i.e., the above-mentioned format conversion operation), and white balance calibration processing on each single-light source image corresponding to the mixed-light source image. The method of this application embodiment can also perform G-channel value (e.g., based on the corresponding single-light source image) based on the G-channel value (e.g., ...) of each single-light source image. Figure 10 G1, G2 and G n ), determine the fusion ratio corresponding to each calibration image (e.g. Figure 10 W1, W2 and W n ).

[0165] Specifically, the calculation process for the fusion ratio can be referred to as follows:

[0166] W1 = G1 / (G1 + G2 + ... + G n W2 = G2 / (G1 + G2 + ... + G) n ...W n =G n / (G1+G2+…+G n )

[0167] Furthermore, the method in this application embodiment can be based on the chromaticity values ​​corresponding to each image block (e.g., Figure 10 L1, L2 and L n The target chromaticity value corresponding to the mixed light source image is generated by combining the chromaticity ratio with the blending ratio.

[0168] Specifically, the calculation process for the target chromaticity value can be referenced as follows:

[0169] L 1～n = L1×W1+L2×W2+…+L n ×W n

[0170] Furthermore, the method in this application embodiment can generate a standard output image corresponding to the mixed light source (i.e., based on the mixed light source image and the target chromaticity value corresponding to the mixed light source image) Figure 10 (in the hybrid light source GT).

[0171] S704: Using images of each mixed light source as training data and the standard output images corresponding to each mixed light source as labels, the neural network is trained to obtain a local white balance model.

[0172] It should be noted that, Figure 7 The trained local white balance model can be used to achieve Figures 2-6 The corresponding color adaptation-based image processing method and its related embodiments are described.

[0173] Please see Figure 11 , Figure 11 This is a schematic diagram of the composition of an electronic device provided in an embodiment of this application.

[0174] like Figure 11 As shown, the electronic device may include: an interaction module 810, an image acquisition module 820, and an image processing module 830;

[0175] The interaction module 810 can be used to receive shooting instructions from the user;

[0176] The image acquisition module 820 can be used to acquire RAW images in response to the user's shooting command;

[0177] The image processing module 830 can be used to perform white balance processing on RAW images and generate a first processed image corresponding to the RAW image.

[0178] The image processing module 830 can also be used to cut the first processed image into i image blocks, where i ≥ 2 and i is a positive integer;

[0179] The image processing module 830 can also be used to perform color adaptation processing on each image block in the i image blocks in parallel based on the color information of each image block in the i image blocks, and generate a second processed image corresponding to each image block.

[0180] The image processing module 830 can also be used to stitch together the second processed images corresponding to each image block to generate a target image.

[0181] In some possible implementations, the electronic device may further include:

[0182] The image processing module 830 can also be used to determine the color adaptation gain matrix corresponding to each image block based on the color information of the pixels contained in each image block. The color information may include color temperature value, brightness coefficient and chromaticity value.

[0183] The image processing module 830 can also be used to generate a second processed image corresponding to each image block based on the color adaptation gain matrix corresponding to each pixel block.

[0184] In some other possible implementations, the electronic device may further include:

[0185] The image processing module 830 can also be used to perform color adaptation processing on each image block in parallel in a multi-process manner.

[0186] In some other possible implementations, the electronic device may further include:

[0187] The image processing module 830 can also be used to input RAW images into a local white balance model and output a first processed image.

[0188] In some other possible implementations, the local white balance model is a model trained using mixed light source images as training samples and the standard output image corresponding to the mixed light source images as labels.

[0189] The scene corresponding to the mixed light source image contains n light sources, where n ≥ 2 and n is a positive integer.

[0190] The standard output image corresponding to the mixed light source image is generated based on the mixed light source image and the target chromaticity value.

[0191] In some other possible implementations, the target chromaticity value is generated by fusing the chromaticity values ​​of each of the n calibration images and the fusion ratio of each calibration image, with a one-to-one correspondence between the n calibration images and the n light sources.

[0192] In some other possible implementations, any one of the n calibration images is generated by preprocessing the single-light source image corresponding to any one of the calibration images. The preprocessing may include lens shading correction processing, image format conversion, and white balance calibration processing.

[0193] The shooting scene corresponding to any of the above calibration images contains a light source that corresponds to any of the above calibration images.

[0194] In some other possible implementations, the image is calibrated as a PNG image;

[0195] The fusion ratio corresponding to each calibration image is generated based on the G channel value of each calibration image.

[0196] Further, please see Figure 12 , Figure 12 This illustration shows a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application.

[0197] Electronic device 100 may include a processor 101, a memory 102, a wireless communication module 103, a mobile communication module 104, an antenna 103A, an antenna 104A, a power switch 105, a sensor module 106, a focusing motor 107, a camera 108, a display screen 109, etc. The sensor module 106 may include a gyroscope sensor 106A, an accelerometer sensor 106B, an ambient light sensor 106C, an image sensor 106D, a proximity sensor 106E, etc. The wireless communication module 103 may include a WLAN communication module, a Bluetooth communication module, etc. All of the above components can transmit data via a bus.

[0198] Processor 101 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0199] Memory 102 can be used to store computer executable program code, which may include instructions. Processor 101 executes various functional applications and data processing of electronic device 100 by running the instructions stored in memory 102. Memory 102 may include a program storage area and a data storage area. In specific implementations, memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0200] The wireless communication function of the electronic device 100 can be implemented through antenna 103A, antenna 104A, mobile communication module 104, wireless communication module 103, modem processor, and baseband processor.

[0201] Antennas 103A and 104A can be used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0202] The mobile communication module 104 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on electronic devices 100.

[0203] The wireless communication module 103 can provide solutions for wireless communication applications on electronic devices 100, including wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR).

[0204] The gyroscope sensor 106A can be used to determine the motion attitude of the electronic device 100.

[0205] Accelerometer 106B can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes).

[0206] Electronic device 100 can perform shooting functions through ISP, camera 108, video codec, GPU, display screen 109 and application processor.

[0207] Electronic device 100 can implement display functions through a GPU, display screen 109, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 109 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 101 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0208] The display screen 109 is used to display images, videos, etc. The display screen 109 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 109, where N is a positive integer greater than 1.

[0209] The structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0210] In the embodiments of this application:

[0211] One or more of the following components can be used to detect the current environmental characteristics of the electronic device 100: wireless communication module 103, mobile communication module 104, sensor module 106, focusing motor 107, camera 108, etc.

[0212] The display screen 109 is used to display the user interface provided by the aforementioned electronic device 100, such as a camera interface or a photo album interface. The user interface displayed on the display screen 109 can be referred to the UI embodiment described above.

[0213] For details on the operations performed by each device in the electronic device 100, please refer to the relevant descriptions in the preceding method embodiments.

[0214] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered mobile operating system as an example to exemplify the software structure of electronic device 100.

[0215] It is understood that the software system of electronic device 100 may adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, and this application does not limit it in this regard. For ease of understanding, please refer to the example provided. Figure 13 , Figure 13 This is a software structure block diagram provided in an embodiment of this application. For example... Figure 13 The layered architecture shown divides the system into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom: application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer.

[0216] The application layer may include a series of application packages. In this embodiment, the application package may include a camera, a gallery, etc.

[0217] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer may include a camera access interface, which may include camera management and camera devices. The camera access interface is used to provide application programming interfaces and programming frameworks for camera applications.

[0218] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In this embodiment, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0219] The camera hardware abstraction layer can provide virtual hardware for camera device 1, camera device 2, or more camera devices. The camera algorithm library may include runtime code and data that implement the shooting methods provided in the embodiments of this application.

[0220] The driver layer is the layer between hardware and software. It includes drivers for various hardware components, such as camera drivers, digital signal processor drivers, and image processor drivers.

[0221] The camera device driver is used to drive the camera sensor to acquire images and to drive the image signal processor to preprocess the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0222] The hardware layer is located below the driver layer. The hardware layer includes sensors, image signal processors, digital signal processors, and image processors. The sensors may include different types of sensors, such as Sensor 1, Sensor 2, Time-of-Flight (TOF) sensors, and multispectral sensors; this application does not limit the type of sensor.

[0223] Specifically, in response to a user's action of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface in the application framework layer to launch the camera application. It then sends a command to start the camera by calling the camera device (Camera Device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this command to the camera device driver in the kernel layer. This camera device driver can then start the corresponding camera sensor and acquire image light signals through the sensor. One camera device in the camera hardware abstraction layer corresponds to one camera sensor in the hardware layer.

[0224] Then, the camera sensor can transmit the acquired image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the raw image to the camera hardware abstraction layer through the camera device driver.

[0225] The camera hardware abstraction layer may include a camera algorithm library for sending raw images. The camera algorithm library stores program code that implements the shooting methods provided in the embodiments of this application. Based on a digital signal processor and an image processor, the camera algorithm library executes the above code to achieve the object recognition and labeling, subject tracking, and close-up image extraction capabilities described above.

[0226] The camera algorithm library can identify and send raw images captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer can then display these images. Simultaneously, the camera algorithm library can also output the center point of objects in the identified image frames, as well as close-up images centered on the main subject. This allows the camera application to display selection boxes on the raw images based on the object center point, and to display close-up images in a small window.

[0227] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0228] This application also provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program; the processor may be used to invoke the computer program in the memory, causing the electronic device to execute the method executed on the electronic device side in any of the above embodiments.

[0229] This application also provides a chip system including at least one processor for implementing the functions involved on the electronic device side in any of the above embodiments.

[0230] In some possible designs, the chip system also includes a memory for storing program instructions and data, which may be located inside or outside the processor.

[0231] The chip system can consist of chips or include chips and other discrete components.

[0232] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0233] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0234] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0235] This application also provides a computer program product, the computer program product comprising: a computer program (also referred to as code or instructions), which, when the computer program is run, causes the computer to perform the method executed on the electronic device side in any of the above embodiments.

[0236] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed on the electronic device side in any of the above embodiments.

[0237] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0238] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0239] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0240] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0241] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. An image processing method based on color adaptation, characterized in that, The method includes: In response to the user's shooting command, acquire RAW images; The RAW image is subjected to white balance processing to generate a first processed image corresponding to the RAW image; The first processed image is cut into i image blocks, where i ≥ 2 and i is a positive integer; Based on the color information of each image block in the i image blocks, color adaptation processing is performed on each image block in parallel to generate a second processed image corresponding to each image block; The second processed images corresponding to each image block are stitched together to generate the target image; The step of performing white balance processing on the RAW image to generate a first processed image corresponding to the RAW image includes: The RAW image is input into a local white balance model, and the first processed image is output. The local white balance model is a model trained using a mixed light source image as a training sample and a standard output image corresponding to the mixed light source image as a label. There are n light sources in the shooting scene corresponding to the mixed light source image, where n ≥ 2 and n is a positive integer. The standard output image corresponding to the mixed light source image is generated based on the mixed light source image and the target chromaticity value.

2. The method according to claim 1, characterized in that, The step of performing parallel color adaptation processing on each image block based on the color information corresponding to each image block in the i image blocks to generate a second processed image corresponding to each image block includes: Based on the color information of the pixels contained in each image block, the color adaptation gain matrix corresponding to each image block is determined. The color information includes color temperature value, luminance coefficient and chromaticity value. Based on the color adaptation gain matrix corresponding to each pixel block, a second processed image corresponding to each image block is generated.

3. The method according to claim 1 or 2, characterized in that, The step of performing color adaptation processing on each image block separately includes: Color adaptation processing is performed on each image block in parallel using a multi-process approach.

4. The method according to claim 3, characterized in that, The target chromaticity value is generated based on the chromaticity value of each of the n calibration images and the fusion ratio of each calibration image. There is a one-to-one correspondence between the n calibration images and the n light sources.

5. The method according to claim 4, characterized in that, Any one of the n calibration images is generated by preprocessing the single-light source image corresponding to the calibration image. The preprocessing includes lens shadow correction processing, image format conversion, and white balance calibration processing. The shooting scene corresponding to any single light source image of any calibration image contains a light source corresponding to any one of the calibration images.

6. The method according to claim 5, characterized in that, The calibration image is a PNG image; The fusion ratio corresponding to each calibration image is generated based on the G channel value of each calibration image.

7. A method for training a local white balance model, characterized in that, The method includes: Obtain a training dataset, which includes m mixed light source images and n single light source images corresponding to each of the m mixed light source images. There are n light sources in the shooting scene corresponding to the mixed light source images. There is a one-to-one correspondence between the single light source images and the light sources. n≥2, where n is a positive integer and m≥2, where m is a positive integer. Based on the chromaticity values ​​of the n single-light source images corresponding to each mixed light source image, determine the target chromaticity value corresponding to each mixed light source image; Based on each mixed light source image and the target chromaticity value corresponding to each mixed light source image, generate a standard output image corresponding to each mixed light source image; Using images of each mixed light source as training data and the standard output images corresponding to each mixed light source as labels, the neural network is trained to obtain a local white balance model.

8. The method according to claim 7, characterized in that, The step of determining the target chromaticity value corresponding to each mixed light source image based on the chromaticity values ​​of n single light source images corresponding to each mixed light source image includes: Preprocessing is performed on n single-light source images corresponding to each mixed light source image to generate n calibration images corresponding to each mixed light source image. The calibration images have a one-to-one correspondence with the single-light source images. The preprocessing includes lens shadow correction processing, image format conversion, and white balance calibration processing. Based on the chromaticity values ​​of each of the n calibration images corresponding to each mixed light source image, and the fusion ratio of each calibration image, the target chromaticity value corresponding to each mixed light source image is determined.

9. The method according to claim 8, characterized in that, The calibration image is a PNG image; The method further includes: Based on the G-channel values ​​of each calibration image corresponding to each mixed light source image, the fusion ratio of each calibration image corresponding to each mixed light source image is determined.

10. An electronic device, characterized in that, The electronic device includes: an interaction module, an image acquisition module, and an image processing module; The interaction module is used to receive the user's shooting instructions; The image acquisition module is used to acquire RAW images in response to the user's shooting command; The image processing module is used to perform white balance processing on the RAW image to generate a first processed image corresponding to the RAW image; The image processing module is further configured to cut the first processed image into i image blocks, where i ≥ 2 and i is a positive integer; The image processing module is also used to perform color adaptation processing on each image block in the i image blocks simultaneously in a multi-process manner to generate a second processed image corresponding to each image block; The image processing module is also used to stitch together the second processed images corresponding to each image block to generate a target image; The image processing module is further configured to input the RAW image into a local white balance model and output the first processed image. The local white balance model is a model trained using a mixed light source image as a training sample and a standard output image corresponding to the mixed light source image as a label. There are n light sources in the shooting scene corresponding to the mixed light source image, where n ≥ 2 and n is a positive integer. The standard output image corresponding to the mixed light source image is generated based on the mixed light source image and the target chromaticity value.

11. An electronic device, characterized in that, The device includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions such that the method as described in any one of claims 1-6 is executed.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the method as described in any one of claims 1-6 to be performed.

Images