Image processing method and device, electronic equipment, storage medium and program product
By synchronously acquiring images and converting white balance decision points between the reference camera component and the target camera component, the problem of inconsistent colors between different camera components in the same scene is solved, achieving consistency in white balance and image color, and improving the smooth zoom effect when switching camera components.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2024-10-29
- Publication Date
- 2026-05-01
AI Technical Summary
In the same scene, different camera components receive different input information, resulting in different results calculated by their respective white balance algorithms, leading to inconsistent image colors.
By simultaneously acquiring images from a reference camera component and a target camera component, the first white balance decision point of the reference image is determined, and the second white balance decision point of the target camera component is obtained through synchronous conversion. The white balance gain coefficient is calculated, and the white balance of the image to be processed is adjusted to achieve white balance consistency among different camera components.
It achieves white balance consistency and image color consistency among different camera components in the same scene, improves the smooth zoom effect when switching camera components, and enhances the user experience.
Smart Images
Figure CN121967906A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, specifically to an image processing method, apparatus, electronic device, storage medium, and program product. Background Technology
[0002] With the development of smart devices, electronic devices are usually equipped with camera components with multiple focal lengths (such as wide-angle lenses, ultra-wide-angle lenses, and telephoto lenses), and optical zoom functions are achieved by switching between camera components.
[0003] However, since the input information (e.g., captured images) received by different camera components are usually different, the white balance results calculated by different camera components using their respective white balance algorithms are usually different. Therefore, even in the same scene, the images obtained by different camera components after image processing based on their respective white balance results will have inconsistent colors. Summary of the Invention
[0004] The purpose of this disclosure is to provide an image processing method, apparatus, electronic device, storage medium, and program product to achieve image color consistency among different camera components when multiple camera components are provided.
[0005] In one aspect, this disclosure provides an image processing method, including:
[0006] After receiving a camera switching command to switch to the target camera component, images are acquired by the reference camera component and the target camera component in the electronic device, respectively, to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component.
[0007] Determine the first white balance decision point corresponding to the reference image; the first white balance decision point is the reference color value used for white balance setting of the reference image.
[0008] Based on the reference image and the first white balance decision point, determine the second white balance decision point corresponding to the image to be processed; the second white balance decision point is the reference color value used for the white balance setting of the image to be processed.
[0009] Based on the second white balance decision point, calculate the white balance gain coefficient corresponding to the image to be processed; the white balance gain coefficient is negatively correlated with the second white balance decision point.
[0010] Based on the white balance gain coefficient, the white balance of the image to be processed is adjusted to obtain a white balance image.
[0011] In one embodiment, determining a second white balance decision point corresponding to the image to be processed based on a reference image and a first white balance decision point includes:
[0012] The first white region in the reference image is determined based on the color values of each pixel block in the reference image and the first white balance decision point.
[0013] Identify the first white region and the matching second white region in the image to be processed;
[0014] The synchronization conversion coefficient is determined based on the color values of the first white area and the second white area;
[0015] The second white balance decision point is determined based on the first white balance decision point and the synchronization conversion coefficient.
[0016] In one embodiment, determining a first white region in the reference image based on the color values of each pixel block in the reference image and a first white balance decision point includes:
[0017] Based on the color value of each pixel block in the reference image, the pixel blocks are clustered to obtain the color category corresponding to each pixel block.
[0018] Based on the color value corresponding to each color category and the similarity between it and the first white balance decision point, the target color category is selected from each color category.
[0019] The first white region is obtained based on the pixel block corresponding to the target color category in the reference image.
[0020] In one embodiment, before clustering the pixel blocks according to their respective color values in the reference image, the method further includes:
[0021] For each target image in both the reference image and the image to be processed, perform the following steps:
[0022] If the target image meets the cropping criteria, then obtain the corresponding calibration coordinates of the target image;
[0023] The target image is cropped based on the calibrated coordinates.
[0024] In one embodiment, before obtaining the calibration coordinates corresponding to the target image, the method further includes:
[0025] Acquire the field of view of each of the multiple camera components of the electronic device; the multiple camera components include a reference camera component and a target camera component;
[0026] Determine the smallest field of view among all field of view angles;
[0027] Based on the minimum field of view, the calibration coordinates corresponding to each camera component are determined.
[0028] In one embodiment, obtaining a first white region based on the pixel block corresponding to the target color category in the reference image includes:
[0029] At least one image region is formed based on the pixel blocks in the reference image corresponding to the target color category;
[0030] The largest area in each image region is designated as the first white area.
[0031] In one embodiment, before determining the largest region in each image region as the first white region, the method further includes:
[0032] If there are multiple image regions and there are adjacent image regions that meet the conditions for connected regions, then the image regions that meet the conditions for connected regions will be merged into the same image region.
[0033] In one embodiment, white balance adjustment is performed on the image to be processed based on the white balance gain coefficient to obtain a white balance image, including:
[0034] The white balance image is obtained by multiplying the color value of each pixel in the image to be processed by the white balance gain coefficient.
[0035] In one aspect, this disclosure provides an image processing apparatus, comprising:
[0036] The acquisition unit is used to acquire images through the reference camera component and the target camera component in the electronic device after receiving a camera switching instruction for switching to the target camera component, so as to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component.
[0037] The determining unit is used to determine the first white balance decision point corresponding to the reference image; the first white balance decision point is a reference color value used for white balance setting of the reference image.
[0038] The conversion unit is used to determine the second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point is a reference color value used for white balance setting of the image to be processed.
[0039] The calculation unit is used to calculate the white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient is negatively correlated with the second white balance decision point.
[0040] The adjustment unit is used to adjust the white balance of the image to be processed according to the white balance gain coefficient to obtain a white balance image.
[0041] In one embodiment, the conversion unit is used for:
[0042] The first white region in the reference image is determined based on the color values of each pixel block in the reference image and the first white balance decision point.
[0043] Identify the first white region and the matching second white region in the image to be processed;
[0044] The synchronization conversion coefficient is determined based on the color values of the first white area and the second white area;
[0045] The second white balance decision point is determined based on the first white balance decision point and the synchronization conversion coefficient.
[0046] In one embodiment, the conversion unit is used for:
[0047] Based on the color value of each pixel block in the reference image, the pixel blocks are clustered to obtain the color category corresponding to each pixel block.
[0048] Based on the color value corresponding to each color category and the similarity between it and the first white balance decision point, the target color category is selected from each color category.
[0049] The first white region is obtained based on the pixel block corresponding to the target color category in the reference image.
[0050] In one embodiment, the conversion unit is further configured to:
[0051] For each target image in both the reference image and the image to be processed, perform the following steps:
[0052] If the target image meets the cropping criteria, then obtain the corresponding calibration coordinates of the target image;
[0053] The target image is cropped based on the calibrated coordinates.
[0054] In one embodiment, the conversion unit is further configured to:
[0055] Acquire the field of view of each of the multiple camera components of the electronic device; the multiple camera components include a reference camera component and a target camera component;
[0056] Determine the smallest field of view among all field of view angles;
[0057] Based on the minimum field of view, the calibration coordinates corresponding to each camera component are determined.
[0058] In one embodiment, the conversion unit is used for:
[0059] At least one image region is formed based on the pixel blocks in the reference image corresponding to the target color category;
[0060] The largest area in each image region is designated as the first white area.
[0061] In one embodiment, the conversion unit is further configured to:
[0062] If there are multiple image regions and there are adjacent image regions that meet the conditions for connected regions, then the image regions that meet the conditions for connected regions will be merged into the same image region.
[0063] In one embodiment, the adjustment unit is used for:
[0064] The white balance image is obtained by multiplying the color value of each pixel in the image to be processed by the white balance gain coefficient.
[0065] In one aspect, this disclosure provides an electronic device, including:
[0066] Processor; and
[0067] The memory stores computer instructions that cause the processor to perform the steps of the methods provided in the various alternative implementations of any of the image processing described above.
[0068] In one aspect, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described image processing methods.
[0069] On one hand, this disclosure provides a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the steps of the method provided in various alternative implementations of any of the above-described image processing methods.
[0070] The image processing method in this embodiment includes, after receiving a camera switching command for switching to a target camera component, acquiring images through a reference camera component and a target camera component in an electronic device to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component; determining a first white balance decision point corresponding to the reference image; the first white balance decision point being a reference color value used for white balance setting of the reference image; determining a second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point being a reference color value used for white balance setting of the image to be processed; calculating a white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient being negatively correlated with the second white balance decision point; and adjusting the white balance of the image to be processed based on the white balance gain coefficient to obtain a white balance image. In this way, by simultaneously operating the reference camera component and the target camera component, and converting the first white balance decision point of the reference camera component to obtain the second white balance decision point of the target camera component, thereby determining the white balance gain coefficient and image adjustment, white balance consistency and image color consistency can be achieved between different camera components. Attached Figure Description
[0071] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure.
[0072] Figure 2 This is a comparison diagram of the FOV area of a multi-lens camera in an embodiment of this disclosure.
[0073] Figure 3 This is an example diagram of image calibration in an embodiment of this disclosure.
[0074] Figure 4 This is an example diagram of a reference image downsampling in an embodiment of this disclosure.
[0075] Figure 5 This is a coordinate graph of the color distribution of a reference image in an embodiment of this disclosure.
[0076] Figure 6 This is an example diagram of a reference image color category in an embodiment of this disclosure.
[0077] Figure 7 This is an example diagram of a target color category distribution in an embodiment of this disclosure.
[0078] Figure 8 This is an example diagram of a first white area in an embodiment of this disclosure.
[0079] Figure 9 This is an example diagram of a second white area in an embodiment of this disclosure.
[0080] Figure 10 This is a flowchart of an image imaging method according to an embodiment of the present disclosure.
[0081] Figure 11 This is a structural block diagram of an image processing apparatus according to an embodiment of the present disclosure.
[0082] Figure 12 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation
[0083] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.
[0084] In practical applications, electronic devices typically incorporate camera components with multiple focal lengths, achieving optical zoom by switching between these components. However, because different camera components often receive different input information (e.g., captured images), the white balance results calculated by each component using its own white balance algorithm are usually different. Therefore, even in the same scene, the images obtained after image processing based on their respective white balance results by different camera components will exhibit color inconsistencies.
[0085] Based on the deficiencies of the aforementioned related technologies, this disclosure provides an image processing method, apparatus, electronic device, storage medium, and program product, which aims to achieve image color consistency among different camera components when multiple camera components are provided.
[0086] This disclosure provides an image processing method that can be applied to electronic devices. This disclosure does not limit the type of electronic device, which can be any suitable type of device, such as terminal devices and servers, etc. This disclosure will not elaborate further.
[0087] See Figure 1 The diagram shown is a flowchart of an image processing method according to an embodiment of this disclosure. The following is a description of the method in conjunction with... Figure 1 The method is described below, and the specific implementation process is as follows:
[0088] Step 101: After receiving the camera switching instruction for switching to the target camera component, images are acquired through the reference camera component and the target camera component respectively to obtain the reference image acquired by the reference camera component and the image to be processed acquired by the target camera component.
[0089] In one embodiment, the electronic device includes multiple camera components, with one of these components designated as a reference camera component, also known as a master camera component. The other camera components besides the reference camera component can be referred to as slave camera components, and there can be at least one. The focal length of the reference camera component is typically different from that of the other camera components. For example, the reference camera component can be a wide-angle lens, while the target camera component can be a telephoto lens. In practical applications, any reference camera component can be selected based on the specific application scenario, and no restrictions are placed here.
[0090] During actual shooting, the reference camera component is typically enabled by default, while other camera sub-components are disabled. When the user wants to zoom, a camera switching command is issued to switch to the target camera component. The target camera component is one of the other camera components besides the reference camera component. When the electronic device receives the camera switching command, it activates the user-selected target camera component according to the command and acquires images through both the reference camera component and the target camera component.
[0091] In this setup, the reference camera component and the target camera typically acquire images simultaneously. Upon receiving the aforementioned camera switching command, the reference camera component remains running in the background, capturing images that serve as reference images. The target camera component runs in the foreground, and the images displayed on the screen are captured by the target camera component.
[0092] In this way, a reference image can be acquired simultaneously through the reference camera component, and the image to be processed can be acquired through the target camera component.
[0093] Step 102: Determine the first white balance decision point corresponding to the reference image; the first white balance decision point is the reference color value used for white balance setting of the reference image.
[0094] In one implementation, the color temperature is determined by a white balance algorithm, and a first white balance decision point of the reference image is determined based on the color temperature.
[0095] In practical applications, the white balance algorithm and the method for determining the first white balance decision point can be set according to the actual application scenario, and no restrictions are imposed here. Optionally, the reference color value can be represented as three-dimensional color coordinates (r, g, b) or two-dimensional color coordinates (x / z, y / z).
[0096] Here, r, g, and b represent the color components of the Red, Green, and Blue (RGB) channels, respectively. z is the color component of the reference channel selected from the RGB channels, and x and y are the color components corresponding to the other two channels. In practical applications, the green channel is usually used as the reference channel, so that in subsequent steps, only the red and blue channels are adjusted to match the green channel, thereby achieving white balance.
[0097] In this way, the color temperature of the scene can be determined by referring to the camera components, and the first white balance decision point can be determined based on the color temperature determination result.
[0098] Step 103: Determine the second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point is the reference color value used for white balance setting of the image to be processed.
[0099] In one implementation, step 103 may be performed using the following steps:
[0100] S1031: Determine the first white region in the reference image based on the color values of each pixel block in the reference image and the first white balance decision point.
[0101] In one implementation, when executing S1031, the following steps may be taken:
[0102] S1031-1: Based on the color values of each pixel block in the reference image, cluster the pixel blocks to obtain the color category corresponding to each pixel block.
[0103] Furthermore, before performing S1031-1, the image can be cropped. In one embodiment, when cropping the image, the following steps are performed for each target image in both the reference image and the image to be processed:
[0104] If the target image meets the cropping conditions, the corresponding calibration coordinates of the target image are obtained, and the target image is cropped according to the calibration coordinates to crop out the region of interest that includes the calibration coordinates.
[0105] In one implementation, the following steps can be used to determine the calibration coordinates:
[0106] S1031-11: Acquire the field of view of each of the multiple camera components of the electronic device; the multiple camera components include a reference camera component and a target camera component.
[0107] S1031-12: Determine the smallest field of view among all field of view angles.
[0108] S1031-13: Determine the calibration coordinates corresponding to each camera component based on the minimum field of view.
[0109] For example, lenses (i.e., camera components) in electronic devices include: ultra-wide (UW) lenses, wide (W) lenses, and telephoto lenses. UW lenses have a magnification of 0.5x, wide lenses have a magnification of 1x, and telephoto lenses have a magnification of 3x.
[0110] The following is combined Figure 2 A comparative explanation of the field of view (FOV) regions of cameras in electronic devices. Figure 2 This is a comparison diagram of the FOV area of a multi-camera setup. Figure 2 The display shows the FOV areas corresponding to the UW lens, Wide lens, and Tele lens. Since the Tele lens has the smallest FOV, its FOV area is the smallest overlapping area among all FOV areas, i.e., the multi-camera overlapping ROI area. This area is then taken as the Region of Interest (ROI), and the coordinates of the ROI area in the FOV area of each lens are determined as the corresponding calibration coordinates. The calibration coordinates of each lens are recorded in the engineering parameters to achieve parameter configuration.
[0111] The following is combined Figure 3 An example is provided to illustrate the calibration coordinates. Figure 3 This is an example diagram for image calibration. Figure 3 (a) is a reference image. Figure 3 (b) is the image to be processed. Figure 3 (c) shows the calibration coordinates corresponding to the reference image and the first ROI region it calibrates. Figure 3 (d) shows the calibration coordinates of the image to be processed and the second ROI region it calibrates.
[0112] The first ROI region can be represented as:
[0113] ROI Master ={(a left b top ), (a right b top ), (a left b bottom， ), (a right b bottom )).
[0114] The second ROI region can be represented as:
[0115] ROI Salver={(c left d top ), (c right d top ), (c left d bottom， ), (c right d bottom )).
[0116] Furthermore, the reference image can be divided to obtain multiple pixel blocks and their corresponding color values.
[0117] In one implementation, downsampling is used to divide the pixels in the reference image into M*N pixel blocks. For each pixel block, the average color value of each pixel in the block is determined (e.g., the average value of each color component is calculated separately) to obtain the color value of the pixel block. M and N are both positive integers.
[0118] The following is combined Figure 4 This section provides examples illustrating image cropping and downsampling. Figure 4 This is an example image of a reference image downsampled. Figure 4 In the reference image (a), the first ROI region is identified, which can also be called the ROI. Master Region, and crop the ROI from the reference image. Master Region, obtain Figure 4 (b), and on Figure 4 (b) Perform downsampling to obtain Figure 4 (c)
[0119] In one implementation, when clustering and classifying pixel blocks according to their respective color values in a reference image to obtain the color category corresponding to each pixel block, the following steps can be used:
[0120] A clustering algorithm is used to divide each pixel block into clusters to obtain the color category corresponding to each pixel block.
[0121] Optionally, the clustering algorithm can be the K-means algorithm. In practical applications, the clustering algorithm can be set according to the actual application scenario, and there are no restrictions here.
[0122] S1031-2: Select the target color category from each color category based on the similarity between the color value corresponding to each color category and the first white balance decision point.
[0123] In one embodiment, for each color category, the average value of the respective color values of at least one pixel block corresponding to the color category is determined to obtain the two-dimensional color coordinates corresponding to the color category. The distance (i.e., similarity) between the two-dimensional color coordinates corresponding to each color category and the first white balance decision point is determined, and the color category corresponding to the smallest distance among all distances is determined as the target color category.
[0124] The color value corresponding to color category i can be represented as follows: The color values corresponding to n color categories can be represented as follows:
[0125]
[0126] Where n is a positive integer, r i b is the color component of the red channel corresponding to color category i. i For the blue channel color component corresponding to color category i, g i This refers to the color component of the green channel corresponding to color category i.
[0127] The following is combined Figure 5 An example is provided to illustrate pixel block clustering. Figure 5 This is a coordinate graph of the color distribution of a reference image, with the horizontal axis representing r / g and the horizontal axis representing b / g. Based on the two-dimensional color coordinates of each pixel block, in... Figure 5 The graph displays the distribution of each pixel block in a coordinate map, and based on this distribution, the pixel blocks are clustered to obtain multiple color categories. For example, the color categories can be 1, 2...n, where n can be 8.
[0128] The following is combined Figure 6 Examples are provided to illustrate the color categories of each pixel block in the reference image. Figure 6 This is an example image representing a reference image color category. Figure 6 The text displays the color category of each pixel block in the reference image, i.e., 1, 2, 3...8. It should be noted that... Figure 6 This image is for instruction manual purposes only. The reference image contains multiple color categories. If the text or lines are unclear, it will not affect the clarity of the instruction manual.
[0129] The following is combined Figure 7 An example is provided to illustrate the determination of the target color category. Figure 7 This is an example diagram showing the distribution of a target color category. Figure 7 The image displays the color categories of each pixel block distributed in the reference image, namely color category 1, color category 2, color category 3... color category 8, as well as the first white balance decision point dp, and determines color category 2, which is closest to the first white balance decision point, as the target color category.
[0130] Since the first white balance decision point is used as the white reference standard in the reference image, then Figure 7 The closer the coordinates are to the first white balance decision point, the closer the corresponding color is to white. Since color category 2 is closest to the first white balance decision point, color category 2 has the highest probability of being white.
[0131] S1031-3: Obtain the first white region based on the pixel block corresponding to the target color category in the reference image.
[0132] In one implementation, when performing S1031-3, the following steps may be adopted:
[0133] S1031-31: Based on the pixel blocks corresponding to the target color category in the reference image, form at least one image region.
[0134] S1031-32: If there is only one image region, then the image region is determined as the first white region;
[0135] S1031-33: If there are multiple image regions, the largest region among all image regions shall be determined as the first white region.
[0136] Furthermore, to reduce the difficulty of subsequent image matching, adjacent image regions can be connected. In one embodiment, if there are multiple image regions and there are adjacent image regions that meet the connectivity condition, then the image regions that meet the connectivity condition are merged into the same image region.
[0137] For example, for each pair of image regions, if the two image regions are adjacent, then they are determined to meet the conditions for a connected region, and the two image regions are determined to be the same image region, that is, a connected region.
[0138] The following is combined Figure 6 as well as Figure 8 An example is provided for the first white area. (See attached image.) Figure 8 The image shown is an example of a first white area.
[0139] Will Figure 6 Multiple image regions are formed by pixel blocks of color category 2, and the result is obtained. Figure 8 (a), and merge adjacent image regions in Figure (a), and select the largest image region from each image region, namely the first white region, i.e. Figure 8 (b)
[0140] S1032: Determine the first white region and the second white region to be matched in the image to be processed.
[0141] The following is combined Figure 9 An example is provided for the second white area. Figure 9 This is an example diagram of a second white area. Figure 9 (a) shows a reference image and its first white area, according to Figure 9 (a) The first white region is identified, and the corresponding second white region in the image to be processed is determined, thus obtaining... Figure 9 (b)
[0142] This is because the content of the ROI regions in the reference image and the image to be processed is similar. Therefore, the second white region in the image to be processed can be determined using an image matching algorithm. In practical applications, the image matching algorithm can be set according to the actual application scenario, and no restrictions are imposed here.
[0143] S1033: Determine the synchronization conversion coefficient based on the color values of the first white area and the second white area.
[0144] In one embodiment, the color value of the first white region is obtained based on the average color value of each pixel in the first white region. Similarly, the color value of the second white region is determined, and the synchronization conversion coefficient is obtained based on the ratio between the color value of the second white region and the color value of the first white region.
[0145] For example, the corresponding two-dimensional color coordinates can be determined based on the color value of the first white area. And based on the color value of the second white area, determine the corresponding two-dimensional color coordinates. Furthermore, the synchronization conversion coefficient trans_AB can be determined using the following formula. ill :
[0146]
[0147] Where, r white_S b is the color component of the red channel in the first white region. white_s The color component of the blue channel in the first white region; g white_s The green channel color component of the first white region; r white_M b is the color component of the red channel in the second white area. white_M For the blue channel color component of the second white region, g white_M The color component of the green channel for the second white area.
[0148] S1034: Determine the second white balance decision point based on the first white balance decision point and the synchronization conversion coefficient.
[0149] In one implementation, the second white balance decision point is obtained by multiplying the first white balance decision point by the synchronization conversion coefficient.
[0150] The following formula can be used to determine the second white balance decision point:
[0151] DP Slaver =DP Master *trans_AB ill ;
[0152]
[0153]
[0154] Among them, DP Master As the first white balance decision point, DP Slaver As the second white balance decision point, (r dp_M g dp_M b dp_M (r) represents the color components of each channel corresponding to the first white balance decision point. dp_s g dp_S b dp_S ) represents the color components of each channel corresponding to the second white balance decision point.
[0155] In related technologies, each camera component typically performs white balance judgment and adjustment independently. However, in this embodiment, only the reference camera component performs white balance judgment through a white balance algorithm to determine the first white balance decision point. Other camera components do not need to perform white balance judgment through a white balance algorithm. They only need to synchronously convert the white balance judgment result of the reference camera component to the target camera component based on the determined second white balance role point. This saves the white balance judgment time of other camera components and improves processing efficiency.
[0156] Step 104: Calculate the white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient is negatively correlated with the second white balance decision point.
[0157] In one implementation, the white balance gain coefficient AWB is determined. S_gain When this is the case, the following formula can be used:
[0158]
[0159] Step 105: Adjust the white balance of the image to be processed according to the white balance gain coefficient to obtain a white balance image.
[0160] In one embodiment, a white balance image is obtained by multiplying the color value of each pixel in the image to be processed by a white balance gain coefficient.
[0161] Optionally, the following formula can be used to determine the white balance image:
[0162]
[0163] in, Let be the pixel matrix of the image to be processed. This is the pixel matrix of the white balance image.
[0164] This disclosure applies to the automatic white balance (AWB) decision-making stage in the image imaging process. The following describes the application of this embodiment in conjunction with... Figure 10 This section explains the image imaging process. (See also...) Figure 10 The diagram shows a flowchart of an image imaging method. The implementation process of this method includes:
[0165] Step 1001: Perform image acquisition to obtain the acquired image.
[0166] One implementation scheme allows for image acquisition using a sensor with a color filter array.
[0167] Step 1002: Perform gain processing on the image.
[0168] One implementation group can perform International Organization for Standardization (ISO) gain on the image to be processed, as well as raw-image processing.
[0169] Step 1003: Perform RGB mosaic removal on the image.
[0170] Demoasicing is a technique in digital image processing, primarily used to convert monochrome channel image data into a full-color image.
[0171] Step 1004: Perform model recognition and processing on the image.
[0172] Pattern recognition and processing (PRC) is an important branch of computer science and artificial intelligence, primarily involving the identification of meaningful patterns or regularities from data and their further analysis and processing. This field has a wide impact in many practical applications, such as image recognition, speech recognition, and natural language processing. The core concepts, key technologies, and applications of PRC will be detailed below. Pattern recognition: refers to the automatic identification of patterns and regularities in data through algorithms and techniques. For example, in image recognition, the algorithm needs to learn how to distinguish different image categories. Data processing: involves the process of collecting, cleaning, transforming, and analyzing data so that pattern recognition algorithms can work more effectively.
[0173] Step 1005: Adjust the white balance of the image.
[0174] AWB (Auto-Action Brush) is a method for adjusting image colors to adapt to different lighting conditions. Its purpose is to ensure that white or other neutral tones in an image remain white or neutral under different light sources, thereby correcting overall color deviations.
[0175] When performing step 1005, please refer to steps 101-105 above for specific steps, which will not be repeated here.
[0176] Step 1006: Perform color manipulation on the image.
[0177] Color manipulation is a key technology in digital image processing. It involves changing and adjusting the color attributes of an image to achieve specific visual effects or extract image features.
[0178] Step 1007: Map the image to standard RGB output.
[0179] Mapping to standard sRGB output typically involves using a color matrix or lookup table to achieve accurate color mapping, ensuring that images are rendered correctly on various display devices. sRGB is a standard color space widely used in monitors, cameras, and internet images. It defines how digital signals are converted into visible light, resulting in images with consistent visual quality across different devices. This usually involves using a color matrix or lookup table to achieve accurate color mapping.
[0180] Step 1008: Compress the image.
[0181] Optionally, the image can be compressed using the Joint Photographic Experts Group (JPEG) image file format.
[0182] Step 1009: Save the compressed file.
[0183] Under the relevant technology, each camera component usually performs white balance judgment and adjustment independently. However, this results in inconsistencies in white balance due to differences among multiple camera components, which in turn leads to inconsistent image colors among multiple camera components. When switching between camera components, it is difficult to achieve smooth zoom between different camera components, resulting in a poor user experience.
[0184] In this embodiment, multiple camera components operate simultaneously, allowing both the reference and target camera components to run concurrently. The first white balance decision point is determined by the reference image acquired in the background by the reference camera component, and the first white balance decision point is synchronously converted to obtain the second white balance decision point corresponding to the target camera component. This achieves white balance transfer between different camera components, ensuring white balance consistency among multiple camera components in the same scene. Consequently, it ensures the consistency of image colors across multiple camera components and enhances smooth zooming across different camera components.
[0185] Furthermore, after determining the first white balance decision point by referencing the camera components, the largest white area in the reference image, i.e., the first white area, is determined based on this first white balance decision point. Then, through image matching, the corresponding second white area in the image to be processed is determined. Based on the first and second white areas, the synchronization conversion coefficient can be determined. Subsequently, the second white balance decision point can be determined through the white balance conversion coefficient, thus realizing white balance mapping between different camera components. Since the white areas in the image can reflect the spectrum relatively completely, the synchronization conversion coefficient calculated by white point / gray point is usually more accurate, improving the accuracy of synchronization conversion, and thus improving the accuracy of white balance consistency and image consistency.
[0186] Based on the same inventive concept, this disclosure also provides an image processing apparatus. Since the principle of the above-described apparatus and device in solving the problem is similar to that of an image processing method, the implementation of the above-described apparatus can refer to the implementation of the method, and repeated details will not be elaborated further. This apparatus can be applied to electronic devices. This disclosure does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this disclosure. The apparatus embodiment can be implemented by software, or by hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined apparatus, it is formed by the processor of the electronic device loading the corresponding computer program instructions from non-volatile memory into memory for execution.
[0187] See Figure 11 The diagram shown is a structural block diagram of an image processing apparatus according to an embodiment of this disclosure. In some embodiments, the image processing apparatus of this disclosure includes:
[0188] The acquisition unit 1101 is used to acquire images through the reference camera component and the target camera component in the electronic device respectively after receiving the camera switching instruction for switching to the target camera component, and to obtain the reference image acquired by the reference camera component and the image to be processed acquired by the target camera component.
[0189] The determining unit 1102 is used to determine the first white balance decision point corresponding to the reference image; the first white balance decision point is a reference color value used for white balance setting of the reference image.
[0190] The conversion unit 1103 is used to determine the second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point is a reference color value used for white balance setting of the image to be processed.
[0191] The calculation unit 1104 is used to calculate the white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient is negatively correlated with the second white balance decision point;
[0192] The adjustment unit 1105 is used to adjust the white balance of the image to be processed according to the white balance gain coefficient to obtain a white balance image.
[0193] In one embodiment, the conversion unit 1103 is used for:
[0194] The first white region in the reference image is determined based on the color values of each pixel block in the reference image and the first white balance decision point.
[0195] Identify the first white region and the matching second white region in the image to be processed;
[0196] The synchronization conversion coefficient is determined based on the color values of the first white area and the second white area;
[0197] The second white balance decision point is determined based on the first white balance decision point and the synchronization conversion coefficient.
[0198] In one embodiment, the conversion unit 1103 is used for:
[0199] Based on the color value of each pixel block in the reference image, the pixel blocks are clustered to obtain the color category corresponding to each pixel block.
[0200] Based on the color value corresponding to each color category and the similarity between it and the first white balance decision point, the target color category is selected from each color category.
[0201] The first white region is obtained based on the pixel block corresponding to the target color category in the reference image.
[0202] In one embodiment, the conversion unit 1103 is further configured to:
[0203] For each target image in both the reference image and the image to be processed, perform the following steps:
[0204] If the target image meets the cropping criteria, then obtain the corresponding calibration coordinates of the target image;
[0205] The target image is cropped based on the calibrated coordinates.
[0206] In one embodiment, the conversion unit 1103 is further configured to:
[0207] Acquire the field of view of each of the multiple camera components of the electronic device; the multiple camera components include a reference camera component and a target camera component;
[0208] Determine the smallest field of view among all field of view angles;
[0209] Based on the minimum field of view, the calibration coordinates corresponding to each camera component are determined.
[0210] In one embodiment, the conversion unit 1103 is used for:
[0211] At least one image region is formed based on the pixel blocks in the reference image corresponding to the target color category;
[0212] The largest area in each image region is designated as the first white area.
[0213] In one embodiment, the conversion unit 1103 is further configured to:
[0214] If there are multiple image regions and there are adjacent image regions that meet the conditions for connected regions, then the image regions that meet the conditions for connected regions will be merged into the same image region.
[0215] In one embodiment, the adjustment unit 1105 is used to:
[0216] The white balance image is obtained by multiplying the color value of each pixel in the image to be processed by the white balance gain coefficient.
[0217] The image processing method in this embodiment includes, after receiving a camera switching command for switching to a target camera component, acquiring images through a reference camera component and a target camera component in an electronic device to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component; determining a first white balance decision point corresponding to the reference image; the first white balance decision point being a reference color value used for white balance setting of the reference image; determining a second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point being a reference color value used for white balance setting of the image to be processed; calculating a white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient being negatively correlated with the second white balance decision point; and adjusting the white balance of the image to be processed based on the white balance gain coefficient to obtain a white balance image. In this way, by simultaneously operating the reference camera component and the target camera component, and converting the first white balance decision point of the reference camera component to obtain the second white balance decision point of the target camera component, thereby determining the white balance gain coefficient and image adjustment, white balance consistency and image color consistency can be achieved between different camera components.
[0218] In this embodiment of the disclosure, an electronic device is also provided, including:
[0219] Processor; and
[0220] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.
[0221] In this embodiment of the disclosure, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.
[0222] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described in any of the above embodiments.
[0223] Figure 12A schematic diagram of the structure of an electronic device 1200 is shown. (See attached diagram.) Figure 12 As shown, the electronic device 1200 includes a processor 1210 and a memory 1220, and optionally may also include a power supply 1230, a display unit 1240, and an input unit 1250.
[0224] The processor 1210 is the control center of the electronic device 1200. It connects various components through various interfaces and lines, and performs various functions of the electronic device 1200 by running or executing software programs and / or data stored in the memory 1220, thereby performing overall monitoring of the electronic device 1200.
[0225] In this embodiment of the present disclosure, the processor 1210 executes the steps in the above embodiments when it calls the computer program stored in the memory 1220.
[0226] Optionally, processor 1210 may include one or more processing units; preferably, processor 1210 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 1210. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.
[0227] The memory 1220 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 1200, etc. In addition, the memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0228] Electronic device 1200 also includes a power supply 1230 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 1210 through a power management system, thereby enabling the management of charging, discharging, and power consumption.
[0229] The display unit 1240 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 1200. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 1200, and the text, pictures, and other objects displayed on the display interfaces. The display unit 1240 may include a display panel 1241. The display panel 1241 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
[0230] The input unit 1250 can be used to receive information such as numbers or characters input by the user. The input unit 1250 may include a touch panel 1251 and other input devices 1252. The touch panel 1251, also known as a touch screen, can collect touch operations on or near the touch panel 1251 by the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1251).
[0231] Specifically, the touch panel 1251 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 1210, and receive and execute commands from the processor 1210. Furthermore, the touch panel 1251 can be implemented using various types of touch technologies, including resistive, capacitive, infrared, and surface acoustic wave. Other input devices 1252 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0232] Of course, the touch panel 1251 can cover the display panel 1241. When the touch panel 1251 detects a touch operation on or near it, it transmits the information to the processor 1210 to determine the type of touch event. Subsequently, the processor 1210 provides corresponding visual output on the display panel 1241 according to the type of touch event. Although in Figure 12 In this embodiment, the touch panel 1251 and the display panel 1241 are two separate components to realize the input and output functions of the electronic device 1200. However, in some embodiments, the touch panel 1251 and the display panel 1241 can be integrated to realize the input and output functions of the electronic device 1200.
[0233] The electronic device 1200 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 1200 may also include other components such as a camera. Since these components are not the focus of this disclosure, therefore... Figure 12It is not shown in the text and will not be described in detail here.
[0234] Those skilled in the art will understand that Figure 12 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.
[0235] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this disclosure, the functions of each module (or unit) can be implemented in one or more software or hardware components.
Claims
1. An image processing method, characterized in that, The method includes: After receiving a camera switching command for switching to the target camera component, images are acquired through the reference camera component and the target camera component in the electronic device, respectively, to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component; Determine the first white balance decision point corresponding to the reference image; the first white balance decision point is a reference color value used for white balance setting of the reference image. Based on the reference image and the first white balance decision point, a second white balance decision point corresponding to the image to be processed is determined; the second white balance decision point is a reference color value used for white balance setting of the image to be processed. Based on the second white balance decision point, calculate the white balance gain coefficient corresponding to the image to be processed; the white balance gain coefficient is negatively correlated with the second white balance decision point. The white balance of the image to be processed is adjusted according to the white balance gain coefficient to obtain a white balance image.
2. The method according to claim 1, characterized in that, The step of determining the second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point includes: Based on the color values of each pixel block in the reference image and the first white balance decision point, a first white region in the reference image is determined. Determine the first white region and the second white region that matches it in the image to be processed; The synchronization conversion coefficient is determined based on the color values of the first white area and the second white area; The second white balance decision point is determined based on the first white balance decision point and the synchronization conversion coefficient.
3. The method according to claim 2, characterized in that, Determining the first white region in the reference image based on the color values of each pixel block in the reference image and the first white balance decision point includes: Based on the color value of each pixel block in the reference image, the pixel blocks are clustered to obtain the color category corresponding to each pixel block. Based on the color value corresponding to each color category and the similarity between it and the first white balance decision point, the target color category is selected from each color category; The first white region is obtained based on the pixel block corresponding to the target color category in the reference image.
4. The method according to claim 3, characterized in that, Before clustering the pixel blocks according to their respective color values in the reference image, the method further includes: For each target image in the reference image and the image to be processed, the following steps are performed respectively: If the target image meets the cropping conditions, then obtain the calibration coordinates corresponding to the target image; The target image is cropped based on the calibrated coordinates.
5. The method according to claim 4, characterized in that, Before obtaining the calibration coordinates corresponding to the target image, the method further includes: The field of view of each of the multiple camera components of the electronic device is obtained; the multiple camera components include the reference camera component and the target camera component; Determine the smallest field of view among all field of view angles; Based on the minimum field of view, the calibration coordinates corresponding to each camera component are determined.
6. The method according to any one of claims 3-5, characterized in that, Obtaining the first white region based on the pixel block corresponding to the target color category in the reference image includes: At least one image region is formed based on the pixel blocks in the reference image corresponding to the target color category; The largest region in each image region is determined as the first white region.
7. The method according to claim 6, characterized in that, Before determining the largest region in each image region as the first white region, the method further includes: If there are multiple image regions and there are adjacent image regions that meet the conditions for connected regions, then the image regions that meet the conditions for connected regions will be merged into the same image region.
8. The method according to any one of claims 1-5, characterized in that, The step of adjusting the white balance of the image to be processed according to the white balance gain coefficient to obtain a white balance image includes: The white balance image is obtained by multiplying the color value of each pixel in the image to be processed by the white balance gain coefficient.
9. An image processing apparatus, characterized in that, The device includes: The acquisition unit is used to acquire images through a reference camera component and the target camera component in the electronic device respectively after receiving a camera switching instruction for switching to the target camera component, to obtain a reference image acquired by the reference camera component and an image to be processed acquired by the target camera component; A determining unit is used to determine a first white balance decision point corresponding to the reference image; the first white balance decision point is a reference color value used for white balance setting of the reference image. The conversion unit is configured to determine a second white balance decision point corresponding to the image to be processed based on the reference image and the first white balance decision point; the second white balance decision point is a reference color value used for white balance setting of the image to be processed. The calculation unit is used to calculate the white balance gain coefficient corresponding to the image to be processed based on the second white balance decision point; the white balance gain coefficient is negatively correlated with the second white balance decision point; The adjustment unit is used to adjust the white balance of the image to be processed according to the white balance gain coefficient to obtain a white balance image.
10. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 8.
12. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device is the method according to any one of claims 1 to 8.