Exposure adjustment method and device, computer readable storage medium, terminal

Abstract

An exposure adjustment method and device, a computer readable storage medium and a terminal, the method comprising: extracting a subject saliency map in an original image; determining a target detection region based on the subject saliency map; determining the luminance values of each sub-region of the original image, the subject saliency map, the target detection region and a background region respectively; determining the weight values of each grid region in the original image, the subject saliency map, the target detection region and the background region respectively; performing weighted operation on the luminance difference values of the original image, the subject saliency map, each sub-region of the target detection region and the background region and the weight values of the corresponding grid regions to obtain a to-be-adjusted luminance difference value; and adjusting the exposure of a shooting device of the original image using the to-be-adjusted luminance difference value. The present application can improve the matching degree of the to-be-adjusted luminance difference value and the actual subject condition of the image, and improve the exposure adjustment accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an exposure adjustment method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] With the development of smartphones and camera devices, camera functions are becoming increasingly important in users' phones. Automatic exposure, which ensures appropriate brightness in captured images, is fundamental to high-quality, high-resolution imaging. Current mobile devices, such as smartphones, use image brightness statistics and facial information to achieve suitable brightness. However, due to the overly comprehensive or singular nature of statistical information, the accuracy of automatic exposure is low.

[0003] Specifically, current automatic exposure in mobile photography is mainly based on global statistical information, dividing the entire image into blocks and calculating brightness or average values ​​for each channel. The level of detail in these blocks tends to favor the image center or certain specific areas, such as the face of a person in the center of the image, resulting in underexposure or overexposure of people or objects in other areas.

[0004] There is an urgent need for an exposure adjustment method that can take into account the brightness values ​​and weights of multiple subjects, perform comprehensive analysis on different subjects using appropriate brightness values ​​and weights to obtain the brightness difference to be adjusted, and then use the brightness difference to be adjusted to adjust the exposure of the original image capturing device, effectively improving the matching degree between the brightness difference to be adjusted and the actual situation of the image. Summary of the Invention

[0005] The technical problem solved by this invention is to provide an exposure adjustment method and apparatus, a computer-readable storage medium, and a terminal, which can improve the matching degree between the brightness difference to be adjusted and the actual subject situation of the image, and improve the accuracy of exposure adjustment.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide an exposure adjustment method, comprising: extracting a subject saliency map from an original image, wherein the original image is divided into multiple grid regions, and the subject saliency map contains the outline regions of one or more subjects of a preset category; determining a target detection region based on the subject saliency map, wherein the target detection region contains some or all of the subjects in the subject saliency map, and each sub-region of the target detection region contains at least a portion of a single subject; determining the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region, wherein the background region is the remaining region in the original image after removing the subject saliency map; and determining the brightness values ​​of the target detection region and the target detection region. The original image, the main saliency map, the target detection region, and the background region are each weighted. A weighted operation is performed on the brightness difference between the original image, the main saliency map, the target detection region, and the background region, along with the weight value of the corresponding grid region, to obtain a brightness difference to be adjusted. The exposure of the original image capturing device is adjusted using this brightness difference. The brightness difference between the original image, the main saliency map, the target detection region, and the background region is obtained by calculating the difference between a target brightness value and the brightness values ​​of the original image, the main saliency map, the target detection region, and the background region.

[0007] Optionally, determining the target detection region based on the subject saliency map includes: performing image recognition on each subject in the subject saliency map and retaining valid subjects; selecting at least a portion of each retained subject; and determining sub-regions of the target detection region based on the selected portion of each subject, wherein each sub-region has a graphic frame of a preset shape and includes the selected portion of the subject.

[0008] Optionally, determining the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region respectively includes: determining the R, Gr, Gb, and B pixel values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region respectively; and using the determined R, Gr, Gb, and B pixel values ​​to determine the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region respectively.

[0009] Optionally, the main saliency map is obtained after downsampling, and / or each sub-region of the target detection region is obtained after downsampling.

[0010] Optionally, determining the weight value of each grid region in the original image, the subject saliency map, the target detection region, and the background region includes: determining the overall weights of the original image, the subject saliency map, the target detection region, and the background region; and determining the weight of each grid region based on its position in the original image and / or the size of the area occupied by the subject to which each grid region belongs, using the overall weights as the weight value of that grid region.

[0011] Optionally, the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the original image, and the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the background region.

[0012] Optionally, the self-weight of the grid region satisfies one or more of the following: the smaller the distance between the grid region and the center point of the original image, the greater the self-weight of the grid region; the larger the area occupied by the subject to which the grid region belongs, the greater the self-weight of the grid region.

[0013] Optionally, the sum of the overall weights of the original image, the subject saliency map, the target detection region, and the background region is 1; and / or, the average weight of the grid region of each region is 1.

[0014] Optionally, a weighted average is performed on the brightness differences of the original image, the main saliency map, each sub-region of the target detection region, and the background region, along with the weight values ​​of their respective grid regions, to obtain the brightness difference to be adjusted. This includes: weighting the brightness differences of the original image with the weights of each grid region to obtain intermediate parameters for the original image; weighting the brightness differences of the main saliency map with the weights of its respective grid regions to obtain intermediate parameters for the main saliency map; weighting the brightness differences of each sub-region of the target detection region with the weights of its respective grid regions to obtain intermediate parameters for the target detection region; weighting the brightness differences of the background region with the weights of its respective grid regions to obtain intermediate parameters for the background region; and weighting the overall weight of the original image as the weight value of the intermediate parameters for the original image, the overall weight of the main saliency map as the weight value of the intermediate parameters for the main saliency map, the overall weight of the target detection region as the weight value of the intermediate parameters for the target detection region, and the overall weight of the background region as the weight value of the intermediate parameters for the background region, and then performing a weighted average to obtain the brightness difference to be adjusted.

[0015] Optionally, adjusting the exposure of the original image capturing device using the brightness difference to be adjusted includes: determining the brightness value to be adjusted of the original image based on the sum of the brightness difference to be adjusted and the brightness value of the original image; determining the exposure value to be adjusted according to a preset mapping relationship between exposure value and brightness value; and adjusting the exposure of the original image capturing device using the exposure value to be adjusted.

[0016] To address the aforementioned technical problems, embodiments of the present invention provide an exposure adjustment device, comprising: an extraction module for extracting a subject saliency map from an original image, wherein the original image is divided into multiple grid regions, and the subject saliency map includes the outline regions of one or more subjects of a preset category; a target detection region determination module for determining a target detection region based on the subject saliency map, wherein the target detection region includes some or all of the subjects in the subject saliency map, and each sub-region of the target detection region includes at least a portion of a single subject; a brightness value determination module for determining the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and a background region, wherein the background region is the remaining region in the original image after removing the subject saliency map; and a weight value. The system includes a determination module for determining the weight value of each grid region in the original image, the main saliency map, the target detection region, and the background region; a weighted calculation module for performing a weighted calculation on the brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region, and the weight value of the corresponding grid region, to obtain a brightness difference to be adjusted; and an adjustment module for adjusting the exposure of the original image capturing device using the brightness difference to be adjusted. The brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region is obtained by calculating the difference between a target brightness value and the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region.

[0017] To address the aforementioned technical problems, embodiments of the present invention provide a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, causes the aforementioned exposure adjustment method to be performed.

[0018] To solve the above-mentioned technical problems, embodiments of the present invention provide a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the exposure adjustment method described above when running the computer program.

[0019] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0020] In this embodiment of the invention, by extracting a saliency map containing one or more subjects of a preset category, the included subjects can be more comprehensive, reducing the chance of ignoring subjects when multiple subjects are included. Furthermore, by determining the target detection region, the brightness value and weight value of each subject can be analyzed separately, reducing the possibility of using the same brightness value for different subjects. The brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region are determined separately, and the weight value of each grid region among the above four is determined. A weighted calculation is then performed to obtain the brightness difference to be adjusted. Compared to existing technologies, this is closer to a global adjustment and does not consider the differences between subjects under multiple subjects. The solution of this embodiment of the invention can take into account the brightness values ​​and weights of multiple subjects separately, and use appropriate brightness values ​​and weights to comprehensively analyze different subjects to obtain the brightness difference to be adjusted. Then, the exposure of the original image capturing device is adjusted using the brightness difference to be adjusted, effectively improving the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0021] Furthermore, image recognition is performed on each subject in the subject saliency map, and valid subjects are retained. At least a portion of each retained subject is selected. Based on the selected portion of each subject, a sub-region of the target detection region is determined. Each sub-region has a graphic frame of a preset shape and contains the selected portion of the subject. Thus, a graphic frame of a preset shape can be determined as the target detection region based on the subject saliency map. Since the subject saliency map is the outline region of the subject, and the target detection region is a graphic frame of a preset shape, the edge contour of the target detection region is more standard than the edge contour of the subject saliency map. This is beneficial to improving the accuracy of weight value judgment in the process of determining weight value based on grid region. Correspondingly, the proportion of subjects in the subject saliency map is higher than that in the target detection region. This is beneficial to improving the accuracy of brightness value judgment for a single subject in the process of determining brightness value. Therefore, combining the subject saliency map and the target detection region to determine brightness value and weight value helps to improve the accuracy of the brightness difference to be adjusted obtained by comprehensive analysis, and further improves the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0022] Furthermore, the main saliency map is obtained after downsampling, and / or each sub-region of the target detection region is obtained after downsampling, thereby reducing the complexity of processing the main saliency map and each sub-region of the target detection region through downsampling.

[0023] Furthermore, the overall weights of the original image, the main saliency map, the target detection region, and the background region are determined respectively. Based on the position of each grid region in the original image and / or the size of the area occupied by the main body to which each grid region belongs, the weight of each grid region is determined according to the overall weights. In this way, the determination of the weight values ​​can take into account both the overall situation of the above four factors and the influence of the position of each grid region and / or the main body to which it belongs. This helps to improve the accuracy of the brightness difference to be adjusted obtained by comprehensive analysis and further improve the matching degree between the brightness difference to be adjusted and the actual main body situation of the image.

[0024] Furthermore, the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the original image, and the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the background region. This can increase the importance of the main saliency map and the target detection region, and further improve the matching degree between the comprehensively determined brightness difference to be adjusted and the main saliency map and the target detection region.

[0025] Furthermore, the self-weight of the grid region satisfies one or more of the following: the smaller the distance between the grid region and the center point of the original image, the greater the self-weight of the grid region; the larger the area occupied by the subject to which the grid region belongs, the greater the self-weight of the grid region. This allows the self-weight of the grid region to be affected by its position and / or the area occupied by the subject to which it belongs, thereby further improving the accuracy of determining the weight value.

[0026] Furthermore, the sum of the overall weights of the original image, the main saliency map, the target detection region, and the background region is 1; and / or, the average weight of each region's grid region is 1. By adjusting the sum of the overall weights to 1 and the average weight of each region to 1, it is beneficial to normalize the weight values ​​and reduce the computational complexity.

[0027] Furthermore, by determining the intermediate parameters of the original image, the intermediate parameters of the subject saliency map, the intermediate parameters of the target detection region, and the intermediate parameters of the background region, and then using the overall weight of the original image as the weight value of the intermediate parameters of the original image, the overall weight of the subject saliency map as the weight value of the intermediate parameters of the subject saliency map, the overall weight of the target detection region as the weight value of the intermediate parameters of the target detection region, and the overall weight of the background region as the weight value of the intermediate parameters of the background region, a weighted average is performed to obtain the brightness difference to be adjusted. Each grid region of the above four parameters can be determined separately, and a weighted operation can be performed to obtain the brightness difference to be adjusted. Compared with the prior art, this is closer to a global adjustment and does not consider the differences between various subjects under multiple subjects. The solution of this embodiment can take into account the brightness values ​​and weights of multiple subjects respectively, and use appropriate brightness values ​​and weights for different subjects to perform comprehensive analysis to obtain the brightness difference to be adjusted, effectively improving the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0028] Furthermore, based on the sum of the brightness difference to be adjusted and the brightness value of the original image, the brightness value to be adjusted of the original image is determined; according to the preset mapping relationship between the exposure value and the brightness value, the exposure value to be adjusted is determined; using the exposure value to be adjusted, the exposure of the original image capturing device is adjusted. Since different environments (such as sunny days, cloudy days, nights, etc.) have their own brightness target values, their respective brightness differences can be obtained, and they have their own mapping relationships. Therefore, the exposure can be adjusted by combining the mapping relationships of different brightness values ​​(such as differences caused by environmental factors), thereby further improving the accuracy of exposure adjustment based on a more accurate brightness difference to be adjusted. Attached Figure Description

[0029] Figure 1 This is a flowchart of an exposure adjustment method according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram illustrating an application scenario of an exposure adjustment method according to an embodiment of the present invention;

[0031] Figure 3 yes Figure 1 A flowchart of a specific implementation of step S12;

[0032] Figure 4 yes Figure 1 A flowchart of a specific implementation of step S15;

[0033] Figure 5 yes Figure 1 A flowchart of a specific implementation of step S16;

[0034] Figure 6This is a schematic diagram of the structure of an exposure adjustment device in an embodiment of the present invention. Detailed Implementation

[0035] As mentioned earlier, the accuracy of automatic exposure in existing technologies is low. Specifically, current automatic exposure in mobile photography is mainly based on global statistical information, dividing the image into blocks and calculating brightness or average values ​​of each channel. The level of detail in these blocks tends to favor the image center or certain specific areas, such as the face of a person in the center of the image, resulting in underexposure or overexposure of people or objects in other areas.

[0036] In one existing improvement scheme, the focus of automatic exposure has gradually shifted to ensuring appropriate exposure for both the subject and the background. For example, post-processing methods are used to further optimize the exposure brightness distribution, such as gamma correction or tone mapping. However, the final result is not entirely satisfactory.

[0037] Research has revealed that, in existing technologies, the statistically determined brightness values ​​do not correspond to the actual subject. Existing correction methods are closer to global adjustments and do not consider the differences between various subjects. For example, when there are both white faces and black animals in an image, there is a tendency for white faces to be overexposed or black animals to be underexposed.

[0038] In this embodiment of the invention, by extracting a saliency map containing one or more subjects of a preset category, the included subjects can be more comprehensive, reducing the chance of ignoring subjects when multiple subjects are included. Furthermore, by determining the target detection region, the brightness value and weight value of each subject can be analyzed separately, reducing the possibility of using the same brightness value for different subjects. The brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region are determined separately, and the weight value of each grid region among the above four is determined. A weighted calculation is then performed to obtain the brightness difference to be adjusted. Compared to existing technologies, this is closer to a global adjustment and does not consider the differences between subjects under multiple subjects. The solution of this embodiment of the invention can take into account the brightness values ​​and weights of multiple subjects separately, and use appropriate brightness values ​​and weights to comprehensively analyze different subjects to obtain the brightness difference to be adjusted. Then, the exposure of the original image capturing device is adjusted using the brightness difference to be adjusted, effectively improving the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0039] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0040] Reference Figure 1 , Figure 1This is a flowchart of an exposure adjustment method according to an embodiment of the present invention. The exposure adjustment method may include steps S11 to S16:

[0041] Step S11: Extract the subject saliency map from the original image, wherein the original image is divided into multiple grid regions, and the subject saliency map contains the outline regions of one or more subjects of a preset category;

[0042] Step S12: Based on the subject saliency map, determine the target detection region, wherein the target detection region contains part or all of the subjects in the subject saliency map, and each sub-region of the target detection region contains at least a part of a single subject;

[0043] Step S13: Determine the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region, respectively, wherein the background region is the remaining region in the original image after removing the main saliency map;

[0044] Step S14: Determine the weight value of each grid region in the original image, the main saliency map, the target detection region, and the background region, respectively;

[0045] Step S15: The brightness difference of the original image, the main saliency map, each sub-region of the target detection region, and the background region is weighted and calculated with the weight value of the corresponding grid region to obtain the brightness difference to be adjusted.

[0046] Step S16: Use the brightness difference to be adjusted to adjust the exposure of the original image capturing device.

[0047] The brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region is obtained by calculating the difference between the brightness target value and the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region.

[0048] In the specific implementation of step S11, the original image can be a raw image obtained by a photographic device, or it can be obtained by cropping the raw image.

[0049] Furthermore, the original image can be a preset size, such as a rectangular image of size W×H obtained after cropping.

[0050] Where W represents the width and H represents the height.

[0051] It should be noted that the original image can also have a preset number of bits per channel, for example, the number of bits per channel can be n bits.

[0052] The subject saliency map can be the outline region of one or more subjects containing a preset category, used to represent the image based on the degree of attention to the image scene, and is also known as the "Region of Interest" (ROI).

[0053] Specifically, images based on the attention level of an image scene can be obtained using manual or automatic methods, where the pixel value of the image represents the attention level of that pixel.

[0054] The methods for obtaining the main saliency map include, but are not limited to: manual annotation, image tool annotation, calculation based on hardware system characteristics, and calculation using image processing, machine learning, or deep learning algorithms.

[0055] Reference Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of an exposure adjustment method according to an embodiment of the present invention.

[0056] like Figure 2 The original image shown can be divided into multiple grid regions, and the extracted subject saliency map can contain the outline regions of subjects in two preset categories: children and animals.

[0057] The main subject in the original image can be a person, animal, plant, object, etc.

[0058] Continue to refer to Figure 1 In the specific implementation of step S12, the target detection area can be determined based on the main saliency map.

[0059] The target detection region includes some or all of the subjects in the subject saliency map, and each sub-region of the target detection region includes at least a portion of a single subject.

[0060] like Figure 2 As shown, the target detection region includes two subjects: children and animals. The sub-region of the target detection region containing children only includes the head area of ​​the children, while the sub-region of the target detection region containing animals includes the entire animal.

[0061] Reference Figure 3 , Figure 3 yes Figure 1 A flowchart illustrating a specific implementation of step S12 is provided. The step of determining the target detection region based on the main saliency map may include steps S31 to S33, which are described below.

[0062] In step S31, image recognition is performed on each subject in the subject saliency map, and valid subjects are retained.

[0063] Specifically, object detection processing methods can be used to perform image recognition on each subject in the subject saliency map in order to determine the category of objects in the image, such as people, animals, plants, etc.

[0064] Then you can keep the valid subjects, such as those that appear too far to the edge or those that are not fully captured.

[0065] The methods for object detection processing include, but are not limited to: manual annotation, image tool annotation, saliency maps calculated using image processing, machine learning, or deep learning algorithms based on hardware system characteristics.

[0066] In step S32, at least a portion of each subject to be retained is selected.

[0067] In practice, the subject can be selected in part or in whole, depending on the need to adjust the exposure.

[0068] like Figure 2 The children shown need more exposure adjustment for their faces, which can be done by selecting the facial area; the animals need overall exposure adjustment, which can be done by selecting the entire animal.

[0069] In step S33, based on the selected portion of each subject, sub-regions of the target detection area are determined respectively. Each sub-region has a graphic frame of a preset shape and includes the selected portion of the subject.

[0070] The preset shape can be selected from: rectangle, square, trapezoid, polygon, circle, ellipse, etc.

[0071] Furthermore, each sub-region can be represented by a rectangular graphic frame to reduce computational complexity.

[0072] like Figure 2 The target detection area shown includes a rectangular area for the child's face and a rectangular area for the animal's whole body.

[0073] In this embodiment of the invention, image recognition is performed on each subject in the subject saliency map, and valid subjects are retained; at least a portion of each retained subject is selected; based on the selected portion of each subject, a sub-region of the target detection region is determined, each sub-region having a graphic frame of a preset shape and containing the selected portion of the subject. Thus, a graphic frame of a preset shape can be determined as the target detection region based on the subject saliency map. Since the subject saliency map is the outline region of the subject, and the target detection region is a graphic frame of a preset shape, the edge contour of the target detection region is more standard than the edge contour of the subject saliency map, which is beneficial to improving the accuracy of weight value judgment in the process of determining weight value based on grid region. Correspondingly, the proportion of subjects in the subject saliency map is higher than that in the target detection region, which is beneficial to improving the accuracy of brightness value judgment of a single subject in the process of determining brightness value. Therefore, combining the subject saliency map and the target detection region to determine brightness value and weight value helps to improve the accuracy of the brightness difference to be adjusted obtained by comprehensive analysis, and further improves the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0074] Continue to refer to Figure 1 In the specific implementation of step S13, the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region are determined respectively.

[0075] The background region is the remaining region in the original image after removing the main saliency map.

[0076] Furthermore, the brightness value can be determined based on the pixel value.

[0077] Specifically, the step of determining the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region may include: determining the R, Gr, Gb, and B pixel values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region; and using the determined R, Gr, Gb, and B pixel values ​​to determine the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region.

[0078] Where R represents the pixel value of a red pixel, Gr and Gb represent the pixel values ​​of green pixels, and B represents the pixel value of a blue pixel.

[0079] In one specific embodiment of the present invention, the brightness value of the corresponding area can be determined using the following formula:

[0080] Luminance = 0.299R + 0.587(Gr + Gb) / 2 + 0.114B;

[0081] R, Gr, Gb, and B are used to represent the pixel values ​​of the corresponding colors in the four regions mentioned above. For example, they can be the pixel values ​​in each Bayer array, or they can be the average of the pixel values ​​in multiple Bayer arrays.

[0082] In another specific embodiment of the present invention, the brightness value of the corresponding area can be determined using the following formula:

[0083] Brightness = 0.2126R + 0.7152(Gr + Gb) / 2 + 0.0722B;

[0084] R, Gr, Gb, and B are used to represent the pixel values ​​of the corresponding colors in the four regions mentioned above. For example, they can be the pixel values ​​in each Bayer array, or they can be the average of the pixel values ​​in multiple Bayer arrays.

[0085] In another specific embodiment of the present invention, the brightness value of the corresponding area can be determined using the following formula:

[0086] Brightness = (Gr + Gb) / 2

[0087] Gr and Gb are used to represent the green pixel values ​​of the corresponding regions in the four regions mentioned above. For example, they can be the green pixel values ​​in each Bayer array, or they can be the average of the green pixel values ​​in multiple Bayer arrays.

[0088] Understandably, for the four regions mentioned above, after determining the pixel values ​​in each Bayer array, the brightness value of the entire region can be determined by averaging.

[0089] It should be noted that other appropriate methods can also be used to determine the brightness value, such as converting the color image to a grayscale image and then calculating the average grayscale value of the image as the brightness value of the image, etc. This application does not limit this.

[0090] Furthermore, the main saliency map is obtained after downsampling processing, and / or each sub-region of the target detection region is obtained after downsampling processing.

[0091] In this embodiment of the invention, the main saliency map is obtained after downsampling, and / or each sub-region of the target detection region is obtained after downsampling, thereby reducing the complexity of processing the main saliency map and each sub-region of the target detection region through downsampling.

[0092] In the specific implementation of step S14, the weight value of each grid region in the original image, the main saliency map, the target detection region, and the background region is determined respectively.

[0093] Specifically, the original image is divided into multiple grid regions, and the main saliency map, the target detection region, and the background region are all contained in some of these grid regions.

[0094] In one specific embodiment of the present invention, the grid regions of the main saliency map, the target detection region, and the background region can be repeated. For example, a certain grid region may contain both the effective subject and the background. In this case, the weight value of the grid region can be calculated in the process of calculating the weight value of each grid region of the main saliency map, calculating the weight value of each grid region of the target detection region, and calculating the weight value of each grid region of the background region.

[0095] Further, the step of determining the weight value of each grid region in the original image, the subject saliency map, the target detection region, and the background region may include: determining the overall weights of the original image, the subject saliency map, the target detection region, and the background region respectively; and determining the weight of each grid region based on its position in the original image and / or the size of the area occupied by the subject to which each grid region belongs, using the overall weights as the weight value of that grid region.

[0096] In this embodiment of the invention, the overall weights of the original image, the main saliency map, the target detection region, and the background region are determined respectively. Based on the position of each grid region in the original image and / or the size of the area occupied by the main body to which each grid region belongs, the weight of each grid region is determined according to the overall weights. Thus, in the process of determining the weight values, both the overall situation of the above four factors and the influence of the position of each grid region and / or the main body to which it belongs are taken into account. This helps to improve the accuracy of the brightness difference to be adjusted obtained by comprehensive analysis and further improve the matching degree between the brightness difference to be adjusted and the actual main body situation of the image.

[0097] Furthermore, the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the original image, and the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the background region.

[0098] The overall weight of the main saliency map and the overall weight of the target detection region may be the same or different, and the overall weight of the original image and the overall weight of the background region may be the same or different.

[0099] In a non-limiting embodiment, the overall weights of the original image, the subject saliency map, the target detection region, and the background region can be 10%, 40%, 40%, and 10%, respectively.

[0100] The overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the original image, and the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the background region. This can increase the importance of the main saliency map and the target detection region, and further improve the matching degree between the comprehensively determined brightness difference to be adjusted and the main saliency map and the target detection region.

[0101] Furthermore, the self-weight of the grid region can satisfy one or more of the following: the smaller the distance between the grid region and the center point of the original image, the greater the self-weight of the grid region; the larger the area occupied by the subject to which the grid region belongs, the greater the self-weight of the grid region.

[0102] In one specific embodiment of the present invention, multiple levels can be preset. The smaller the distance between the grid region and the center point of the original image, the greater the weight of the corresponding level's grid region. Alternatively, the larger the area occupied by the subject, the greater the weight of the corresponding level's grid region.

[0103] In one specific embodiment of the present invention, three levels can be preset, namely 1.2, 1.0 and 0.8. Then, as the distance between the grid region and the center point of the original image increases, the weight of the grid region itself is 1.2, 1.0 and 0.8 respectively. Alternatively, as the area occupied by the subject to which the grid region belongs decreases, the weight of the grid region itself is 1.2, 1.0 and 0.8 respectively.

[0104] In another specific embodiment of the present invention, the weight of the center position can be set to a preset value, and then the weight of the grid region is proportionally reduced as the distance between it and the center point of the original image increases; or the weight of the grid region whose subject occupies a preset size can be set to a preset value, and then the weight of the grid region is proportionally reduced as the area occupied by the subject to which the grid region belongs decreases.

[0105] In this embodiment of the invention, the self-weight of the grid region satisfies one or more of the following: the smaller the distance between the grid region and the center point of the original image, the greater the self-weight of the grid region; the larger the area occupied by the subject to which the grid region belongs, the greater the self-weight of the grid region. This allows the self-weight of the grid region to be affected by its position and / or the area occupied by the subject to which it belongs, thereby further improving the accuracy of determining the weight value.

[0106] Furthermore, the sum of the overall weights of the original image, the main saliency map, the target detection region, and the background region is 1; and / or, the average weight of the grid region of each region is 1.

[0107] As shown above, the overall weights of the original image, the main saliency map, the target detection region, and the background region can be 10%, 40%, 40%, and 10%, respectively, and the individual weights of the grid regions can be 1.2, 1.0, and 0.8, respectively.

[0108] In this embodiment of the invention, the sum of the overall weights of the original image, the main saliency map, the target detection region, and the background region is 1; and / or, the average weight of each region's grid region is 1. By adjusting the sum of the overall weights to 1 and the average weight of each region to 1, it is beneficial to normalize the weight values ​​and reduce the computational complexity.

[0109] In the specific implementation of step S15, the brightness difference of the original image, the main saliency map, each sub-region of the target detection region, and the background region can be weighted and averaged with the weight value of the corresponding grid region to obtain the brightness difference to be adjusted.

[0110] Furthermore, the weighting method can be selected from: average weighting, Gaussian weighting, and center weighting.

[0111] In practice, a weighted average method can be used to reduce computational complexity.

[0112] It should be noted that the median value can also be obtained by weighting the brightness difference of the original image, the main saliency map, each sub-region of the target detection region, and the background region with the weight value of the corresponding grid region.

[0113] The brightness difference is obtained based on the target brightness value.

[0114] It should be noted that different environments (such as sunny days, cloudy days, nighttime, etc.) have their own target brightness values ​​and their own mapping relationships. In one specific embodiment of the present invention, the calculation of the target brightness value can be combined with information such as the ambient brightness calculated by the automatic exposure algorithm, the distribution of the brightness statistical histogram information in the region, and the dynamic range of the image, and combined with the adjustment parameters to comprehensively obtain the target brightness value of each region.

[0115] It is understandable that the target brightness value can also be determined based on empirical data, such as a preset target brightness value determined based on historical data of the same environment.

[0116] Reference Figure 4, Figure 4 yes Figure 1 A flowchart of a specific implementation of step S15. The step of performing a weighted calculation on the brightness difference of the original image, the main saliency map, each sub-region of the target detection region, and the background region, along with the weight value of the corresponding grid region, to obtain the brightness difference to be adjusted may include steps S41 to S45, and each step is described below.

[0117] In step S41, the brightness difference of the original image is weighted and averaged with the weight of each grid region to obtain the intermediate parameters of the original image.

[0118] Specifically, the brightness difference of the original image is obtained by calculating the difference between the brightness target value of the original image and the brightness value of the original image.

[0119] In step S42, the brightness difference of the main saliency map is weighted and averaged with the weight of the corresponding grid region to obtain the intermediate parameters of the main saliency map.

[0120] Specifically, the brightness difference of the main saliency map is obtained by calculating the difference between the brightness target value of the main saliency map and the brightness value of the main saliency map.

[0121] In step S43, the brightness difference of each sub-region of the target detection area is weighted and averaged with the weight of the corresponding grid region to obtain the intermediate parameters of the target detection area.

[0122] Specifically, the brightness difference between each sub-region of the target detection area is obtained by calculating the difference between the target brightness value of the sub-region and the brightness value of each sub-region of the target detection area.

[0123] In step S44, the brightness difference of the background area is weighted and averaged with the weight of the corresponding grid area to obtain the intermediate parameters of the background area.

[0124] Specifically, the brightness difference of the background area is obtained by calculating the difference between the target brightness value of the background area and the brightness value of the background area.

[0125] In step S45, the weights of the original image as the weights of the intermediate parameters of the original image, the main saliency map as the weights of the intermediate parameters of the main saliency map, the target detection region as the weights of the intermediate parameters of the target detection region, and the background region as the weights of the intermediate parameters of the background region are used to perform a weighted average to obtain the brightness difference to be adjusted.

[0126] In this embodiment of the invention, by determining intermediate parameters of the original image, intermediate parameters of the subject saliency map, intermediate parameters of the target detection region, and intermediate parameters of the background region, and then using the overall weight of the original image as the weight value of the intermediate parameters of the original image, the overall weight of the subject saliency map as the weight value of the intermediate parameters of the subject saliency map, the overall weight of the target detection region as the weight value of the intermediate parameters of the target detection region, and the overall weight of the background region as the weight value of the intermediate parameters of the background region, a weighted average is performed to obtain the brightness difference to be adjusted. Each grid region of the above four parameters can be determined separately, and a weighted operation can be performed to obtain the brightness difference to be adjusted. Compared with the prior art, this is closer to a global adjustment and does not consider the differences between various subjects under multiple subjects. By adopting the solution of this embodiment of the invention, the brightness values ​​and weights of multiple subjects can be considered separately, and appropriate brightness values ​​and weights can be used for comprehensive analysis of different subjects to obtain the brightness difference to be adjusted, effectively improving the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0127] Continue to refer to Figure 1 In the specific implementation of step S16, the exposure of the original image capturing device can be adjusted using the brightness difference to be adjusted.

[0128] Reference Figure 5 , Figure 5 yes Figure 1 A flowchart of a specific implementation of step S16. The step of adjusting the exposure of the capturing device of the original image using the brightness difference to be adjusted may include steps S51 to S53, which are described below.

[0129] In step S51, the brightness value to be adjusted of the original image is determined based on the sum of the brightness difference to be adjusted and the brightness value of the original image.

[0130] In step S52, the exposure value to be adjusted is determined according to the preset mapping relationship between the exposure value and the brightness value.

[0131] For more information on target brightness values, please refer to the previous text; it will not be repeated here.

[0132] More specifically, calibration can be performed first under a known specific environment to obtain reference information: that is, under a specific brightness environment, how much exposure is required for this model of module (lens characteristics, such as aperture value, OB value, lens transmittance, etc.) to achieve a suitable image brightness. Then, under any environment, by referring to the image brightness, exposure, and sensitivity at the time of calibration, as well as the current exposure, the current ambient brightness can be calculated.

[0133] The brightness statistical histogram within the specified area can be used in the automatic exposure calculation process. After obtaining the image, the number of values ​​for each pixel is counted to obtain the histogram information. By analyzing the distribution of the histogram, the numerical value of the image's dynamic range can be further characterized.

[0134] The debugging parameters can be features of the automatic exposure algorithm. Industry-standard exposure algorithms offer debuggable parameters, allowing technicians to adjust values ​​to achieve suitable exposure in any scenario.

[0135] In step S53, the exposure of the original image capturing device is adjusted using the exposure value to be adjusted.

[0136] In this embodiment of the invention, the brightness value to be adjusted of the original image is determined based on the sum of the brightness difference to be adjusted and the brightness value of the original image; the exposure value to be adjusted is determined according to the preset mapping relationship between the exposure value and the brightness value; the exposure of the original image capturing device is adjusted using the exposure value to be adjusted. Since different environments (such as sunny days, cloudy days, nights, etc.) have their own brightness target values, their own brightness differences can be obtained, and they have their own mapping relationships. Therefore, the exposure can be adjusted by combining the mapping relationships of different brightness values ​​(such as differences caused by environmental factors), thereby further improving the accuracy of exposure adjustment based on a more accurate brightness difference to be adjusted.

[0137] In this embodiment of the invention, by extracting a saliency map containing one or more subjects of a preset category, the included subjects can be more comprehensive, reducing the chance of ignoring subjects when multiple subjects are included. Furthermore, by determining the target detection region, the brightness value and weight value of each subject can be analyzed separately, reducing the possibility of using the same brightness value for different subjects. The brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region are determined separately, and the weight value of each grid region among the above four is determined. A weighted calculation is then performed to obtain the brightness difference to be adjusted. Compared to existing technologies, this is closer to a global adjustment and does not consider the differences between subjects under multiple subjects. The solution of this embodiment of the invention can take into account the brightness values ​​and weights of multiple subjects separately, and use appropriate brightness values ​​and weights to comprehensively analyze different subjects to obtain the brightness difference to be adjusted. Then, the exposure of the original image capturing device is adjusted using the brightness difference to be adjusted, effectively improving the matching degree between the brightness difference to be adjusted and the actual subject situation of the image.

[0138] Reference Figure 6 , Figure 6 This is a schematic diagram of an exposure adjustment device according to an embodiment of the present invention. The exposure adjustment device may include:

[0139] Extraction module 61 is used to extract a subject saliency map from an original image, wherein the original image is divided into multiple grid regions, and the subject saliency map contains the outline regions of one or more subjects of a preset category;

[0140] The target detection region determination module 62 is used to determine a target detection region based on the subject saliency map, wherein the target detection region contains part or all of the subjects in the subject saliency map, and each sub-region of the target detection region contains at least a part of a single subject;

[0141] Brightness value determination module 63 is used to determine the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region and the background region respectively, wherein the background region is the remaining region in the original image after removing the main saliency map;

[0142] The weight value determination module 64 is used to determine the weight value of each grid region in the original image, the main saliency map, the target detection region, and the background region, respectively.

[0143] The weighted operation module 65 is used to perform a weighted operation on the brightness difference of the original image, the main saliency map, each sub-region of the target detection region and the background region and the weight value of the corresponding grid region to obtain the brightness difference to be adjusted.

[0144] The adjustment module 66 is used to adjust the exposure of the original image capturing device using the brightness difference to be adjusted.

[0145] The brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region is obtained by calculating the difference between the brightness target value and the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region.

[0146] For more details regarding the working principle, working method, and beneficial effects of the exposure adjustment device in this application embodiment, please refer to the above description of the exposure adjustment method, which will not be repeated here.

[0147] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is run by a processor, the above-described exposure adjustment method is executed. The storage medium may include ROM, RAM, magnetic disk, or optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0148] This application also provides a terminal, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the communication method described above. The terminal includes, but is not limited to, terminal devices such as mobile phones, computers, tablets, servers, and cloud platforms.

[0149] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0150] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0151] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0152] It should be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0153] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0154] In the embodiments of this application, "multiple" refers to two or more.

[0155] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0156] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An exposure adjustment method, characterized in that, include: Extract a subject saliency map from the original image, which is divided into multiple grid regions, and the subject saliency map contains the outline regions of one or more subjects of a preset category; Based on the subject saliency map, a target detection region is determined, wherein the target detection region contains part or all of the subjects in the subject saliency map, and each sub-region of the target detection region contains at least a part of a single subject; The brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region are determined respectively, wherein the background region is the remaining region in the original image after removing the main saliency map; Determining the weight value of each grid region in the original image, the subject saliency map, the target detection region, and the background region includes: determining the overall weights of the original image, the subject saliency map, the target detection region, and the background region; and determining the weight of each grid region based on its position in the original image and / or the size of the area occupied by the subject to which each grid region belongs, using the overall weights as the weight value of that grid region. The method for obtaining the brightness difference to be adjusted is as follows: A weighted average is performed on the brightness differences of the original image, the main saliency map, each sub-region of the target detection region, and the background region, along with the weight values ​​of their respective grid regions. This includes: averaging the brightness differences of the original image with the weights of each grid region to obtain intermediate parameters for the original image; averaging the brightness differences of the main saliency map with the weights of its respective grid regions to obtain intermediate parameters for the main saliency map; averaging the brightness differences of each sub-region of the target detection region with the weights of its respective grid regions to obtain intermediate parameters for the target detection region; averaging the brightness differences of the background region with the weights of its respective grid regions to obtain intermediate parameters for the background region; and finally, averaging the overall weight of the original image, the overall weight of the main saliency map, the overall weight of the target detection region, and the overall weight of the background region as the weight values ​​for the intermediate parameters of the background region to obtain the brightness difference to be adjusted. The exposure of the original image capturing device is adjusted using the brightness difference to be adjusted. The brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region is obtained by calculating the difference between the brightness target value and the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region.

2. The exposure adjustment method according to claim 1, characterized in that, The step of determining the target detection region based on the main saliency map includes: Image recognition is performed on each subject in the subject saliency map, and valid subjects are retained; Select at least a portion of each subject to be retained; Based on the selected portion of each subject, sub-regions of the target detection area are determined respectively. Each sub-region has a graphic frame of a preset shape and contains the selected portion of the subject.

3. The exposure adjustment method according to claim 1, characterized in that, Determining the brightness values ​​of the original image, the subject saliency map, each sub-region of the target detection region, and the background region, respectively, includes: The R, Gr, Gb, and B pixel values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region are determined respectively. The brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region are determined using the R, Gr, Gb, and B pixel values ​​respectively.

4. The exposure adjustment method according to claim 1 or 3, characterized in that, The main saliency map is obtained after downsampling, and / or each sub-region of the target detection region is obtained after downsampling.

5. The exposure adjustment method according to claim 1, characterized in that, The overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the original image, and the overall weight of the main saliency map and the overall weight of the target detection region are both greater than the overall weight of the background region.

6. The exposure adjustment method according to claim 1, characterized in that, The weight of the grid region satisfies one or more of the following: The smaller the distance between the grid region and the center point of the original image, the greater the weight of the grid region. The larger the area occupied by the main body to which the grid region belongs, the greater the weight of the grid region itself.

7. The exposure adjustment method according to claim 1, 5, or 6, characterized in that, The sum of the overall weights of the original image, the main saliency map, the target detection region, and the background region is 1; And / or, The average weight of each grid region is 1.

8. The exposure adjustment method according to claim 1, characterized in that, Adjusting the exposure of the capturing device for the original image using the brightness difference to be adjusted includes: The brightness value to be adjusted of the original image is determined based on the sum of the brightness difference to be adjusted and the brightness value of the original image. Based on the preset mapping relationship between exposure value and brightness value, determine the exposure value to be adjusted; The exposure of the device used to capture the original image is adjusted using the exposure value to be adjusted.

9. An exposure adjustment device, characterized in that, include: An extraction module is used to extract a subject saliency map from an original image, the original image being divided into multiple grid regions, and the subject saliency map containing the outline regions of one or more subjects of a preset category; The target detection region determination module is used to determine a target detection region based on the subject saliency map, wherein the target detection region contains part or all of the subjects in the subject saliency map, and each sub-region of the target detection region contains at least a part of a single subject; A brightness value determination module is used to determine the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region, respectively, wherein the background region is the remaining region in the original image after removing the main saliency map; The weight value determination module is used to determine the weight value of each grid region in the original image, the main saliency map, the target detection region, and the background region, respectively, including: determining the overall weight of the original image, the main saliency map, the target detection region, and the background region; and determining the weight of each grid region based on its position in the original image and / or the size of the area occupied by the main body to which each grid region belongs, based on the overall weight, as the weight value of the grid region. The weighted calculation module is used to perform a weighted calculation on the brightness difference of the original image, the main saliency map, each sub-region of the target detection region, and the background region, along with the weight value of the corresponding grid region, to obtain the brightness difference to be adjusted. This includes: performing a weighted average of the brightness difference of the original image and the weight of each grid region to obtain intermediate parameters of the original image; performing a weighted average of the brightness difference of the main saliency map and the weight of the corresponding grid region to obtain intermediate parameters of the main saliency map; performing a weighted average of the brightness difference of each sub-region of the target detection region and the weight of the corresponding grid region to obtain intermediate parameters of the target detection region; performing a weighted average of the brightness difference of the background region and the weight of the corresponding grid region to obtain intermediate parameters of the background region; and performing a weighted average of the overall weight of the original image as the weight value of the intermediate parameters of the original image, the overall weight of the main saliency map as the weight value of the intermediate parameters of the main saliency map, the overall weight of the target detection region as the weight value of the intermediate parameters of the target detection region, and the overall weight of the background region as the weight value of the intermediate parameters of the background region to obtain the brightness difference to be adjusted. An adjustment module is used to adjust the exposure of the capturing device of the original image using the brightness difference to be adjusted. The brightness difference between the original image, the main saliency map, each sub-region of the target detection region, and the background region is obtained by calculating the difference between the brightness target value and the brightness values ​​of the original image, the main saliency map, each sub-region of the target detection region, and the background region.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, the exposure adjustment method according to any one of claims 1 to 8 is executed.

11. A terminal comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the exposure adjustment method according to any one of claims 1 to 8.

Images