Image brightness evaluation method and apparatus, readable storage medium and electronic device

Abstract

This disclosure provides an image brightness evaluation method and apparatus, a readable storage medium, and an electronic device. [Solution] The image brightness evaluation method includes the steps of: acquiring a raw image to be evaluated for brightness; performing brightness weight estimation for each pixel in the raw image to obtain a brightness weight set corresponding to each pixel; for any pixel in the raw image, calculating a weighted sum for each color component included in the first target region to which the pixel belongs, based on the brightness weight set corresponding to the pixel, to obtain a brightness value corresponding to the pixel; and generating a brightness evaluation image corresponding to the raw image based on the brightness value corresponding to each pixel in the raw image. This disclosure adaptively adjusts the brightness weights of each color component of a pixel, enabling the obtained brightness evaluation image to accurately reflect the brightness distribution of a scene perceived by the human eye, and contributing to providing more accurate brightness evaluation data.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and in particular, to an image brightness evaluation method and apparatus, a readable storage medium, and an electronic device.

Background Art

[0002] Brightness evaluation is a process of quantitatively analyzing the light and dark levels of pixels in an image, and is a basic and important step in the field of RAW image (i.e., raw data directly collected by a camera sensor) processing. The currently commonly used brightness evaluation method generally performs a convolution operation on RAW format array data using a fixed filter kernel (such as a 3×3 average value filter or a weighted average filter mask) to estimate the brightness value of each pixel region.

[0003] The current brightness evaluation method is realized based on global statistics of the pixels of an image, and cannot adapt to local spectral changes caused by different illumination conditions and reflection characteristics of the object surface in the image. Such a brightness estimation that does not match the local content causes a problem that the brightness image obtained by conversion cannot accurately reflect the light and dark distribution of the actual scene, and the brightness image cannot meet the actual image processing requirements.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to solve the above technical problems, the present disclosure provides an image brightness evaluation method and apparatus, a readable storage medium, and an electronic device, and solves the problem that the deviation between the brightness image caused by calculating the brightness using a fixed weight for each color component in brightness evaluation and the actual scene is large.

Means for Solving the Problems

[0005] A first embodiment of the embodiments of the present disclosure provides an image brightness evaluation method comprising: acquiring a raw image to be brightness evaluated; performing brightness weight estimation for each pixel in the raw image to obtain a brightness weight set corresponding to each pixel, wherein each brightness weight in the brightness weight set corresponds to one color component; calculating a weighted sum for each color component included in a first target region to which the pixel belongs, based on the brightness weight set corresponding to the pixel, to obtain a brightness value corresponding to the pixel, wherein the first target region is an image region of a first preset size with the pixel as the reference point; and generating a brightness evaluation image corresponding to the raw image based on the brightness value corresponding to each pixel in the raw image.

[0006] An image luminance evaluation apparatus according to a second embodiment of the embodiments of the present disclosure includes: an acquisition module for acquiring a raw image to be evaluated for luminance; an estimation module for performing luminance weight estimation for each pixel in the raw image and obtaining a set of luminance weights corresponding to each pixel, wherein each luminance weight in the luminance weight set corresponds to one color component; a calculation module for calculating a weighted sum for each color component included in a first target region to which an arbitrary pixel in the raw image belongs, based on the luminance weight set corresponding to the pixel, and obtaining a luminance value corresponding to the pixel, wherein the first target region is an image region of a first preset size with the pixel as the reference point; and a generation module for generating a luminance evaluation image corresponding to the raw image based on the luminance value corresponding to each pixel in the raw image.

[0007] A computer-readable storage medium according to a third embodiment of the embodiments of this disclosure stores a computer program for implementing the image brightness evaluation method described above when executed.

[0008] An electronic device according to a fourth embodiment of the embodiments of the present disclosure includes a processor and a memory for storing instructions that the processor can execute, wherein the processor realizes the image brightness evaluation method described above by reading and executing executable instructions from the memory.

[0009] A fifth embodiment of the embodiments of this disclosure provides a computer program product which includes computer program instructions, and when the computer program instructions are executed by an instruction processor, the image brightness evaluation method described above is performed. [Effects of the Invention]

[0010] Based on the image brightness evaluation method and apparatus, readable storage medium, and electronic device according to the embodiments of this disclosure, brightness weight estimation is performed for each pixel in the raw image to obtain a brightness weight set corresponding to each pixel. Next, a weighted sum is calculated for each color component included in the first target region to which each pixel belongs, based on the brightness weight set corresponding to each pixel, to obtain a brightness value corresponding to that pixel. Finally, a brightness evaluation image is generated based on the brightness value corresponding to each pixel in the raw image. This allows the brightness weights of each color component of a pixel to be adaptively adjusted according to the local brightness characteristics at the position of each pixel in the image. The brightness evaluation image obtained using the brightness weights can more accurately reflect the brightness distribution of the scene as perceived by the human eye. The local brightness in the image reflected by the brightness evaluation image better matches the visual perception characteristics of the human eye, contributing to the provision of more accurate brightness evaluation data and meeting the need for high-precision brightness evaluation images in various image processing scenes. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic flowchart of an image brightness evaluation method according to one exemplary embodiment of the present disclosure. [Figure 2] This is a schematic flowchart of an image brightness evaluation method according to another exemplary embodiment of the present disclosure. [Figure 3] This is a schematic flowchart of an image brightness evaluation method according to yet another exemplary embodiment of the present disclosure. [Figure 4] This is a schematic flowchart of an image brightness evaluation method according to yet another exemplary embodiment of the present disclosure. [Figure 5]This is a schematic flowchart of an image brightness evaluation method according to yet another exemplary embodiment of the present disclosure. [Figure 6] This is a schematic flowchart of an image brightness evaluation method according to yet another exemplary embodiment of the present disclosure. [Figure 7] This is a schematic diagram of the structure of a luminance estimation model according to one exemplary embodiment of the present disclosure. [Figure 8] This is a schematic diagram of the learning flow of a luminance estimation model according to yet another exemplary embodiment of the present disclosure. [Figure 9] This is a schematic flowchart of an image brightness evaluation method according to yet another exemplary embodiment of the present disclosure. [Figure 10] This is a structural diagram of an image brightness evaluation device according to one exemplary embodiment of the present disclosure. [Figure 11] This is a structural diagram of an image brightness evaluation device according to another exemplary embodiment of the present disclosure. [Figure 12] This is a structural diagram of an electronic device relating to one exemplary embodiment of the present disclosure. [Modes for carrying out the invention]

[0012] To illustrate this disclosure, exemplary embodiments of this disclosure will be described in detail below with reference to the drawings. Clearly, the embodiments described are only a selection of, and not all, embodiments of this disclosure. It should be understood that this disclosure is not limited by the exemplary embodiments.

[0013] Unless otherwise specifically stated, the relative arrangements of the parts and steps, formulas, and numerical values ​​described in these embodiments do not limit the scope of this disclosure.

[0014] Application Summary In the field of RAW image processing, luminance evaluation is a fundamental and important step, and the accuracy of its results directly affects the effectiveness of a series of image processing tasks such as tone mapping, automatic exposure, and white balance. Related luminance estimation methods usually perform convolution operations on RAW format Bayer array data using a fixed filter kernel (such as a 3×3 mean filter or a weighted mean filter mask) to estimate the luminance value of each pixel region.

[0015] The basic flow of this method is to pre-set a fixed filter mask (for example, a mean mask with all weight coefficients equal or a Gaussian blur mask with high central weight), and calculate with the corresponding pixels through a sliding window method in the RAW image. For each pixel position, the pixel values within its neighborhood (for example, a 3×3 window) are weighted and summed according to the filter mask, and the obtained result is used as the luminance estimation value of the region. In this process, different color channels of red, green, and blue are usually assigned fixed weight ratios (for example, synthesized according to the empirical value regarding the green sensitivity of the human eye, adopting the ratio of R:G:B = 0.299:0.587:0.114) to convert the original Bayer data into a single-channel luminance image.

[0016] The drawback of this method lies in the pattern of "uniform processing", which does not take into account the differences between different regions of the image content itself. For example, the human visual system (HVS) has a non-linear luminance perception for different luminance environments and different color regions, and adaptively changes according to local content. The fixed R:G:B weight ratio is an average value derived from global statistics or specific assumptions, and cannot adapt to local spectral changes caused by different illumination conditions and reflection characteristics of object surfaces in the image. For example, in a scene dominated by a red object (e.g., a red earth field at sunset), the luminance evaluation algorithm defaults that the green component is dominant and underestimates the contribution of the red channel to the overall luminance. The estimated luminance value of the region is lower than the true luminance visually perceived. Conversely, in a scene dominated by a large area of blue or green (e.g., a forest or an ocean), the fixed weight may overestimate or underestimate the luminance contribution of the corresponding color. Such luminance estimation that does not match the local content causes the luminance image obtained by the conversion to be unable to accurately reflect the light and dark distribution of the scene perceived by the human eye, resulting in the use of deviated luminance information in subsequent processing and the problem of not meeting the requirements of image processing.

[0017] To solve the above problems, the image luminance evaluation method according to an embodiment of the present disclosure can adaptively adjust the luminance weight according to the local characteristics of the image, improve the accuracy of luminance estimation, and further provide accurate luminance data for other image processing tasks (such as luminance mapping, tone mapping, etc.).

[0018] Exemplary method FIG. 1 is a schematic flowchart of an image luminance evaluation method according to an exemplary embodiment of the present disclosure. This embodiment is applicable to an electronic device. As shown in FIG. 1, it includes the following steps 101 to step 104.

[0019] In step 101, an original image of the luminance evaluation target is acquired.

[0020] Here, the raw image is an image composed of raw data collected by a camera sensor, i.e., a RAW image. In one embodiment, the RAW image may be an image collected by a Bayer color filter array (CFA), and the colors of each pixel in the Bayer color filter array are arranged in the proportions of 50% green, 25% red, and 25% blue.

[0021] In step 102, brightness weight estimation is performed for each pixel in the raw image, and a set of brightness weights corresponding to each pixel is obtained.

[0022] Here, each luminance weight in the luminance weight set corresponds to one color component. For example, for a given pixel, the luminance weight set corresponding to that pixel contains three luminance weights, each corresponding to the R, G, and B color components.

[0023] In this embodiment, luminance weight estimation can be performed for each pixel in the raw image using various methods. For example, a fixed-size sliding window can be set, and for each pixel in the raw image, the ratio of the sum of the values ​​of each color component in the sliding window corresponding to that pixel to the sum of the values ​​of all color components in the sliding window can be calculated, and the luminance weight of each color component corresponding to that pixel can be determined based on the ratio corresponding to each color component.

[0024] Alternatively, for example, a pre-trained convolutional neural network model can be used to extract luminance features from each pixel in a raw image, and then the obtained luminance features can be upsampled to obtain luminance weights corresponding to each color component for each pixel.

[0025] In step 103, for any pixel in the raw image, a weighted sum is calculated for each color component included in the first target region to which the pixel belongs, based on the luminance weight set corresponding to that pixel, and the luminance value corresponding to that pixel is obtained.

[0026] Here, the first target region is an image region of a first preset size with the pixel in question as the reference point. The first preset size can be set arbitrarily, for example, 3x3. The position of the reference point can be set in advance, for example, to the center point within the 3x3 image region.

[0027] The method for determining the values ​​of the color components to be used in the weighted sum calculation can be predetermined. For example, the average value of each color component within the first target region can be used as the value of the color component to be used in the weighted sum calculation.

[0028] For example, if the luminance weight set of a pixel in a raw image is expressed as R:G:B=2:7:1, the luminance value corresponding to that pixel is expressed as lum=(2×ave_r+7×ave_g+1×ave_b) / 10. Here, ave_r, ave_g, and ave_b are the average values ​​of the three color components R, G, and B within the first target region, respectively.

[0029] In step 104, a brightness evaluation image corresponding to the raw image is generated based on the brightness value corresponding to each pixel in the raw image.

[0030] Specifically, the obtained luminance values ​​can be combined to create a luminance evaluation image, and a luminance evaluation image can also be obtained after performing processes such as downsampling or convolution based on the luminance image obtained by combining the values.

[0031] The image luminance evaluation method according to the embodiment of this disclosure obtains a set of luminance weights corresponding to each pixel by estimating luminance weights for each pixel in the raw image, then calculates a weighted sum for each color component included in the first target region to which each pixel belongs based on the set of luminance weights corresponding to each pixel to obtain a luminance value corresponding to that pixel, and finally generates a luminance evaluation image based on the luminance value corresponding to each pixel in the raw image. This allows the luminance weights of each color component of a pixel to be adaptively adjusted according to the local luminance characteristics at the position of each pixel in the image, and the luminance evaluation image obtained using the luminance weights can more accurately reflect the brightness distribution of the scene as perceived by the human eye. This contributes to providing more accurate luminance evaluation data, as the local luminance in the image reflected in the luminance evaluation image better matches the visual perception characteristics of the human eye, thus meeting the need for high-precision luminance evaluation images in various image processing scenes.

[0032] In several selectable implementations, as shown in Figure 2, step 102 includes the following steps 1021 to 1022.

[0033] In step 1021, local color feature extraction is performed for each pixel based on the color component values ​​of each pixel in the raw image, and a set of local color features is obtained.

[0034] Here, each local color feature in the local color feature set corresponds to a single pixel in the raw image, and the local color feature represents the magnitude of the values ​​of each color component within the local region to which the corresponding pixel belongs. For example, for a given pixel in the raw image, the values ​​of each color component within the local region to which that pixel belongs are statistically calculated, and the sum and ratio of the values ​​of each color component and the total values ​​of all color components within the local region are used as the local color feature. That is, the local color feature corresponding to that pixel includes three ratios corresponding to the R, G, and B color components, respectively.

[0035] For example, a raw image can be computed using a pre-trained convolutional neural network model, and the resulting feature map can be used as a local color feature set. That is, each pixel in the feature map corresponds to a set of local color features, and each set of local color features includes feature values ​​that represent the magnitudes of the R, G, and B values.

[0036] In step 1022, a set of luminance weights corresponding to each pixel in the raw image is determined based on the local color feature set.

[0037] Each set of local color features in the local color feature set represents the magnitude of different color components, so the luminance weight of each color component can be determined according to the values ​​corresponding to different color components. For example, the values ​​corresponding to each color component can be normalized to obtain the luminance weight of each color component.

[0038] This embodiment contributes to accurate brightness evaluation according to differences in local regions within a raw image by performing local color feature extraction for each pixel in the raw image and determining a set of brightness weights that can reflect the color features of different regions in the raw image based on the obtained set of local color features.

[0039] In several selectable implementations, as shown in Figure 3, step 1021 described above includes the following steps 10211 to 10212.

[0040] In step 10211, for any pixel in the raw image, the energy ratio of the driving energy of each color component to the total driving energy of all color components is determined as the local color feature corresponding to that pixel, within the range of the second target region to which the pixel belongs.

[0041] Here, the aforementioned driving energy represents the energy consumed to make a pixel of the corresponding color emit light, and can usually be represented by a grayscale value in the range of 0 to 255. The second target area is an image area of ​​a second preset size with the pixel in question as the reference point. The second preset size may be the same as or different from the first preset size described above, and the method for selecting the position of the reference point may be the same as or different from the method for selecting the position of the reference point in the first target area described above.

[0042] As an example, the second preset size is set to 3x3, the second target region is set as a window in a sliding window format, the stride is set to 1, and the local color features of each pixel in the raw image are calculated sequentially. Taking an arbitrary window as an example, the center point of the window is the reference point, and the energy ratios corresponding to the red, green, and blue components of the window are str_r=sum_r / (sum_r+sum_g+sum_b), str_g=sum_g / (sum_r+sum_g+sum_b), and str_b=sum_b / (sum_r+sum_g+sum_b). Here, sum_r, sum_g, and sum_b are the driving energies of the red, green, and blue components in the window, i.e., the sum of the numerical values ​​of the pixels marked as red, green, and blue, respectively. The calculated str_r, str_g, and str_b are the local color features corresponding to the reference point pixel. The sum of the three energy ratios is 1, and can represent the distribution of each color within the window.

[0043] In step 10212, the set of local color features obtained is determined to be the local color feature set.

[0044] In other words, a set of local color features is created by combining the local color features corresponding to each pixel in the raw image.

[0045] In this embodiment, the local color features obtained by statistically analyzing the driving energy of each color component within the second target region to which each pixel in the raw image belongs, using a ratio method, can accurately represent the energy distribution of each color component. Depending on the energy distribution, this contributes to obtaining a more accurate luminance weight set.

[0046] In several selectable implementations, as shown in Figure 4, step 1022 described above includes the following steps 10221 to 10222.

[0047] In step 10221, for any pixel in the raw image, the dominant color component is determined from each color component in the second target region to which the pixel belongs.

[0048] Here, the dominant color component is the color component with the highest energy ratio in the second target region.

[0049] In step 10222, the luminance weights corresponding to each color component in the pixel are determined based on the energy ratio of the dominant color component.

[0050] As a selectable method, a corresponding dominant color filter mask can be set for each dominant color component, and this dominant color filter mask includes a luminance weight corresponding to each color component. For example, a red filter mask is represented as r:g:b=6:3:1, indicating that the luminance weights corresponding to the three color components R, G, and B are 0.6, 0.3, and 0.1, respectively; a green filter mask is represented as r:g:b=2:7:1; and a blue filter mask is represented as r:g:b=1:3:6.

[0051] This embodiment determines the dominant color component in the second target region and sets the luminance weight corresponding to the dominant color component. This allows for more accurate setting of the luminance weight of each color component based on the color characteristics of different regions in the raw image, enabling the luminance evaluation image to more accurately reflect the color differences between different regions in the raw image.

[0052] In several selectable implementations, as shown in Figure 5, step 10222 includes steps 102221 to 102224.

[0053] In step 102221, if the dominant color component is a pre-specified target color component, it is determined whether the energy ratio of the dominant color component is equal to or greater than the first ratio threshold.

[0054] Here, the target color component and the first ratio threshold can be set in advance. For example, the target color components are the red (R) component and the blue (B) component, and the first ratio threshold is 0.7. That is, if the dominant color component is the R component or the B component, it is further determined whether the energy ratio str_r corresponding to the R component or the energy ratio str_b corresponding to the B component is 0.7 or greater.

[0055] In step 102222, if the energy ratio of the dominant color component is greater than or equal to the first ratio threshold, the dominant color filter mask set in advance for the dominant color component is obtained, and the luminance weights corresponding to each color component corresponding to the pixel are determined based on the dominant color filter mask.

[0056] Continuing with the above example, if the dominant color component is red and str_r ≥ 0.7, the dominant color filter mask becomes an R-dominant filter mask. This mask assigns a high weight to the red component and reduces the weights of the green and blue components. For example, the weight ratio of the R, G, and B color components of an R-dominant filter mask is 6:3:1, and the corresponding luminance weights are 0.6, 0.3, and 0.1, respectively. This mask is designed for scenes where red is dominant.

[0057] If the dominant color component is blue and str_b ≥ 0.7, the dominant color filter mask becomes a B-dominant filter mask. This mask assigns a high weight to the blue component and reduces the weights of the green and red components. For example, the weight ratio of the R, G, and B color components of a B-dominant filter mask is 1:3:6, and the corresponding luminance weights are 0.1, 0.3, and 0.6, respectively. This mask is designed for scenes where blue is dominant.

[0058] In step 102223, if the energy ratio of the dominant color component is less than the first ratio threshold and greater than or equal to the second ratio threshold, a weight adjustment coefficient corresponding to the energy ratio of the dominant color component is determined based on the first and second ratio thresholds.

[0059] The second ratio threshold can be set in advance; for example, the second ratio threshold is 0.5.

[0060] Continuing with the above example, if the dominant color component is red or blue, and 0.5 ≤ str_r < 0.7 or 0.5 ≤ str_b < 0.7, then the weight adjustment coefficients corresponding to the R component or B component can be further determined.

[0061] Based on a pre-set calculation formula, the energy ratio of the current dominant color component, the first ratio threshold, and the second ratio threshold can be calculated to obtain the weight adjustment coefficient. For example, if the dominant color component is the R component, the formula for calculating the corresponding weight adjustment coefficient α is shown in equation (1) below.

number

[0062] When the dominant color component is component B, the formula for calculating the corresponding weight adjustment coefficient β is shown in equation (2) below.

number

[0063] In step 102224, the luminance weights included in the dominant color filter mask are adjusted using a weight adjustment coefficient, and the luminance weights corresponding to each color component corresponding to the pixel are determined based on the adjusted color filter mask.

[0064] After obtaining the weight adjustment coefficients, each luminance weight included in the dominant color filter mask can be adjusted using these coefficients. For example, the adjusted color filter mask can be obtained by multiplying each luminance weight included in the dominant color filter mask by the weight adjustment coefficient and then adding these multipliers to the corresponding luminance weights of the color components in a pre-set reference mask. According to the weight ratio of each color component in the adjusted color filter mask, the luminance weights corresponding to each color component corresponding to a single pixel can be determined.

[0065] This embodiment achieves improved regional adaptability of luminance evaluation by adaptively evaluating luminance for local regions of a dominant color by determining luminance weights using a corresponding dominant color filter mask when the intensity of a particular dominant color component is too high. When the intensity of the dominant color component is moderate (0.5 ≤ str_r < 0.7 or 0.5 ≤ str_b < 0.7), it indicates that the pixel currently undergoing luminance estimation is located in a transition region between dominant color regions (e.g., between a red-dominant region and a blue-dominant region). Within this region, each luminance weight in the final color filter mask is determined after being adjusted based on the dominant color filter mask, and the weight adjustment coefficient is determined by the intensity of the dominant color component. This avoids abrupt fluctuations in luminance weights at region boundaries and improves the accuracy of luminance estimation.

[0066] In several possible implementations, step 102224 described above can be performed as follows:

[0067] First, obtain the pre-configured balanced color filter mask.

[0068] Here, a balanced color filter mask represents a relatively uniform energy distribution for each color component. Typically, in Bayer-format RAW images, the number of green pixels is inherently large, so a green-dominant filter mask is used as the balanced color filter mask. For example, the weight ratio of the R, G, and B color components of a balanced color filter mask is 2:7:1. This mask aims to process regions where the color distribution is relatively balanced, with a moderate weight ratio for the red and blue channels while maintaining a relatively high weight for the green component.

[0069] Next, using the weight adjustment coefficient, a weighted sum is calculated for the luminance weights included in the dominant color filter mask and the balanced color filter mask, respectively, to obtain the adjusted color filter mask.

[0070] Specifically, when the dominant color component is the R component, the weighted sum formula is shown in equation (3) below.

number

[0071] Here, w_red represents the R-dominant filter mask, w_balance represents the balanced color filter mask, and α represents the weight adjustment coefficient. The formula represents calculating a weighted sum for the two luminance weights corresponding to the same color in the two masks. For example, the luminance weight of the R component in w_red is multiplied by α, the luminance weight of the R component in w_balance is multiplied by (1-α), and then the two products are added to obtain the luminance weight of the R component in the adjusted color filter mask.

[0072] Similarly, when the dominant color component is component B, β represents the corresponding weight adjustment coefficient, and the formula for the weighted sum is shown in equation (4) below.

number

[0073] This embodiment uses a balanced color filter mask and weight adjustment coefficients to perform smooth interpolation on the luminance weights of the color balance region, enabling smooth transitions in luminance values ​​between dominant color regions (for example, between a red-dominant region and a blue-dominant region), thereby avoiding abrupt changes in luminance and improving the accuracy of luminance evaluation.

[0074] In several selectable implementations, as shown in Figure 5, step 10222 includes steps 102225 to 102226.

[0075] In step 102225, if the dominant color component is not the pre-specified target color component, a pre-set equilibrium color filter mask is obtained.

[0076] For example, if the target colors are red and blue and the dominant color is green, a filter mask with the green component as the dominant color component (G-dominant filter mask) can be obtained as the balanced color filter mask. For an explanation of the balanced color filter mask, please refer to the examples.

[0077] In step 102226, the luminance weights corresponding to each color component in the pixel are determined based on the balanced color filter mask.

[0078] The luminance weights corresponding to each color component in a single pixel can be determined according to the weight ratio of each color component in the balanced color filter mask. For example, if the weight ratio of R, G, and B in the balanced color filter mask is 2:7:1, the corresponding luminance weights are 0.2, 0.7, and 0.1, respectively.

[0079] This embodiment improves the efficiency and accuracy of luminance evaluation by directly determining the luminance weights using a balanced color filter mask when the dominant color component is not the target color component, thereby achieving a balanced luminance evaluation based on the color characteristics of a specific dominant color region without requiring further judgment on the value of the dominant color component.

[0080] In several selectable implementations, as shown in Figure 6, step 102 includes steps 1023 to 1026.

[0081] In step 1023, downsampling encoding is performed on the raw image using the encoder included in the pre-trained brightness estimation model to obtain an initial feature map.

[0082] Here, the luminance estimation model may be a model constructed based on neural networks of various structures. As shown in Figure 7, the luminance estimation model 700 may include an encoder 701, a feature enhancement network 702, a decoder 703, and a weight calculation unit 704.

[0083] Encoders can be constructed using convolutional neural networks (containing a series of convolutional layers). The encoder performs convolution layer by layer on the input raw image, gradually decreasing the spatial resolution of the image while simultaneously increasing the number of channels, thereby obtaining a multi-scale, hierarchical initial feature map.

[0084] In step 1024, the feature enhancement network included in the luminance estimation model is used to enhance the luminance features of the initial feature map to obtain the enhanced feature map.

[0085] Feature enhancement networks are designed to enhance features on an initial feature map and can be built upon related neural network models. For example, they can be constructed by stacking multiple deformable transformer blocks. These stacked deformable transformer blocks can implement a deformable self-attention mechanism, which adaptively concentrates computational resources on areas more important for brightness evaluation (object contours, areas with complex textures, large monochromatic areas, etc.) rather than mechanically processing the entire image, significantly improving the model's ability to adapt to complex scenes and irregular local content.

[0086] In step 1025, upsampling decoding is performed on the enhanced feature map using the decoder included in the luminance estimation model to obtain the reconstructed feature map.

[0087] Here, the resolution of the reconstructed feature map is the same as the resolution of the raw image, and each pixel in the reconstructed feature map corresponds to a preset number of color feature values. The preset number here may be the same as the number of color components. For example, each pixel in the reconstructed feature map contains color feature values ​​corresponding to the three color components: R, G, and B. The magnitude of the color feature values ​​can reflect the magnitude of the driving energy required for each color component.

[0088] The decoder described above can be implemented based on the decoding of the relevant feature map. For example, upsampling can be performed using a deconvolutional decoder to gradually recover the spatial resolution of the image and obtain a reconstructed feature map with the same resolution as the raw image. As an optional approach, residual connections (skip connections) can be introduced between the corresponding layers of the decoder and encoder to prevent the loss of detailed information during the downsampling and upsampling processes. These connections directly transmit the fine details captured at the encoding stage to the decoding stage, ensuring that the final output reconstructed feature map retains the detailed features of the raw image.

[0089] In step 1026, the weight calculation unit included in the luminance estimation model is used to perform normalization on each color feature value of each pixel included in the reconstructed feature map, thereby obtaining a set of luminance weights corresponding to each pixel.

[0090] Specifically, the reconstructed feature map input to the weight calculation unit is a three-dimensional feature map, its size is the same as the input raw image, and it has three color components: R, G, and B. The values ​​of the three channels of the reconstructed feature map are arbitrary real numbers, have no clear physical meaning, and cannot be used directly in calculations. Therefore, a weight calculation unit is provided to perform normalization on each color component of the reconstructed feature map.

[0091] For example, normalization can be performed using the Softmax function. The Softmax function treats the three channel values ​​of each pixel as a set of unnormalized scores and converts them into a probability distribution whose sum is 1. After processing with the Softmax function, at each pixel location, the output values ​​of the three channels do not represent the original intensity, but rather the weight or ratio of the contribution of the R, G, and B channels to the final brightness of that point (location). Because it is a probability distribution, this ratio brightness weight is adaptive, meaning that the magnitude of the weight is dynamically determined by the network based on the image content (contours, texture, etc.). For example, in green vegetation areas, the G channel may be assigned a higher weight, while in red object areas, the R channel may have a higher weight.

[0092] The three color feature values ​​of a pixel in the reconstructed feature map are each z r =3, z g =1, z b Assuming that = 0.2, the process of applying the Softmax function to calculate the brightness weights corresponding to R, G, and B of the pixel in question is as follows:

[0093] First, perform the exponential calculation as shown in equations (5) to (7) below.

number

number

number

[0094] Next, calculate the sum of exponents as shown in equation (8) below.

number

[0095] Finally, normalization is performed as shown in equations (9) to (11) below to obtain the luminance weights.

number

number

number

[0096] For the pixel in question, the ratio of each color component adaptively assigned by the weight calculation unit is R:G:B ≈ 0.836:0.113:0.051. This ratio satisfies the condition 0.836 + 0.113 + 0.051 = 1, where each number in the ratio is a luminance weight corresponding to R, G, and B respectively, and the set of these luminance weights constitutes the luminance weight set corresponding to the pixel in question.

[0097] This embodiment achieves adaptive luminance weight sets for pixels in different regions of an image, based on the characteristic features of the color distribution in each region of the raw image, by constructing a luminance estimation model. The luminance estimation model can directly learn the complex mapping relationships necessary for luminance evaluation from the raw data. The luminance evaluation image generated based on the luminance weight set of the model output can more accurately reflect the details of different regions in the raw image, further improving the accuracy of luminance evaluation and scene generalization ability.

[0098] In several selectable implementation methods, as shown in Figure 8, the luminance estimation model is pre-trained by the following steps 801 to 808.

[0099] In step 801, a sample image is obtained.

[0100] Here, the sample image is the same type of image as the raw image in the example described above. For example, it is a RAW image taken with a camera.

[0101] In step 802, the sample image is converted into a sample luminance image.

[0102] Here, a sample luminance image can be obtained by employing a luminance image conversion method in which the ratios of R, G, and B are constant. This sample luminance image is used as the basis for luminance estimation.

[0103] In step 803, the sample brightness image is segmented, and a training task label value is set for each pixel in the segmented region based on the current training task type.

[0104] Here, region segmentation can be performed manually or automatically based on the relevant image segmentation method, thereby marking large monochromatic regions, regions with clear contour features, etc., within the sample luminance image.

[0105] Pixels in different regions of the segmented sample luminance image have corresponding training task label values. These training task label values, or annotated reference values, are used to compare the error with the actual predicted values ​​after the subsequent prediction task is performed. The setting of training task label values ​​is usually done manually. For example, the luminance values ​​of a region are adjusted empirically, and the training task label values, i.e., the luminance label values ​​of each pixel in that region, reflect the expected model output luminance values.

[0106] In step 804, the initial brightness estimation model is used to estimate the brightness weight for each pixel in the sample image, and a set of predicted brightness weights corresponding to each pixel is obtained.

[0107] Here, the specific execution process for performing brightness weight estimation for each pixel in the sample image can be found by referring to steps 1023 to 1026 described above.

[0108] In step 805, the predicted luminance value corresponding to each pixel in the sample image is determined based on the predicted luminance weight set.

[0109] Here, the execution process for determining the predicted brightness value can be found by referring to step 103 described above.

[0110] In step 806, the prediction error corresponding to the type of training task is determined based on the predicted brightness value and the training task label value.

[0111] Specifically, by setting a relevant loss function (e.g., an L1 loss function), a loss value representing the error between the predicted brightness value and the trained task label value can be calculated and obtained.

[0112] In step 807, the parameters of the initial luminance estimation model are adjusted based on the error.

[0113] Specifically, by employing machine learning methods and adjusting the parameters of the initial brightness estimation model using backpropagation and gradient descent, the aforementioned errors can be gradually reduced.

[0114] In step 808, if the initial luminance estimation model after parameter adjustment satisfies the pre-set learning termination conditions, the current initial luminance estimation model is determined to be the luminance estimation model after training.

[0115] The pre-set learning termination conditions may include, but are not limited to, at least one of the following: convergence of the loss value, reaching a pre-set learning time, or reaching a pre-set number of learning iterations.

[0116] This embodiment achieves efficient acquisition of training sample data by pre-dividing sample luminance images and setting training task label values, thereby enabling the training of a more accurate luminance estimation model for actual application scenarios and contributing to improved efficiency and accuracy of luminance estimation.

[0117] In several selectable implementations, step 803 may include setting a luminance label value for each pixel contained in each of the divided regions if the learning task type is a luminance value prediction task, or setting a perception label value for each pixel contained in each of the divided regions if the learning task type is a visual perception task.

[0118] When the learning task is a brightness value prediction task, setting brightness label values ​​for pixels in each divided region is usually done manually. This involves adjusting the brightness values ​​of specific regions in the sample brightness image based on experience, and matching the brightness values ​​of these regions to subjective human perception. Ultimately, a highly accurate brightness image is obtained as the true value label for supervised learning.

[0119] When the learning task is a visual perception task, and a perception label value is set for each pixel contained in each region after division, specifically, when a luminance estimation model is applied to a particular high-level visual task (e.g., target detection, image classification), the luminance image generated based on the luminance weight set of the model output during training can be directly output to the downstream perception network, and the loss can be calculated using the perception label values ​​(e.g., detection box, category label), allowing the luminance estimation model and the downstream perception network to learn together. This method guides the luminance estimation model to generate the luminance weight set most favorable to the visual perception task, and further generates the luminance image most favorable to the visual perception task.

[0120] Step 806 may include, if the learning task type is a brightness value prediction task, determining the error between the predicted brightness value and the brightness label value using a pre-configured first loss function; or, if the learning task type is a visual perception task, performing image perception prediction based on the predicted brightness value using a pre-configured perception network to obtain a perception prediction result, and determining the error between the perception prediction result and the perception label value using a pre-configured second loss function.

[0121] When the learning task is a brightness value prediction task, a pre-configured first loss function is used to determine the error between the predicted brightness value and the brightness label value. This first loss function can be a regression loss function, such as an L1 loss function. A brightness estimation model trained based on the error between the predicted brightness value and the brightness label value can generate highly accurate brightness images.

[0122] When the learning task is a visual perception task, image perception prediction is performed using a pre-configured perception network based on predicted luminance values ​​to obtain perception prediction results, and the error between the perception prediction results and the perception label values ​​is determined using a pre-configured second loss function. This second loss function can be a loss function used for classification, for example, a cross-entropy loss function. The perception prediction results mentioned above correspond to the perception label values, i.e., target detection boxes, target categories, etc., obtained by performing visual perception using the actually generated luminance image. A luminance estimation model trained based on the error between the perception prediction results and the perception label values ​​can generate luminance images that are favorable for the visual perception task.

[0123] This embodiment enables accurate learning of the luminance estimation model according to different application scenes by setting label values ​​for different learning tasks, thereby improving the accuracy of the model's luminance evaluation in specific scenes.

[0124] In several selectable implementations, as shown in Figure 9, step 103 includes steps 1031 to 1033.

[0125] In step 1031, for any pixel in the raw image, the driving energy of each color component and the number of pixels included in each color component are determined within the range of the first target region to which the pixel belongs.

[0126] Here, the aforementioned driving energy represents the energy consumed to make the corresponding color pixels emit light, and can usually be expressed as a grayscale value in the range of 0 to 255. For the R, G, and B color components in the current first target region, the number of pixels for each color component is represented as num_r, num_g, and num_b, respectively.

[0127] In step 1032, the pixel-average energy of each color component is determined based on the driving energy of each color component and the number of pixels included in each color component.

[0128] For example, for the R, G, and B color components within the current first target region, the pixel-average energy of the R component is expressed as ave_r = sum_r / num_r, the average value of the G component is expressed as ave_g = sum_g / num_g, and the average value of the B component is expressed as ave_b = sum_b / num_b. Here, sum_r, sum_g, and sum_b are the sums of the driving energies of the R, G, and B color components, respectively.

[0129] In step 1033, a weighted sum is calculated for each pixel-average energy obtained based on the brightness weight set corresponding to the pixel, and the brightness value corresponding to the pixel is obtained.

[0130] For example, if the ratio of the luminance weights corresponding to the pixel in question is 2:7:1, the luminance value corresponding to that pixel will be lum = (2 × ave_r + 7 × ave_g + 1 × ave_b) / 10.

[0131] This embodiment obtains a luminance value by calculating a weighted sum using a luminance weight set and the pixel-average energy of each color component. This allows the adjusted luminance value of each pixel to reflect the luminance characteristics of the surrounding area, and enables the luminance evaluation image to more accurately reflect the luminance differences of each area in the raw image.

[0132] Exemplary device Figure 10 is a schematic diagram of the structure of an image brightness evaluation device according to one exemplary embodiment of the present disclosure. This embodiment is applicable to electronic devices, and as shown in Figure 10, the image brightness evaluation device includes an acquisition module 1001 for acquiring a raw image to be brightness evaluated, an estimation module 1002 for performing brightness weight estimation for each pixel in the raw image and obtaining a brightness weight set corresponding to each pixel, wherein each brightness weight in the brightness weight set corresponds to one color component, an estimation module 1002 for calculating a weighted sum for each color component included in a first target region to which a pixel belongs, based on the brightness weight set corresponding to the pixel, and obtaining a brightness value corresponding to the pixel, wherein the first target region is an image region of a first preset size with the pixel as the reference point, and a generation module 1004 for generating a brightness evaluation image corresponding to the raw image based on the brightness value corresponding to each pixel in the raw image.

[0133] Referring to Figure 11, Figure 11 is a schematic diagram of the structure of an image brightness evaluation device according to another exemplary embodiment of the present disclosure.

[0134] In several selectable implementations, the estimation module 1002 includes an extraction unit 10021 for extracting local color features for each pixel based on the color component values ​​of each pixel in the raw image to obtain a set of local color features, and a first determination unit 10022 for determining a set of luminance weights corresponding to each pixel in the raw image based on the set of local color features.

[0135] In several selectable implementations, the extraction unit 10021 includes a first determination subunit 100211 for determining, for any pixel in a raw image, the energy ratio of the driving energy of each color component to the total driving energy of all color components within the range of a second target region to which the pixel belongs, as a local color feature corresponding to that pixel, wherein the second target region is an image region of a second preset size with the pixel as the reference point, and a second determination subunit 100212 for determining the set of obtained local color features as a local color feature set.

[0136] In several selectable implementations, the first decision unit 10022 includes a third decision subunit 100221 for determining the dominant color component from each color component in the second target region to which any pixel in the raw image belongs, wherein the dominant color component is the color component with the highest energy ratio in the second target region, and a fourth decision subunit 100222 for determining the luminance weights corresponding to each color component contained in the pixel based on the energy ratio of the dominant color component.

[0137] In several selectable implementation methods, the fourth decision subunit 100222 is configured to further determine whether the energy ratio of the dominant color component is greater than or equal to a first ratio threshold if the dominant color component is a pre-specified target color component, to obtain a dominant color filter mask pre-set for the dominant color component if the energy ratio of the dominant color component is greater than or equal to the first ratio threshold, to determine the luminance weights corresponding to each color component corresponding to the pixel based on the dominant color filter mask, and if the energy ratio of the dominant color component is less than the first ratio threshold and greater than or equal to a second ratio threshold, to determine a weight adjustment coefficient corresponding to the energy ratio of the dominant color component based on the first and second ratio thresholds, to adjust the luminance weights included in the dominant color filter mask using the weight adjustment coefficient, and to determine the luminance weights corresponding to each color component corresponding to the pixel based on the adjusted color filter mask.

[0138] In several selectable implementation methods, the fourth decision subunit 100222 is further configured to obtain a pre-set balanced color filter mask, calculate a weighted sum of the luminance weights included in the dominant color filter mask and the balanced color filter mask, respectively, using a weight adjustment coefficient, and obtain an adjusted color filter mask.

[0139] In several selectable implementations, the fourth decision subunit 100222 is further configured to obtain a preset equilibrium color filter mask if the dominant color component is not a preset target color component, and to determine the luminance weights corresponding to each color component contained in the pixel based on the equilibrium color filter mask.

[0140] In several selectable implementations, the estimation module 1002 includes an encoding unit 10023 for performing downsampling encoding on a raw image using an encoder included in a pre-trained luminance estimation model to obtain an initial feature map; an enhancement unit 10024 for performing luminance feature enhancement on the initial feature map using a feature enhancement network included in the luminance estimation model to obtain an enhanced feature map; and a decoding unit 10025 for performing upsampling decoding on the enhanced feature map using a decoder included in the luminance estimation model to obtain a reconstructed feature map, wherein the resolution of the reconstructed feature map is the same as the resolution of the raw image, and each pixel in the reconstructed feature map corresponds to a preset number of color feature values; and a normalization unit 10026 for performing normalization processing on each color feature value of each pixel in the reconstructed feature map using a weight calculation unit included in the luminance estimation model to obtain a set of luminance weights corresponding to each pixel.

[0141] In several selectable implementations, the luminance estimation model is pre-trained by the following steps: acquiring a sample image; converting the sample image into a sample luminance image; performing region segmentation on the sample luminance image and setting a training task label value for each pixel in each segmented region based on the current training task type; estimating luminance weights for each pixel in the sample image using the initial luminance estimation model to obtain a set of predicted luminance weights corresponding to each pixel; determining a predicted luminance value corresponding to each pixel in the sample image based on the set of predicted luminance weights; determining a prediction error corresponding to the training task type based on the predicted luminance value and the training task label value; adjusting the parameters of the initial luminance estimation model based on the error; and determining the current initial luminance estimation model as the trained luminance estimation model if the parameter-adjusted initial luminance estimation model satisfies a pre-set training completion condition.

[0142] In several selectable implementations, the step of setting a learning task label value for each pixel contained in each divided region based on the current learning task type includes, if the learning task type is a brightness value prediction task, setting a brightness label value for each pixel contained in each divided region, and if the learning task type is a visual perception task, setting a perception label value for each pixel contained in each divided region. The step of determining the prediction error corresponding to the learning task type based on the predicted brightness value and the learning task label value includes, if the learning task type is a brightness value prediction task, determining the error between the predicted brightness value and the brightness label value using a pre-configured first loss function, if the learning task type is a brightness value prediction task, performing image perception prediction using a pre-configured perception network based on the predicted brightness value to obtain a perception prediction result, and determining the error between the perception prediction result and the perception label value using a pre-configured second loss function.

[0143] In several selectable implementations, the computation module 1003 includes: a second determination unit 10031 for determining the driving energy of each color component and the number of pixels included in each color component within the range of a first target region to which any pixel in the raw image belongs; a third determination unit 10032 for determining the pixel-average energy of each color component based on the driving energy of each color component and the number of pixels included in each color component; and a computation unit 10033 for calculating a weighted sum of the obtained pixel-average energies based on the luminance weight set corresponding to the pixel, and obtaining a luminance value corresponding to the pixel.

[0144] The exemplary embodiment of this apparatus and the exemplary method portion described above correspond to each other in their implementation methods, and their corresponding contents can be mutually referenced, combined, and cited; therefore, redundant explanations are omitted here. For beneficial technical effects corresponding to the exemplary embodiment of this apparatus, refer to the corresponding beneficial technical effects of the exemplary method portion described above; therefore, redundant explanations are omitted here.

[0145] Exemplary electronic device Figure 12 is a structural diagram of an electronic device 1200 according to an embodiment of the present disclosure, the electronic device 1200 includes at least one processor 1201 and a memory 1202.

[0146] The processor 1201 may be a central processing unit (CPU) or another type of processing unit having data processing capability and / or instruction execution capability, and can control other components in the electronic device 1200 to perform desired functions.

[0147] Memory 1202 may include one or more computer program products, including various forms of computer-readable storage media such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or high-speed cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored in the computer-readable storage media, and the processor 1201 can execute one or more computer program instructions to implement the image brightness evaluation method and / or other desired functions of each embodiment of the present disclosure described above.

[0148] In one example, the electronic device 1200 may further include an input device 1203 and an output device 1204, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0149] The input device 1203 may further include, for example, a keyboard, a mouse, etc.

[0150] The output device 1204 can output various types of information to the outside, and may include, for example, a display, a speaker, a printer, and a communication network and a remote output device connected thereto.

[0151] Of course, for the sake of simplification, Figure 12 shows only some of the components in the electronic device 1200 that are relevant to this disclosure, and components such as buses and input / output interfaces are omitted. In addition, depending on the specific application, the electronic device 1200 may also include other appropriate components.

[0152] Exemplary computer program products and computer-readable storage media Embodiments of the present disclosure may also be computer program products that, in addition to the methods and apparatus described above, include computer program instructions that, when executed by a processor, cause the processor to perform steps of the image brightness evaluation method of the various embodiments of the present disclosure described in the “Exemplary Methods” section above.

[0153] Computer program products can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java® and C++, and common procedural programming languages ​​such as the C language or similar programming languages, to create program code for performing the operations of the embodiments of this disclosure. The program code may run entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device, partially on a remote computing device, or entirely on a remote computing device or server.

[0154] Furthermore, embodiments of the present disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform steps of the image brightness evaluation method of various embodiments of the present disclosure as described in the “Exemplary Methods” section above.

[0155] Computer-readable storage media may employ any combination of one or more readable media. The readable media may be readable signal media or readable storage media. Readable storage media may include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (non-exclusive list) of readable storage media include electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0156] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, advantages, and effects mentioned in this disclosure are merely illustrative and not limiting, and it is not assumed that these advantages, advantages, and effects must be present in each example of this disclosure. Furthermore, the specific details disclosed above are merely illustrative and intended to facilitate understanding, and are not limiting, nor do they imply that this disclosure must be implemented in the specific details described above.

[0157] Those skilled in the art can make various modifications and variations to this disclosure without departing from the spirit and scope of the application. Thus, if such modifications and variations of the application fall within the scope of the claims of this disclosure and the equivalent art, this disclosure is intended to include such modifications and variations.

Claims

1. An image brightness evaluation method, wherein each step is performed by an image brightness evaluation device, Steps include obtaining the raw image to be evaluated for brightness, A step of performing luminance weight estimation for each pixel in the raw image and obtaining a set of luminance weights corresponding to each pixel, wherein each luminance weight in the luminance weight set corresponds to one color component, A step of obtaining a luminance value for an arbitrary pixel in the raw image by calculating a weighted sum for each color component included in a first target region to which the pixel belongs, based on a luminance weight set corresponding to the pixel, wherein the first target region is an image region of a first preset size with the pixel as the reference point. A method for evaluating image brightness, comprising the step of generating a brightness evaluation image corresponding to the raw image based on the brightness value corresponding to each pixel in the raw image.

2. The step of performing brightness weight estimation for each pixel in the raw image and obtaining a brightness weight set corresponding to each pixel is: The steps include: extracting local color features for each pixel based on the color component values ​​of each pixel in the raw image, and obtaining a set of local color features; The image brightness evaluation method according to claim 1, comprising the step of determining a brightness weight set corresponding to each pixel in the raw image based on the local color feature set.

3. The step of extracting local color features for each pixel based on the color component values ​​of each pixel in the raw image and obtaining a set of local color features is as follows: A step of determining, for any pixel in the raw image, the energy ratio of the driving energy of each color component to the total driving energy of all color components within the range of a second target region to which the pixel belongs, as a local color feature corresponding to that pixel, wherein the second target region is an image region of a second preset size with the pixel as the reference point. The image brightness evaluation method according to claim 2, comprising the step of determining the obtained set of local color features as the local color feature set.

4. The step of determining a set of luminance weights corresponding to each pixel in the raw image based on the set of local color features is: A step of determining the dominant color component from the color components in the second target region to which any pixel in the raw image belongs, wherein the dominant color component is the color component with the highest energy ratio in the second target region. The image brightness evaluation method according to claim 3, comprising the step of determining brightness weights corresponding to each color component contained in the pixel based on the energy ratio of the dominant color component.

5. The step of determining the luminance weights corresponding to each color component contained in the pixel based on the energy ratio of the dominant color component is as follows: If the dominant color component is a pre-specified target color component, the step is to determine whether the energy ratio of the dominant color component is equal to or greater than a first ratio threshold. If the energy ratio of the dominant color component is greater than or equal to the first ratio threshold, the steps include obtaining a dominant color filter mask that has been set in advance for the dominant color component, and determining the luminance weights corresponding to each color component corresponding to the pixel based on the dominant color filter mask. If the energy ratio of the dominant color component is less than the first ratio threshold and greater than or equal to the second ratio threshold, the step of determining a weight adjustment coefficient corresponding to the energy ratio of the dominant color component based on the first ratio threshold and the second ratio threshold, The image brightness evaluation method according to claim 4, comprising the steps of adjusting the brightness weights included in the dominant color filter mask using the weight adjustment coefficient, and determining the brightness weights corresponding to each color component corresponding to the pixel based on the adjusted color filter mask.

6. The step of adjusting the luminance weights included in the dominant color filter mask using the weight adjustment coefficient is: The steps include obtaining a pre-set balanced color filter mask, The image luminance evaluation method according to claim 5, comprising the step of using the weight adjustment coefficient to calculate a weighted sum for the luminance weights included in the dominant color filter mask and the balanced color filter mask, respectively, to obtain an adjusted color filter mask.

7. The step of determining the luminance weights corresponding to each color component contained in the pixel based on the energy ratio of the dominant color component is as follows: If the dominant color component is not a pre-specified target color component, the steps include obtaining a pre-set equilibrium color filter mask, The image brightness evaluation method according to claim 4, comprising the step of determining brightness weights corresponding to each color component contained in the pixel based on the balanced color filter mask.

8. The step of performing brightness weight estimation for each pixel in the raw image and obtaining a brightness weight set corresponding to each pixel is: The steps include: using an encoder included in a pre-trained brightness estimation model to perform downsampling encoding on the raw image and obtain an initial feature map; The steps include: performing luminance feature enhancement on the initial feature map using the feature enhancement network included in the luminance estimation model to obtain an enhanced feature map; A step of obtaining a reconstructed feature map by performing upsampling decoding on the enhanced feature map using a decoder included in the brightness estimation model, wherein the resolution of the reconstructed feature map is the same as the resolution of the raw image, and each pixel in the reconstructed feature map corresponds to a preset number of color feature values. The image brightness evaluation method according to claim 1, comprising the step of using a weight calculation unit included in the brightness estimation model to perform normalization processing on each color feature value of each pixel included in the restored feature map to obtain a brightness weight set corresponding to each pixel.

9. The aforementioned luminance estimation model is, Steps to obtain a sample image, The steps include converting the aforementioned sample image into a sample brightness image, The steps include performing region segmentation on the aforementioned sample brightness image and setting a training task label value for each pixel contained in each segmented region based on the current training task type, The steps include: using an initial brightness estimation model to estimate brightness weights for each pixel in the sample image and obtaining a set of predicted brightness weights corresponding to each pixel; The steps include determining a predicted brightness value corresponding to each pixel in the sample image based on the predicted brightness weight set, The steps include determining the prediction error corresponding to the type of learning task based on the predicted brightness value and the learning task label value, The steps include adjusting the parameters of the initial brightness estimation model based on the aforementioned error, The image brightness evaluation method according to claim 8, wherein the initial brightness estimation model after parameter adjustment satisfies a pre-set learning termination condition, and the current initial brightness estimation model is determined to be the brightness estimation model after learning.

10. The step of setting a learning task label value for each pixel contained in each of the divided regions, based on the current learning task type, is: If the learning task type is a brightness value prediction task, the steps include setting brightness label values ​​for each pixel included in each of the divided regions, If the learning task type is a visual perception task, the step includes setting a perception label value for each pixel included in each of the divided regions. The step of determining the prediction error corresponding to the type of learning task based on the predicted brightness value and the learning task label value is: If the learning task type is a brightness value prediction task, the steps include determining the error between the predicted brightness value and the brightness label value using a pre-set first loss function, If the learning task type is a visual perception task, the steps include: performing image perception prediction using a pre-configured perception network based on the predicted brightness value and obtaining a perception prediction result; The image brightness evaluation method according to claim 9, comprising the step of determining the error between the sensing prediction result and the sensing label value using a pre-set second loss function.

11. For any pixel in the raw image, the step of calculating a weighted sum for each color component included in the first target region to which the pixel belongs, based on the luminance weight set corresponding to the pixel, and obtaining the luminance value corresponding to the pixel, is: For any pixel in the raw image, the steps include determining the driving energy of each color component and the number of pixels included in each color component within the range of the first target region to which the pixel belongs. The steps include determining the pixel-average energy of each color component based on the driving energy of each color component and the number of pixels included in each color component, A method for evaluating image brightness according to any one of claims 1 to 10, comprising the steps of: calculating a weighted sum for each obtained pixel average energy based on a brightness weight set corresponding to the pixel, and obtaining a brightness value corresponding to the pixel.

12. An acquisition module for obtaining the raw image to be evaluated for brightness, An estimation module for performing luminance weight estimation for each pixel in the raw image and obtaining a luminance weight set corresponding to each pixel, wherein each luminance weight in the luminance weight set corresponds to one color component, A calculation module for obtaining a luminance value for an arbitrary pixel in the raw image, by calculating a weighted sum for each color component included in a first target region to which the pixel belongs, based on a luminance weight set corresponding to the pixel, wherein the first target region is an image region of a first preset size with the pixel as the reference point. An image brightness evaluation apparatus, comprising: a generation module for generating a brightness evaluation image corresponding to the raw image based on the brightness value corresponding to each pixel in the raw image.

13. A computer-readable storage medium storing a computer program for implementing the image brightness evaluation method described in any one of claims 1 to 10, which is executed by a processor.

14. Processor and The processor includes a memory for storing executable instructions, The processor is an electronic device that realizes the image brightness evaluation method according to any one of claims 1 to 10 by reading and executing the executable instructions from the memory.

Images