Image contrast enhancement method, apparatus, readable storage medium, and electronic device

Abstract

This disclosure provides an image contrast enhancement method, apparatus, readable storage medium, and electronic device. [Solution] This method includes the steps of: obtaining a source image to be subjected to contrast enhancement; performing contrast enhancement on the source image to obtain an initial contrast-enhanced image; processing the source image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map; and fusing the initial contrast-enhanced image and the source image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

Description

Technical Field

[0001] The present disclosure relates to digital image processing technology and computer vision technology, and particularly to an image contrast enhancement method, apparatus, readable storage medium, and electronic device.

Background Art

[0002] Image contrast enhancement, also known as contrast stretching or gray-scale stretching, is a fundamental and important image processing technology. Its core purpose is to change the gray-scale value of each pixel in the input image one by one through a specific conversion function, stretch or compress the pixel luminance value range that was originally concentrated in a narrow dynamic range, and expand the luminance difference between different objects, so as to make the details of the image clearer and the visual effect more vivid. }

[0003] Related image contrast enhancement methods include methods such as global histogram equalization. When processing images with complex content, these methods perform a unified conversion on the entire image, ignoring the differences in the content characteristics of different regions.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to solve the above technical problems, the present disclosure provides an image contrast enhancement method, apparatus, readable storage medium, and electronic device for solving the problems that noise amplification and excessive enhancement are likely to occur in flat regions of an image, and the contrast enhancement intensity of the texture region is insufficient.

Means for Solving the Problems

[0005] A first embodiment of the embodiments of the present disclosure provides an image contrast enhancement method comprising: acquiring a source image to be contrast enhanced; performing contrast enhancement on the source image to obtain an initial contrast-enhanced image; processing the source image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map reduce the contrast enhancement intensity of flat areas in the source image and increase the contrast enhancement intensity of textured areas in the source image; and fusing the initial contrast-enhanced image and the source image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

[0006] An image contrast enhancement apparatus according to a second embodiment of the embodiments of the present disclosure includes: an acquisition module for acquiring a source image to be contrast enhanced; an enhancement module for performing contrast enhancement on the source image to obtain an initial contrast-enhanced image; an estimation module for processing the source image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map are calculated by reducing the contrast enhancement intensity of flat regions in the source image and increasing the contrast enhancement intensity of textured regions in the source image; and a fusion module for fusing the initial contrast-enhanced image and the source image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

[0007] A computer-readable storage medium according to a third embodiment of the embodiments of the present disclosure stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the steps of the image contrast enhancement method described above.

[0008] An electronic device according to a fourth embodiment of the embodiments of the present disclosure includes a processor and a memory for storing instructions that the processor can execute, and the image contrast enhancement method described above is realized by the processor reading an executable instruction from the memory and executing the instruction. [Effects of the Invention]

[0009] According to the image contrast enhancement method, apparatus, readable storage medium, and electronic device of the embodiment of the present disclosure, an initial contrast-enhanced image is obtained by performing contrast enhancement on the original image, and a fusion coefficient map is obtained by processing the original image using a pre-trained fusion coefficient estimation model, where the fusion coefficient is used to reduce the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image, and finally, the initial contrast-enhanced image and the original image are fused using the fusion coefficient included in the fusion coefficient map to obtain a fused contrast-enhanced image. The embodiment of the present application realizes that different areas in the image are targeted with different degrees of contrast enhancement according to the degree of texture intensity of different areas, so that the fused contrast-enhanced image can effectively embody the differences in content characteristics in the image, avoid noise amplification and over-enhancement in flat areas and insufficient contrast enhancement in textured areas, and can effectively highlight texture details, thereby improving the adaptability and accuracy of image contrast enhancement. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic flowchart of an image contrast enhancement method according to an exemplary embodiment of the present disclosure. [Figure 2] Figure 2 is a schematic flowchart of an image contrast enhancement method according to another exemplary embodiment of the present disclosure. [Figure 3] Figure 3 is a schematic flowchart of an image contrast enhancement method according to yet another exemplary embodiment of the present disclosure. [Figure 4] Figure 4 is a schematic flowchart of an image contrast enhancement method according to yet another exemplary embodiment of the present disclosure. [Figure 5] Figure 5 is a schematic flowchart of an image contrast enhancement method according to yet another exemplary embodiment of the present disclosure. [Figure 6] Figure 6 is a schematic diagram of the structure of an image texture estimation model according to an exemplary embodiment of the present disclosure. [Figure 7] Figure 7 is a schematic flowchart of an image contrast enhancement method according to yet another exemplary embodiment of the present disclosure. [Figure 8] Figure 8 is a schematic diagram of the training flow of an image texture estimation model according to an exemplary embodiment of the present disclosure. [Figure 9] Figure 9 is a schematic flowchart of an image contrast enhancement method according to yet another exemplary embodiment of the present disclosure. [Figure 10] Figure 10 is a structural diagram of an image contrast enhancement device according to an exemplary embodiment of the present disclosure. [Figure 11] Figure 11 is a structural diagram of an image contrast enhancement device according to another exemplary embodiment of the present disclosure. [Figure 12] Figure 12 is a structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. [Modes for carrying out the invention]

[0011] Hereinafter, exemplary embodiments of this disclosure will be described in detail with reference to the drawings in order to interpret this disclosure. The embodiments described are only a selection of embodiments of this disclosure, not all embodiments, and this disclosure is not limited to exemplary embodiments.

[0012] Unless otherwise specified, the relative arrangements of the components and steps, formulas, and numerical values ​​described in these embodiments do not limit the scope of this disclosure.

[0013] [Summary of the application] Image contrast enhancement by methods such as related global histogram equalization has an inherent defect that when processing an image with complex content, a uniform transformation is applied to the entire image, ignoring the differences in content characteristics of different regions. As a result, in flat regions (such as sky, wall surface, etc.), noise amplification and excessive enhancement are likely to occur after enhancement, resulting in unnatural blocky or grainy sensations. On the other hand, in texture detail regions (such as object edges, complex patterns, etc.), due to insufficient enhancement intensity, it may not be possible to effectively highlight the details.

[0014] In order to solve the above problems, the present application provides a method that can adaptively enhance the contrast according to the image content, realize the differentiation and intelligent enhancement for different image regions, and improve the adaptability and accuracy (targetability) of image contrast enhancement.

[0015] [Exemplary Method] FIG. 1 is a schematic flowchart of an image contrast enhancement method according to an exemplary embodiment of the present disclosure. This embodiment is applicable to various types of electronic devices. As shown in FIG. 1, it includes the following steps 101 to 104.

[0016] In step 101, an original image to be subjected to contrast enhancement is acquired.

[0017] Here, the above original image can be an image taken in any scene, for example, an image taken by a camera on a vehicle, an image taken by a user using a mobile phone, etc. Optionally, the original image may be a grayscale image or a color image.

[0018] In step 102, contrast enhancement is performed on the original image to obtain an initial contrast-enhanced image.

[0019] Electronic devices performing this method can perform this step using various relevant contrast enhancement methods. Selectively, contrast enhancement can be performed by employing at least one algorithm from among histogram equalization algorithms, linear contrast stretching algorithms, and deep learning-based image contrast enhancement algorithms. The initial contrast-enhanced image may be a grayscale image or a color image obtained after restoring a grayscale image.

[0020] In step 103, the original image is processed using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map.

[0021] Here, the fusion coefficient in the fusion coefficient map is used to decrease the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image. The size of the fusion coefficient map can be the same as the size of the original image, that is, the fusion coefficient in the fusion coefficient map corresponds one-to-one with the pixels in the original image. The fusion coefficient in the fusion coefficient map can represent the proportion of grayscale values ​​obtained from the initial contrast-enhanced image when generating the fused contrast-enhanced image. That is, the larger the fusion coefficient, the greater the proportion of grayscale values ​​in the initial contrast-enhanced image in the fused contrast-enhanced image, and the smaller the fusion coefficient, the greater the proportion of grayscale values ​​in the fused contrast-enhanced image in the fused contrast-enhanced image in the original image.

[0022] Selectively, the fusion coefficient can employ data that reflects the degree of texture (or flatness) of the region where the pixel is located. Generally, a higher fusion coefficient indicates a higher probability that the corresponding pixel is located in a textured region, while a lower fusion coefficient indicates a higher probability that the corresponding pixel is located in a flat region.

[0023] The above fusion coefficient estimation model is intended to represent the correspondence between the original image and the fusion coefficient map. This fusion coefficient estimation model can be constructed using various methods, such as setting a conversion formula for grayscale values ​​or training a neural network model using machine learning methods.

[0024] When selectively training a fusion coefficient estimation model using machine learning methods, a large number of sample images can be collected beforehand, and the fusion coefficient can be annotated for each pixel in each sample image. Then, the error between the fusion coefficient output by the model and the annotated fusion coefficient is gradually reduced through iterative adjustment of the initial model parameters. When the error converges, the current initial model can be used as the trained fusion coefficient estimation model.

[0025] Selectively, an image texture estimation model can be used to perform texture probability estimation on the original image to obtain a texture probability estimation map, and then a transformation can be performed on the texture probability estimates included in the texture probability estimation map to obtain a fusion coefficient map.

[0026] In step 104, the initial contrast-enhanced image and the original image are fused using the fusion coefficients included in the fusion coefficient map to obtain the fused contrast-enhanced image.

[0027] Since there is a one-to-one correspondence between the fusion coefficients included in the fusion coefficient map and the pixels included in the initial contrast-enhanced image, a fused grayscale image can be obtained by performing calculations on the grayscale value of any pixel in the initial contrast-enhanced image or the original image based on the corresponding fusion coefficient. For example, a fused grayscale image can be obtained by multiplying the average value of the grayscale value of the corresponding pixel in the initial contrast-enhanced image and the original image by the corresponding fusion coefficient.

[0028] Depending on the actual demand, the resulting grayscale image may be used as the contrast-enhanced image after fusion, or the resulting grayscale image may be restored to a color image and used as the contrast-enhanced image after fusion.

[0029] The image contrast enhancement method according to the embodiment of this disclosure involves performing contrast enhancement on the original image to obtain an initial contrast-enhanced image, processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, where the fusion coefficients are used to reduce the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image, and finally, fusing the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image. The embodiment of this application realizes that different areas in the image are targeted with different degrees of contrast enhancement, according to the degree of texture intensity of different areas, so that the fused contrast-enhanced image can effectively embody the differences in content characteristics in the image, avoid noise amplification and excessive enhancement in flat areas and insufficient contrast enhancement in textured areas, and can effectively highlight texture details, thereby improving the adaptability and accuracy of image contrast enhancement.

[0030] In several selectable implementations, as shown in Figure 2, step 102 includes the following steps 1021 to 1023.

[0031] In step 1021, for any pixel in the original image, the grayscale value variance of the target region in which that pixel is located is determined.

[0032] Here, the target region is an image region of a predetermined size with the pixel in question as the reference point. The position of the reference point for each image region can be arbitrarily specified, and usually the reference point can be the center point of the image region. The predetermined size mentioned above can be arbitrarily set, for example, an image block of size such as 3x3 or 5x5.

[0033] The formula for calculating grayscale value variance is as follows:

number

[0034] Here, σ 2 This is the grayscale value variance, and x i μ is the grayscale value of the i-th pixel within the target region, μ is the grayscale average value of each pixel, and N is the number of pixels within the target region.

[0035] Note that the above-mentioned pre-set size is the maximum dimension of each image region. In the boundary and corner regions of the original image, the number of pixels around the reference point does not meet the number of pixels in the image region specified by the above-mentioned pre-set size. Therefore, the size of the image region corresponding to that reference point can be smaller than the pre-set size.

[0036] In step 1022, a variance-weighted grayscale histogram is obtained by performing a weighted statistical analysis on the number of different grayscale pixels in the original image, based on the grayscale value variance corresponding to each pixel.

[0037] A grayscale histogram is a statistical representation of the frequency of occurrence of all pixels in an image, according to the magnitude of their grayscale values. In other words, a grayscale histogram is a function of grayscale levels and represents the number of pixels in an image that have a particular grayscale level.

[0038] In conventional histogram equalization mapping methods, when statistically analyzing a grayscale histogram, the number of pixels corresponding to each grayscale value appearing in the image is statistically determined. However, in this embodiment, when statistically analyzing a grayscale histogram, each pixel is not statistically analyzed as a single unit. Instead, a corresponding statistical weight is determined according to the grayscale value variance corresponding to that pixel, and a variance-weighted grayscale histogram is obtained by performing weighted statistics according to the statistical weight. Typically, the relationship between the weight and the grayscale value variance is such that the larger the grayscale value variance, the larger the statistical weight, and the smaller the grayscale value variance, the smaller the statistical weight.

[0039] In step 1023, an equalization mapping is performed on the original image based on a variance-weighted grayscale histogram to obtain an initial contrast-enhanced image.

[0040] Here, the histogram-based equalization mapping method is a type of image processing technique that enhances image contrast by changing the grayscale level distribution of the image. The steps to achieve this mainly involve calculating the grayscale histogram, the total number of pixels, the grayscale distribution frequency, and the cumulative distribution frequency, then normalizing each pixel and mapping it to a new grayscale level, finally obtaining an image with enhanced contrast.

[0041] This embodiment uses a variance-weighted grayscale histogram that can reflect the weighted grayscale statistical results when performing histogram equalization mapping. The weighted grayscale statistical results can reflect the grayscale differences in the region where each pixel is located, that is, they can reflect the degree of texture intensity in the region where each pixel is located. The resulting initial contrast-enhanced image can be made more adaptable to the texture features of the original image by applying different degrees of contrast enhancement to the textured and flat regions.

[0042] In several selectable implementations, as shown in Figure 3, step 1022 includes the following steps 10221 to 10223.

[0043] Step 10221 involves statistically counting the number of pixels corresponding to various grayscale values ​​in the original image.

[0044] In other words, the number of pixels with the same grayscale value in the original image is counted to determine the number of pixels corresponding to each grayscale value.

[0045] In step 10222, the statistical count weight corresponding to each pixel in the original image is determined based on the pre-set correspondence between the grayscale value variance and the statistical count weight.

[0046] Here, the statistical count weight increases with increasing grayscale value variance. The correspondence between grayscale value variance and statistical count weight can be expressed in the form of a table, formula, etc. The principle embodied by this correspondence is that the larger the grayscale value variance, the larger the statistical count weight, and the smaller the grayscale value variance, the smaller the statistical count weight. For example, a mapping curve can be set up, and the statistical count weight corresponding to the grayscale value variance can be determined based on this mapping curve.

[0047] In step 10223, a weighted statistical analysis is performed on the number of pixels corresponding to various grayscale values ​​in the original image, based on the statistical count weights corresponding to each pixel in the original image, to obtain a variance-weighted grayscale histogram.

[0048] Specifically, when statistically counting the number of pixels corresponding to various grayscale values, a calculation (e.g., multiplication, addition, etc.) is performed on each pixel against the corresponding statistical count weight and a pre-set statistical base (e.g., 1). The resulting calculation is taken as the actual statistical increment for that pixel, and this statistical increment is accumulated in the number of pixels corresponding to the grayscale value of that pixel. Optionally, the accumulation may be performed directly on the above calculation result, or it may be performed again (e.g., normalization) on the calculation result before accumulation.

[0049] This embodiment improves the degree of consistency between the variance of grayscale values ​​and the actual scene by setting a correspondence between the grayscale value variance and the statistical count weight, so that the variance-weighted grayscale histogram reflects the degree of grayscale value difference in the region where each pixel is located, that is, it reflects the degree of texture in that region, thereby contributing to improved adaptability of image contrast enhancement.

[0050] In several selectable implementations, as shown in Figure 4, step 104 includes the following steps 1041 to 1043.

[0051] In step 1041, a first weight corresponding to each pixel in the initial contrast-enhanced image and a second weight corresponding to each pixel in the original image are determined based on the fusion coefficients included in the fusion coefficient map.

[0052] Typically, the sum of the first weight and the second weight can be 1. Selectively, for any pixel in the initial contrast-enhanced image, the fusion coefficient corresponding to that pixel can be used as the first weight, and the corresponding second weight can be calculated from this. For example, if the fusion coefficient of a certain pixel is α, then α can be used as the first weight w to obtain the first weight w. Correspondingly, the second weight is 1-w.

[0053] In step 1042, a weighted sum is calculated for the grayscale values ​​of the corresponding pixels in the initial contrast-enhanced image and the original image, based on the first and second weights.

[0054] Specifically, the grayscale value after merging any of the pixels can be calculated according to the following formula.

number

[0055] Here, (x,y) is the coordinate of a pixel, I_final(x,y) is the grayscale value of a pixel in the fused grayscale image, w is the first weight, 1-w is the second weight, I_enhanced(x,y) is the grayscale value in the initial contrast-enhanced image corresponding to that pixel, and I_original(x,y) is the grayscale value in the original image corresponding to that pixel.

[0056] As can be seen from equation (2), pixels with a high fusion coefficient tend to retain more of the corresponding grayscale values ​​in the initial contrast-enhanced image, the more likely they are to be located in a texture region, and the fusion result tends to embody richer texture details by applying high-intensity contrast enhancement to the texture region. For pixels with a low fusion coefficient, the more likely they are to be located in a flat region, the more of the corresponding grayscale values ​​in the original image are retained, and the fusion result tends to retain the appearance of the original image.

[0057] In step 1043, a contrast-enhanced image is generated after fusion based on the weighted sum of grayscale values.

[0058] Specifically, the weighted sum of the grayscale values ​​of each pixel can be combined into the fused grayscale image. Depending on the actual needs, this fused grayscale image may be used as a contrast-enhanced image, or the fused grayscale image may be restored to a color image and used as a contrast-enhanced image.

[0059] This embodiment utilizes fusion coefficients to determine the weights of corresponding pixels in the initial contrast-enhanced image and the original image. By using these weights to perform grayscale value fusion, the fused contrast-enhanced image can achieve high-intensity contrast enhancement in textured areas while retaining the appearance of the original image in flat areas. This allows the contrast-enhanced image to more accurately reflect the appearance and characteristics of the actual scene.

[0060] In several selectable implementations, as shown in Figure 5, step 103 above includes the following steps 1031 to 1032.

[0061] In step 1031, a texture probability estimation is performed on the original image using a pre-trained image texture estimation model to obtain a texture probability estimation map.

[0062] Here, the texture probability estimates included in the texture probability estimation map correspond one-to-one with the pixels in the original image.

[0063] The texture probability estimates included in the texture probability estimation map above correspond one-to-one with the pixels in the original image. The texture probability estimate represents the probability that the corresponding pixel is located in a textured area. A larger texture probability estimate indicates a higher probability that the corresponding pixel is located in a textured area, while a smaller texture probability estimate indicates a higher probability that the corresponding pixel is located in a flat area.

[0064] The image texture estimation model described above can be obtained by training it beforehand using machine learning methods. For example, a large number of sample images can be collected in advance, and texture probability values ​​can be annotated for each pixel in each sample image. Then, by iteratively adjusting the parameters of the initial model, the error between the predicted texture probability estimation map output by the model and the annotated texture probability estimation map is gradually reduced. When the error converges, the current initial model can be used as the trained image texture estimation model.

[0065] In step 1032, based on the pre-configured transformation relationships, the texture probability estimates included in the texture probability estimation map are transformed to obtain a fusion coefficient map.

[0066] Here, the above-mentioned conversion relationships can be set in advance. These conversion relationships can be embodied in the form of calculation formulas, correspondence tables, etc. By using these conversion relationships, each texture probability estimate can be converted into a fusion coefficient. The resulting fusion coefficients are combined to form a single fusion coefficient map.

[0067] Typically, there can be a positive relationship between the texture probability estimate and the fusion coefficient. That is, the larger the texture probability estimate, the higher the probability that the corresponding pixel belongs to the texture region, and consequently, the larger the fusion coefficient. A larger fusion coefficient allows for a higher degree of grayscale value acquisition from the initial contrast-enhanced image and contributes to a higher degree of contrast enhancement of texture details. Conversely, the smaller the texture probability estimate, the lower the probability that the corresponding pixel belongs to the texture region, and consequently, the smaller the fusion coefficient. A smaller fusion coefficient allows for a higher degree of grayscale value acquisition from the original image and contributes to a higher degree of preservation of the original appearance features of flat regions.

[0068] This embodiment uses an image texture estimation model to perform texture probability estimation on the original image, and then performs a transformation on the texture probability estimate to obtain a fusion coefficient map. This enables the determination of a fusion coefficient that corresponds to a target based on the texture probability of each region in the image, thereby contributing to improved accuracy in fusion coefficient generation.

[0069] In several selectable implementations, step 1032 is performed. The procedure may include a step to obtain a fusion coefficient map by mapping the texture probability estimates included in the texture probability estimation map to fusion coefficients using a pre-configured mapping function.

[0070] The mapping function can reflect the relationship between the texture probability estimate and the fusion coefficient. The form of the mapping function can be arbitrarily set, and typically, the goal of setting the mapping function is to make the fusion coefficient relatively large in textured areas and relatively small in flat areas.

[0071] This embodiment sets a mapping function to target arbitrary texture probability estimates to fusion coefficients, enabling the fusion coefficients corresponding to each pixel to more accurately reflect the texture of the pixel's location.

[0072] In several possible implementations, the mapping function is a monotonically increasing mapping function.

[0073] The fusion coefficient map can be calculated and obtained by following the steps below.

[0074] For any texture probability estimate included in the texture probability estimation map, a monotonically increasing mapping function is used to map the texture probability estimate to a first fusion coefficient if the corresponding pixel is located in a texture region, and to map the texture probability estimate to a second fusion coefficient if the corresponding pixel is located in a flat region.

[0075] Here, the first fusion coefficient is greater than the second fusion coefficient.

[0076] The form of the monotonically increasing mapping function can be arbitrarily set. For example, the monotonically increasing mapping function can be α = a + β × p, where a and β are tunable hyperparameters greater than 0, and p is the texture probability estimate. The significance of this function is that when p for a given pixel is close to 1, it indicates that the pixel is located in the texture region, and the corresponding α is close to a + β, playing a role in improving the contrast enhancement intensity. When p is close to 0, it indicates that the pixel is located in the flat region, and the corresponding α is close to a, playing a role in decreasing the contrast enhancement intensity.

[0077] In this embodiment, by setting a monotonically increasing mapping function, different fusion coefficients of varying sizes are assigned to the textured region and the flat region, thereby improving the targeting and accuracy of the image contrast enhancement area.

[0078] In several selectable implementations, as shown in Figure 6, the image texture estimation model includes an image block segmentation module 601, a feature extraction module 602, an encoder 603, a low-resolution probability estimation module 604, and an upsampling module 605.

[0079] As shown in Figure 7, step 1031 includes the following steps 10311 to 10315.

[0080] In step 10311, the image block division module is used to divide the original image into blocks and obtain a set of image blocks.

[0081] This embodiment does not limit the method used to perform block division on an image. For example, by using a convolutional layer with a convolution kernel size and stride of P (e.g., 16) as an image block division module and performing a convolution operation on the input original image, the image can be divided into multiple non-overlapping P×P size image blocks.

[0082] In step 10312, the feature extraction module is used to extract features from each image block in the image block set to obtain an image token sequence.

[0083] Here, each image token in the image token sequence corresponds to one image block.

[0084] The feature extraction module can be implemented using a learnable linear projection layer. This feature extraction module can unpack each image block into a single vector and map that vector to a fixed-dimensional embedding space using a linear projection layer (usually a fully connected layer). For example, each image token in an image token sequence is a 1 × C size feature vector, where C is the feature dimension. The image token sequence can constitute a feature map with size (H / P) × (W / P) × C, where H and W are the height and width of the original image.

[0085] In step 10313, an encoder is used to encode the image token sequence to obtain an encoded data sequence.

[0086] Each encoded data in the encoded data sequence is obtained by the encoder encoding the corresponding image token. Selectively, the encoder can be a deformable Transformer encoder. The above token sequence is input to a deformable Transformer encoder in which multiple (e.g., 2 to 4) encoded blocks are stacked, and the encoder's deformable self-attention mechanism allows the model to dynamically focus on other blocks that are most contentically relevant to the current image block, thereby more effectively capturing global context information and accurately determining whether a single image block belongs to a textured region or a flat region.

[0087] In step 10314, a low-resolution probability estimation module is used to perform texture probability prediction for each encoded data in the encoded data sequence to obtain a low-resolution texture probability map.

[0088] Specifically, each encoded data in the encoded data sequence is input to a low-resolution probability estimation module, which then performs texture probability prediction for each encoded data to obtain a corresponding texture probability value. Each texture probability value constitutes a low-resolution texture probability map. The size of the low-resolution texture probability map is (H / P) × (W / P), and the value of each point on the map is between 0 and 1, representing the probability that each image block included in the original image belongs to the texture region.

[0089] In step 10315, the upsampling module is used to upsample the low-resolution texture probability map to obtain a texture probability estimation map with the same resolution as the original image.

[0090] The upsampling module can be implemented based on the relevant interpolation algorithm. For example, using a bilinear interpolation algorithm, a low-resolution texture probability map P_low can be upsampled to the full resolution H×W of the original image to obtain a full-resolution texture probability estimation map P_full. The bilinear interpolation algorithm can generate smooth transitions, effectively avoiding the appearance of blocking effects in the final fused coefficient map.

[0091] This embodiment enables texture probability estimation based on a deep learning model by configuring an image texture estimation model with an image block segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module, thereby effectively improving the accuracy of texture probability estimation.

[0092] In several selectable implementations, as shown in Figure 6, the low-resolution probability estimation module 604 includes a fully connected layer 6041 and an activation function layer 6042.

[0093] Step 10314 can be performed as follows:

[0094] First, a fully connected layer is used to map each encoded data point in the encoded data sequence to a texture feature estimate.

[0095] Subsequently, an activation function layer is used to convert each obtained texture feature estimate into a low-resolution texture probability value, thereby obtaining a low-resolution texture probability map.

[0096] The function of the fully connected layer described above is to integrate the features extracted in the previous layer of the image texture estimation model and output classification data. The type of activation function used by the activation function layer described above can usually be a sigmoid activation function, but it may also be selectively other activation functions such as a tanh activation function. The function of the activation function layer is to perform binary classification on the texture feature estimates and output the probability that the corresponding image region is a texture region.

[0097] This embodiment improves the accuracy of texture probability estimation map generation by enabling accurate texture probability prediction for each encoded data by installing a fully connected layer and an activation function layer in the low-resolution probability estimation module.

[0098] In several selectable implementations, as shown in Figure 8, the image texture estimation model is obtained by being trained in advance according to the following steps 801 to 806.

[0099] Step 801 involves obtaining a sample image.

[0100] Here, the sample image may be an image obtained in advance by capturing a specific type of scene, and the application scene of the image contrast enhancement method according to the embodiment of this disclosure belongs to that type of scene. Selectively, the sample image may be a grayscale image or a color image.

[0101] In step 802, the annotation texture probability value corresponding to each pixel in the sample image is determined.

[0102] Here, the annotation texture probability value is a label between 0 and 1 that has been artificially annotated to each pixel beforehand. Typically, to improve annotation efficiency, the sample image is partitioned, and the size of each partitioned image block is P×P, and the annotator can annotate each image block with a label between 0 and 1. The final label consists of a (H / P)×(W / P) matrix, where each value in the matrix represents the probability that the region at the corresponding position belongs to the texture region.

[0103] In step 803, the initial image texture estimation model is used to perform texture probability estimation on the sample image, and a predicted texture probability estimate corresponding to each pixel in the sample image is obtained.

[0104] The structure of the initial image texture estimation model can be constructed by employing various relevant neural network models, for example, as shown in Figure 6. The initial image texture estimation model can perform texture probability estimation on a sample image according to the flow shown in Figure 7.

[0105] In step 804, the error between the predicted texture probability estimate and the annotation texture probability value is determined using a pre-configured loss function.

[0106] The loss function can be any of the relevant types, for example, an L1 loss function, where the loss value is the mean absolute error between the texture probability value predicted by the network and the true label. This loss function is less sensitive to outliers and is more stable during training.

[0107] In step 805, the parameters of the initial texture estimation model are adjusted based on the error.

[0108] Typically, gradient descent and backpropagation methods can be used to iteratively adjust the parameters of the initial texture estimation model until the model meets the training termination conditions.

[0109] In step 806, if the initial texture estimation model after parameter adjustment meets the pre-set training termination conditions, the current initial texture estimation model is confirmed as the post-trained image texture estimation model.

[0110] Selectively, the above training termination conditions may include at least one of the following: the loss value has converged, the number of training iterations has reached a predetermined number, or the training time has reached a predetermined length of time.

[0111] The image texture estimation model training method according to this embodiment allows the model training process to be adapted to actual application scenarios, thereby improving the accuracy of texture probability estimation by the model.

[0112] In several selectable implementations, step 802 described above can be performed as follows:

[0113] First, the sample image is divided into blocks to obtain a set of sample block regions.

[0114] Subsequently, for each sample block region in the sample block region set, the proportion of the texture region area within that sample block region is determined, and this proportion is determined as the annotation texture probability value for each pixel included in that sample block region.

[0115] Here, the texture region in any of the sample block regions may be artificially defined or automatically defined by another texture recognition model.

[0116] This embodiment simplifies the annotation flow and improves annotation efficiency and model training efficiency by using the proportion of texture area occupied as the annotation texture probability value when performing annotation on a sample image.

[0117] Combining the above embodiments, Figure 9 shows an exemplary flowchart relating to an embodiment of the present disclosure. This flow includes two main paths: a contrast enhancement path and a deep learning texture probability estimation path, both of which can be processed in parallel. The contrast enhancement path calculates the local variance of each pixel to generate a variance-weighted grayscale histogram, and then obtains an initial contrast-enhanced image N1 by histogram equalization mapping (see the corresponding embodiments in Figures 2 and 3 above). The deep learning texture probability estimation path generates a low-resolution texture probability map P_low by block partitioning and embedding of the image, deformable Transformer coding, a fully connected layer, and a sigmoid activation function in that order. Subsequently, a full-resolution probability map P_full is generated by bilinear interpolation upsampling, and the full-resolution probability map P_full is mapped to a fusion coefficient map α_map using a fusion coefficient mapping function. Finally, a weighted fusion is performed on the initial contrast-enhanced image N1 output by the contrast enhancement path and the original image using the fusion coefficient map α_map to obtain the final contrast-enhanced image N2.

[0118] [Example device] Figure 10 is a schematic diagram of the structure of an image contrast enhancement device according to an exemplary embodiment of the present disclosure. This embodiment is applicable to electronic devices, and as shown in Figure 10, the image contrast enhancement device is An acquisition module 1001 for obtaining the source image to which contrast enhancement should be performed, A contrast enhancement module 1002 for obtaining an initial contrast-enhanced image by performing contrast enhancement on the original image, An estimation module 1003 for obtaining a fusion coefficient map by processing the original image using a pre-trained fusion coefficient estimation model, wherein the fusion coefficients in the fusion coefficient map are such that the fusion coefficients decrease the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image. The system includes a fusion module 1004 for fusing an initial contrast-enhanced image with the original image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

[0119] Referring to Figure 11, Figure 11 is a schematic diagram of the structure of an image contrast enhancement device according to another exemplary embodiment of the present disclosure.

[0120] In several selectable implementations, the enhancement module 1002 includes a first determination unit 10021 for determining the grayscale value variance of a target region in which any pixel in the original image is located, wherein the target region is an image region of a predetermined size with the pixel as the reference point; a statistical unit 10022 for obtaining a variance-weighted grayscale histogram by performing weighted statistics on the number of pixels of different grayscales in the original image based on the grayscale value variance corresponding to each pixel; and an enhancement unit 10023 for obtaining an initial contrast-enhanced image by performing equalization mapping on the original image based on the variance-weighted grayscale histogram.

[0121] In several selectable implementations, the statistical unit 10022 includes a first statistical subunit 100221 for statistically counting the number of pixels corresponding to various grayscale values ​​in the original image, a determinative subunit 100222 for determining the statistical count weight corresponding to each pixel in the original image based on a pre-set correspondence between grayscale value variance and statistical count weight, wherein the statistical count weight increases with increasing grayscale value variance, and a second statistical subunit 100223 for performing weighted statistics on the number of pixels corresponding to various grayscale values ​​in the original image based on the statistical count weight corresponding to each pixel in the original image to obtain a variance-weighted grayscale histogram.

[0122] In several selectable implementations, the fusion module 1004 includes a second determination unit 10041 for determining a first weight corresponding to each pixel in the initial contrast-enhanced image and a second weight corresponding to each pixel in the original image, based on the fusion coefficients included in the fusion coefficient map; a calculation unit 10042 for calculating a weighted sum of the grayscale values ​​of the corresponding pixels in the initial contrast-enhanced image and the original image based on the first and second weights; and a generation unit 10043 for generating a fused contrast-enhanced image based on the grayscale value of the weighted sum.

[0123] In several selectable implementations, the estimation module 1003 is an estimation unit 10031 for obtaining a texture probability estimation map by performing texture probability estimation on the original image using a pre-trained image texture estimation model, wherein the texture probability estimates included in the texture probability estimation map include an estimation unit 10031 that corresponds one-to-one with pixels included in the original image, and a transformation unit 10032 that performs transformations on the texture probability estimates included in the texture probability estimation map based on pre-set transformation relationships to obtain a fusion coefficient map.

[0124] In several selectable implementations, the conversion unit 10032 further uses a pre-configured mapping function to map the texture probability estimates included in the texture probability estimation map to fusion coefficients, thereby obtaining a fusion coefficient map.

[0125] In several selectable implementations, the mapping function is a monotonically increasing mapping function, and the transformation unit 10032 further uses the monotonically increasing mapping function to map any texture probability estimate included in the texture probability estimation map to a first fusion coefficient if the texture probability estimate indicates that the corresponding pixel is located in a texture region, and to a second fusion coefficient if the texture probability estimate indicates that the corresponding pixel is located in a flat region, where the first fusion coefficient is greater than the second fusion coefficient.

[0126] In several selectable implementations, the image texture estimation model includes an image block partitioning module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module. The estimation unit 10031 further performs block partitioning on the original image using the image block partitioning module to obtain a set of image blocks, and performs feature extraction on each image block in the set of image blocks using the feature extraction module to obtain an image token sequence, where each image token in the image token sequence corresponds to one image block. The encoder is used to encode the image token sequence to obtain an encoded data sequence, the low-resolution probability estimation module is used to predict the texture probability of each encoded data in the encoded data sequence to obtain a low-resolution texture probability map, and the upsampling module is used to upsample the low-resolution texture probability map to obtain a texture probability estimation map with the same resolution as the original image.

[0127] In several selectable implementations, the low-resolution probability estimation module includes a fully connected layer and an activation function layer. The estimation unit 10031 further uses the fully connected layer to map each encoded data in the encoded data sequence to a texture feature estimate, and uses the activation function layer to convert each of the obtained texture feature estimates into a low-resolution texture probability value to obtain a low-resolution texture probability map.

[0128] In several selectable implementations, the image texture estimation model is obtained by pre-training according to the following steps: acquiring a sample image; determining the annotation texture probability value corresponding to each pixel in the sample image; performing texture probability estimation on the sample image using the initial image texture estimation model to obtain a predicted texture probability estimate corresponding to each pixel in the sample image; determining the error between the predicted texture probability estimate and the annotation texture probability value using a pre-set loss function; adjusting the parameters of the initial texture estimation model based on the error; and, if the parameter-adjusted initial texture estimation model meets the pre-set training termination conditions, confirming the current initial texture estimation model as the trained image texture estimation model.

[0129] In several selectable implementations, the step of determining the annotation texture probability value corresponding to each pixel in the sample image includes the steps of: dividing the sample image into blocks to obtain a set of sample block regions; and determining the proportion of the texture region area within each sample block region in the set of sample block regions, and determining this proportion as the annotation texture probability value for each pixel included in the sample block region.

[0130] The exemplary embodiment of this apparatus and the exemplary method portion described above correspond to each other in their implementation forms, and the corresponding content between them can be mutually referenced, combined, and cited; however, such a description is omitted here. The beneficial technical effects corresponding to the exemplary embodiment of this apparatus can be found by referring to the corresponding beneficial technical effects of the exemplary method portion described above; however, such a description is omitted here.

[0131] [Example electronic device] Figure 12 is a structural diagram of an electronic device 1200 according to an embodiment of the present disclosure, which includes at least one processor 1201 and a memory 1202.

[0132] The processor 1201 can be a central processing unit (CPU) or another form of processing unit having data processing capability and / or instruction execution capability, and can control other components in the electronic device 1200 to perform a desired function.

[0133] Memory 1202 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. Computer-readable storage media may store one or more computer program instructions, and processor 1201 may execute one or more computer program instructions to realize the image contrast enhancement method and / or other desired functions of each embodiment of the present disclosure described above.

[0134] As an example, the electronic device 1200 may further include an input device 1203 and an output device 1204 connected to each other via a bus system and / or other form of connection mechanism (not shown).

[0135] The input device 1203 may include, for example, a keyboard, a mouse, or the like.

[0136] The output device 1204 can output various types of information to the outside. This output device 1204 may include, for example, a display, speaker, printer, communication network, and remote output devices connected thereto.

[0137] For simplicity, Figure 12 shows only some of the components relating to this disclosure in the electronic device 1200, omitting components such as buses and input / output interfaces. Otherwise, the electronic device 1200 may further include any other suitable components depending on the specific application.

[0138] [Examples of computer program products and computer-readable storage media] Embodiments of this disclosure provide a computer program product that includes computer program instructions in addition to the methods and apparatus described above. When the computer program instructions are executed by a processor, the processor is caused to perform the steps in the image contrast enhancement method of the various embodiments of this disclosure described in the “Exemplary Methods” portion above.

[0139] Computer program products can be created using one or any combination of programming languages ​​to produce program code for performing the operations of the embodiments of this disclosure, including object-oriented programming languages ​​such as Java® and C++, and conventional procedural programming languages ​​such as the C language or similar programming languages. The program code may run entirely on a user computing device, partially on a user device, run as a standalone software package, run partially on a user computing device and partially on a remote computing device, or run entirely on a remote computing device or a server.

[0140] Furthermore, embodiments of the present disclosure further provide a computer-readable storage medium in which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor is caused to perform the steps in the image contrast enhancement method of the various embodiments of the present disclosure described in the “Exemplary Methods” portion above.

[0141] Any combination of one or more readable media can be used as a computer-readable storage medium. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (non-exclusive list) of readable storage media include electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0142] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, merits, and effects mentioned in this disclosure are merely illustrative and not limiting, and these advantages, merits, and effects are not necessarily present in every example of this disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand effects and are not limiting, and the above details do not necessarily limit this disclosure to being realized by the above specific details.

[0143] Those skilled in the art can make various modifications and alterations to this disclosure without departing from the spirit and scope of the present application. Thus, if such modifications and alterations of the present application fall within the claims of this disclosure and the equivalent art thereto, this disclosure also includes such modifications and alterations.

Claims

1. An image contrast enhancement method in which each step is performed by an image contrast enhancement device, The steps include obtaining the source image to which contrast enhancement should be applied, The steps include: performing contrast enhancement on the original image to obtain an initial contrast-enhanced image, A step of processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map decrease the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image. An image contrast enhancement method characterized by comprising the step of fusing the initial contrast-enhanced image and the original image using the fusing coefficients included in the fusing coefficient map to obtain a fusing contrast-enhanced image.

2. The step of performing contrast enhancement on the original image to obtain an initial contrast-enhanced image is: A step of determining the grayscale value variance of any pixel contained in the original image in the target region where the pixel is located, wherein the target region is an image region of a predetermined size with the pixel as the reference point. The steps include: performing weighted statistics on the number of different grayscale pixels in the original image based on the grayscale value variance corresponding to each pixel to obtain a variance-weighted grayscale histogram; The image contrast enhancement method according to claim 1, comprising the step of performing equalization mapping on the original image based on the dispersion-weighted grayscale histogram to obtain an initial contrast-enhanced image.

3. The step of obtaining a variance-weighted grayscale histogram by performing weighted statistics on the number of different grayscale pixels in the original image based on the grayscale value variance corresponding to each of the aforementioned pixels is as follows: The steps include: statistically counting the number of pixels corresponding to various grayscale values ​​in the original image; A step of determining the statistical count weight corresponding to each pixel in the original image based on a pre-set correspondence between grayscale value variance and statistical count weight, wherein the statistical count weight increases as the grayscale value variance increases. The image contrast enhancement method according to claim 2, characterized by comprising the step of performing weighted statistics on the number of pixels corresponding to various grayscale values ​​in the original image based on the statistical count weights corresponding to each pixel in the original image, in order to obtain a variance-weighted grayscale histogram.

4. The step of obtaining a contrast-enhanced image after fusion by fusing the initial contrast-enhanced image and the original image using the fusing coefficients included in the fusing coefficient map is as follows: A step of determining a first weight corresponding to each pixel in the initial contrast-enhanced image and a second weight corresponding to each pixel in the original image, based on the fusion coefficients included in the fusion coefficient map, A step of calculating a weighted sum of the grayscale values ​​of corresponding pixels in the initial contrast-enhanced image and the original image based on the first weight and the second weight, The image contrast enhancement method according to claim 1, comprising the step of generating a contrast-enhanced image after fusion based on a weighted sum of grayscale values.

5. The step of processing the original image using the aforementioned pre-trained fusion coefficient estimation model to obtain a fusion coefficient map is as follows: A step of obtaining a texture probability estimation map by performing texture probability estimation on the original image using a pre-trained image texture estimation model, wherein the texture probability estimates included in the texture probability estimation map correspond one-to-one with the pixels included in the original image. The image contrast enhancement method according to claim 1, comprising the step of performing a transformation on each of the texture probability estimates included in the texture probability estimation map based on a pre-set transformation relationship to obtain a fusion coefficient map.

6. The step of obtaining a fusion coefficient map by performing transformations on the texture probability estimates included in the texture probability estimation map based on the pre-set transformation relationships is as follows: The image contrast enhancement method according to claim 5, characterized in that it includes the step of using a pre-set mapping function to map the texture probability estimates included in the texture probability estimation map to fusion coefficients, thereby obtaining a fusion coefficient map.

7. The aforementioned mapping function is a monotonically increasing mapping function, The step of using the aforementioned pre-configured mapping function to map the texture probability estimates included in the texture probability estimation map to fusion coefficients, thereby obtaining a fusion coefficient map, is as follows: The image contrast enhancement method according to claim 6, comprising the steps of: mapping any texture probability estimate included in the texture probability estimation map to a first fusion coefficient using the monotonically increasing mapping function if the texture probability estimate indicates that the corresponding pixel is located in a texture region; and mapping the texture probability estimate to a second fusion coefficient if the texture probability estimate indicates that the corresponding pixel is located in a flat region, wherein the first fusion coefficient is greater than the second fusion coefficient.

8. The image texture estimation model includes an image block segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module. The step of obtaining a texture probability estimation map by performing texture probability estimation on the original image using the aforementioned pre-trained image texture estimation model is as follows: The steps include: using the aforementioned image block division module to divide the original image into blocks to obtain a set of image blocks; A step of obtaining an image token sequence by performing feature extraction on each image block in the image block set using the feature extraction module, wherein each image token in the image token sequence corresponds to one image block, The steps include: using the encoder to encode the image token sequence to obtain an encoded data sequence; The steps include: using the low-resolution probability estimation module to perform texture probability prediction for each encoded data in the encoded data sequence to obtain a low-resolution texture probability map; The image contrast enhancement method according to claim 5, comprising the step of performing upsampling on the low-resolution texture probability map using the upsampling module to obtain a texture probability estimation map having the same resolution as the original image.

9. The low-resolution probability estimation module includes a fully connected layer and an activation function layer, The step of obtaining a low-resolution texture probability map by performing texture probability prediction for each encoded data in the encoded data sequence using the low-resolution probability estimation module is as follows: The steps include: using the fully connected layer to map each encoded data in the encoded data sequence to a texture feature estimate; The image contrast enhancement method according to claim 8, comprising the step of using the activation function layer to convert each obtained texture feature estimate into a low-resolution texture probability value to obtain the low-resolution texture probability map.

10. The aforementioned image texture estimation model was pre-configured as follows: Steps to obtain a sample image, The steps include determining the annotation texture probability value corresponding to each pixel in the aforementioned sample image, The steps include: performing texture probability estimation on the sample image using an initial image texture estimation model to obtain predicted texture probability estimates corresponding to each pixel in the sample image; A step of determining the error between the predicted texture probability estimate and the annotation texture probability value using a pre-configured loss function, The steps include adjusting the parameters of the initial image texture estimation model based on the aforementioned error, The image contrast enhancement method according to claim 5, characterized in that it is obtained by training according to the step of determining the current initial image texture estimation model as the trained image texture estimation model if the initial image texture estimation model after parameter adjustment meets the pre-set training termination conditions.

11. The step of determining the annotation texture probability value corresponding to each pixel in the aforementioned sample image is as follows: The steps include: performing block division on the aforementioned sample image to obtain a set of sample block regions, The image contrast enhancement method according to claim 10, comprising the step of determining the proportion of the texture area within each sample block region in the set of sample block regions, and determining this proportion as the annotation texture probability value of each pixel included in the sample block region.

12. A module for obtaining the source image to which contrast enhancement should be applied, A contrast enhancement module for obtaining an initial contrast-enhanced image by performing contrast enhancement on the aforementioned original image, An estimation module for obtaining a fusion coefficient map by processing the original image using a pre-trained fusion coefficient estimation model, wherein the fusion coefficients in the fusion coefficient map are such that the fusion coefficients decrease the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas in the original image. An image contrast enhancement apparatus comprising: a fusion module for fusing the initial contrast enhancement image and the original image using the fusion coefficients included in the fusion coefficient map to obtain a contrast enhancement image after fusion.

13. A computer-readable storage medium on which computer programs are stored, A computer-readable storage medium characterized in that, when the computer program is executed by the processor, the processor is instructed to perform the steps of the image contrast enhancement method described in any one of claims 1 to 11.

14. An electronic device comprising a processor and a memory for storing instructions that the processor can execute, An electronic device characterized in that the processor reads and executes the executable instructions from the memory to realize the image contrast enhancement method according to any one of claims 1 to 11.

Images