Image contrast enhancement method and device, readable storage medium and electronic equipment

Abstract

This disclosure provides an image contrast enhancement method, apparatus, readable storage medium, and electronic device. The method includes: acquiring an original image to be contrast enhanced; enhancing the contrast of the original image to obtain an initial contrast-enhanced image; processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map; and fusing the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image. This method achieves contrast enhancement based on the texture strength of different regions in the image. The fused contrast-enhanced image can effectively reflect the differences in content characteristics in the image, avoid noise amplification and over-enhancing in flat areas, and avoid insufficient contrast enhancement in textured areas, thus effectively highlighting texture details and improving the adaptability and targeting of image contrast enhancement.

Description

Technical Field

[0001] This disclosure relates to digital image processing technology, computer vision technology, and in particular to an image contrast enhancement method, apparatus, readable storage medium, and electronic device. Background Technology

[0002] Image contrast enhancement, also known as contrast boosting or grayscale stretching, is a fundamental and crucial image processing technique. Its core purpose is to use a specific transformation function to change the grayscale value of each pixel in the input image point by point, stretching or compressing the pixel brightness values ​​that were originally concentrated in a narrow dynamic range. This expands the brightness differences between different objects, making image details more apparent and the visual effect clearer.

[0003] Related image contrast enhancement methods include global histogram equalization. However, when processing images with complex content, these methods apply a uniform transformation to the entire image, ignoring the differences in content characteristics between different regions. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an image contrast enhancement method, apparatus, readable storage medium, and electronic device to solve the problems of noise amplification and over-enhancement in flat areas of an image, and insufficient contrast enhancement intensity in textured areas.

[0005] A first aspect of this disclosure provides an image contrast enhancement method, comprising: acquiring an original image to be contrast enhanced; performing contrast enhancement on the original image to obtain an initial contrast-enhanced image; processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map are used to reduce the contrast enhancement intensity of flat regions in the original image and increase the contrast enhancement intensity of texture regions in the original image; and fusing the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

[0006] A second aspect of this disclosure provides an image contrast enhancement apparatus, comprising: an acquisition module for acquiring an original image to be contrast enhanced; an enhancement module for enhancing the contrast of the original image to obtain an initial contrast-enhanced image; an estimation module for processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map are used to reduce the contrast enhancement intensity of flat regions in the original image and increase the contrast enhancement intensity of texture regions in the original image; and a fusion module for fusing the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map to obtain a fused contrast-enhanced image.

[0007] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed, implements the image contrast enhancement method described above.

[0008] A fourth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to read executable instructions from the memory and execute the instructions to implement the image contrast enhancement method described above.

[0009] The image contrast enhancement method, apparatus, readable storage medium, and electronic device provided in this disclosure enhance the contrast of an original image to obtain an initial contrast-enhanced image. A pre-trained fusion coefficient estimation model is then used to process the original image to obtain a fusion coefficient map. The fusion coefficients are used to reduce the contrast enhancement intensity of flat regions in the original image and increase the contrast enhancement intensity of textured regions. Finally, the initial contrast-enhanced image and the original image are fused using the fusion coefficient map to obtain a fused contrast-enhanced image. This application embodiment achieves targeted contrast enhancement of different regions based on their texture intensity. The fused contrast-enhanced image effectively reflects the differences in content characteristics within the image, avoids noise amplification and over-enhancing in flat regions, and avoids insufficient contrast enhancement in textured regions, effectively highlighting texture details and thus improving the adaptability and targeting of image contrast enhancement. Attached Figure Description

[0010] Figure 1 This is a schematic flowchart of an image contrast enhancement method provided in an exemplary embodiment of this disclosure; Figure 2 This is a schematic flowchart of an image contrast enhancement method provided in another exemplary embodiment of this disclosure; Figure 3 This is a schematic flowchart of an image contrast enhancement method provided in yet another exemplary embodiment of this disclosure; Figure 4 This is a schematic flowchart of an image contrast enhancement method provided in yet another exemplary embodiment of this disclosure; Figure 5 This is a schematic flowchart of an image contrast enhancement method provided in yet another exemplary embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an image texture estimation model provided in an exemplary embodiment of this disclosure; Figure 7This is a schematic flowchart of an image contrast enhancement method provided in yet another exemplary embodiment of this disclosure; Figure 8 This is a schematic diagram of the training process of an image texture estimation model provided in an exemplary embodiment of this disclosure; Figure 9 This is a schematic flowchart of an image contrast enhancement method provided in yet another exemplary embodiment of this disclosure; Figure 10 This is a structural diagram of an image noise level estimation apparatus provided in an exemplary embodiment of the present disclosure; Figure 11 This is a structural diagram of an image noise level estimation apparatus provided in another exemplary embodiment of this disclosure; Figure 12 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0011] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0012] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0013] Application Overview Methods such as global histogram equalization for image contrast enhancement have inherent drawbacks when dealing with complex images: they apply a uniform transformation to the entire image, ignoring the differences in content characteristics across different regions. This leads to flat areas (such as the sky and walls) being prone to noise amplification and over-enhancement after enhancement, resulting in unnatural blocky or grainy textures; while areas with detailed textures (such as object edges and complex patterns) may suffer from insufficient enhancement, failing to effectively highlight details.

[0014] To address the aforementioned issues, this application provides a method for adaptively enhancing contrast based on image content, thereby achieving differentiated and intelligent enhancement of different image regions and improving the adaptability and relevance of image contrast enhancement.

[0015] Exemplary methods Figure 1 This is a schematic flowchart of an image contrast enhancement method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to various types of electronic devices, such as... Figure 1 As shown, it includes the following steps: Step 101: Obtain the original image to be contrast-enhanced.

[0016] The original image can be taken in any scene, such as an image taken by a camera on a vehicle or an image taken by a user with a mobile phone. Optionally, the original image can be a grayscale image or a color image.

[0017] Step 102: Enhance the contrast of the original image to obtain an initial contrast-enhanced image.

[0018] The electronic device implementing this method can perform this step using various relevant contrast enhancement methods. Optionally, at least one of the following algorithms can be used for contrast enhancement: histogram equalization algorithm, linear contrast stretching algorithm, deep learning-based image contrast enhancement algorithm, etc. The initial contrast-enhanced image can be a grayscale image or a color image restored from a grayscale image.

[0019] Step 103: Using a pre-trained fusion coefficient estimation model, process the original image to obtain a fusion coefficient map.

[0020] In the fusion coefficient map, the fusion coefficients are used to reduce the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of textured areas. The size of the fusion coefficient map can be the same as the size of the original image, meaning that the fusion coefficients in the fusion coefficient map correspond one-to-one with the pixels in the original image. The fusion coefficients in the fusion coefficient map represent the proportion of grayscale values ​​obtained from the initial contrast-enhanced image when generating the fused contrast-enhanced image. That is, the larger the fusion coefficient, the greater the proportion of grayscale values ​​from the initial contrast-enhanced image in the fused contrast-enhanced image; the smaller the fusion coefficient, the greater the proportion of grayscale values ​​from the original image in the fused contrast-enhanced image.

[0021] Optionally, the blending coefficient can be data that reflects the texture (or flatness) of the region where the pixel is located. Generally, a higher blending coefficient indicates that the corresponding pixel is more likely to be in a textured region; a lower blending coefficient indicates that the corresponding pixel is more likely to be in a flat region.

[0022] The aforementioned fusion coefficient estimation model is used to represent the correspondence between the original image and the fusion coefficient map. This fusion coefficient estimation model can be constructed in various ways, such as setting a conversion formula for grayscale values ​​or training a neural network model using machine learning methods.

[0023] Optionally, when training the fusion coefficient estimation model using machine learning methods, a large number of sample images can be collected in advance, and the fusion coefficient can be labeled for each pixel in each sample image. Then, the parameters of the initial model are iteratively adjusted to gradually reduce the error between the model's output fusion coefficient and the labeled fusion coefficient. When the error converges, the current initial model is used as the trained fusion coefficient estimation model.

[0024] Optionally, an image texture estimation model can be used to estimate the texture probability of the original image, obtain a texture probability estimation map, and then the texture probability estimates included in the texture probability estimation map can be transformed to obtain a fusion coefficient map.

[0025] Step 104: Using the fusion coefficients included in the fusion coefficient map, the initial contrast-enhanced image and the original image are fused to obtain the fused contrast-enhanced image.

[0026] Since there is a one-to-one correspondence between the fusion coefficients in the fusion coefficient map and the pixels in the initial contrast-enhanced image, the grayscale value of any pixel in the initial contrast-enhanced image and the original image can be calculated based on the corresponding fusion coefficient to obtain the fused grayscale image. For example, the mean of the grayscale values ​​of corresponding pixels in the initial contrast-enhanced image and the original image can be multiplied by the corresponding fusion coefficient to obtain the fused grayscale image.

[0027] Depending on the actual needs, the fused grayscale image can be used as the fused contrast-enhanced image, or the fused grayscale image can be restored to a color image and used as the fused contrast-enhanced image.

[0028] The image contrast enhancement method provided in this disclosure enhances the contrast of an original image to obtain an initial contrast-enhanced image. A pre-trained fusion coefficient estimation model is then used to process the original image to obtain a fusion coefficient map. The fusion coefficients reduce the contrast enhancement intensity in flat areas of the original image and increase the contrast enhancement intensity in textured areas. Finally, the initial contrast-enhanced image and the original image are fused using the fusion coefficient map to obtain a fused contrast-enhanced image. This application embodiment achieves targeted contrast enhancement of different regions based on their texture intensity. The fused contrast-enhanced image effectively reflects the differences in content characteristics within the image, avoids noise amplification and over-enhancing in flat areas, and avoids insufficient contrast enhancement in textured areas, effectively highlighting texture details and thus improving the adaptability and targeting of image contrast enhancement.

[0029] In some alternative implementations, such as Figure 2 As shown, step 102 includes: Step 1021: For any pixel in the original image, determine the variance of the grayscale value of the target region where the pixel is located.

[0030] The target area is an image region of a preset size, with the pixel as the reference point. The position of the reference point for each image region can be arbitrarily specified; typically, the reference point can be the center point of the image region. The preset size can be set arbitrarily, for example, an image block of 3×3, 5×5, etc.

[0031] The formula for calculating the variance of grayscale values ​​is as follows: (1) in, The variance of grayscale values. Let be the grayscale value of the i-th pixel within the target region. is the average gray value of each pixel, and N is the number of pixels in the target area.

[0032] It should be understood that the above-mentioned preset size is the maximum size of each image region. In the boundary and apex regions of the original image, since the number of pixels around the reference point does not meet the number of pixels specified by the above-mentioned preset size, the size of the image region corresponding to the reference point can be smaller than the preset size.

[0033] Step 1022: Based on the variance of the gray value corresponding to each pixel, the number of pixels with different gray values ​​in the original image is weighted and statistically analyzed to obtain a variance-weighted gray-level histogram.

[0034] A grayscale histogram is a statistical representation of the frequency of occurrence of all pixels in an image, based on their grayscale values. In other words, a grayscale histogram is a function of grayscale levels, representing the number of pixels in an image that possess a specific grayscale level.

[0035] Conventional histogram equalization methods count the number of pixels corresponding to each grayscale value in the image when calculating the grayscale histogram. However, this embodiment does not count each pixel as 1. Instead, it determines a statistical weight based on the variance of the grayscale value corresponding to that pixel, and then performs weighted statistics according to these weights to obtain a variance-weighted grayscale histogram. Generally, the relationship between weight and grayscale variance is: the larger the grayscale variance, the larger the statistical weight; the smaller the grayscale variance, the smaller the statistical weight.

[0036] Step 1023: Based on the variance-weighted grayscale histogram, perform equalization mapping on the original image to obtain the initial contrast-enhanced image.

[0037] Histogram-based equalization mapping is an image processing technique that enhances image contrast by altering the gray-level distribution. The main steps include: calculating the gray-level histogram, the total number of pixels, the gray-level distribution frequency, and the cumulative distribution frequency; then, normalizing and mapping each pixel to a new gray level, ultimately obtaining an image with enhanced contrast.

[0038] In this embodiment, a variance-weighted grayscale histogram is used when performing histogram equalization mapping. This histogram reflects the weighted grayscale statistical results, which in turn reflect the grayscale differences in the region where each pixel is located. In other words, it reflects the strength of the texture in the region, allowing the initial contrast enhancement image to perform different degrees of contrast enhancement on textured and flat regions, thereby improving the adaptability of the initial contrast enhancement image to the texture features in the original image.

[0039] In some alternative implementations, such as Figure 3 As shown, step 1022 includes: Step 10221: Count the number of pixels corresponding to each gray value in the original image.

[0040] This involves counting pixels with the same grayscale value in the original image to determine the number of pixels corresponding to each grayscale value.

[0041] Step 10222: Based on the preset correspondence between gray value variance and statistical frequency weight, determine the statistical frequency weight corresponding to each pixel in the original image.

[0042] The statistical frequency weight increases with the increase of the grayscale value variance. The correspondence between grayscale value variance and statistical frequency weight can be represented by tables, formulas, etc. The principle behind this correspondence is: the larger the grayscale value variance, the larger the statistical frequency weight; the smaller the grayscale value variance, the smaller the statistical frequency weight. For example, a mapping curve can be set up, and based on this curve, the statistical frequency weight corresponding to the grayscale value variance can be determined.

[0043] Step 10223: Based on the statistical frequency weights corresponding to each pixel in the original image, the number of pixels corresponding to each gray value in the original image is weighted and statistically analyzed to obtain a variance-weighted gray-level histogram.

[0044] Specifically, when counting the number of pixels corresponding to each grayscale value, for any given pixel, the statistical frequency weight corresponding to that pixel can be calculated (e.g., multiplied, added, etc.) with a preset statistical base (e.g., 1). The calculated result is used as the actual statistical increment for that pixel, and this statistical increment is accumulated into the number of pixels corresponding to that pixel's grayscale value. Optionally, the above calculation results can be directly accumulated, or the calculation results can be processed again (e.g., normalized) before accumulation.

[0045] This embodiment sets a correspondence between the variance of gray values ​​and the weight of statistical frequency, so that the variance-weighted gray histogram can reflect the degree of difference in gray values ​​in the region where each pixel is located, that is, it can reflect the texture of the region, thereby improving the matching degree between the variance-weighted gray histogram and the real scene, and thus helping to improve the adaptability of image contrast enhancement.

[0046] In some alternative implementations, such as Figure 4 As shown, step 104 includes: Step 1041: Based on the fusion coefficients included in the fusion coefficient map, determine the first weight corresponding to each pixel in the initial contrast-enhanced image and the second weight corresponding to each pixel in the original image.

[0047] Typically, the sum of the first and second weights can be 1. Optionally, for any pixel in the initial contrast-enhanced image, the fusion coefficient corresponding to that pixel can be used as the first weight, and then the corresponding second weight can be calculated. For example, if the fusion coefficient of a pixel is α, then α can be used as the first weight w, resulting in the first weight w. Correspondingly, the second weight is 1-w.

[0048] Step 1042: Based on the first weight and the second weight, perform a weighted summation of the gray values ​​of corresponding pixels in the initial contrast-enhanced image and the original image.

[0049] Specifically, the grayscale value of any pixel after fusion can be calculated using the following formula: (2) Where (x, y) are the coordinates of a certain pixel. This represents the grayscale value of a pixel in the merged grayscale image, where w is the first weight and 1-w is the second weight. The gray value corresponding to this pixel in the initial contrast-enhanced image. This is the grayscale value corresponding to that pixel in the original image.

[0050] As can be seen from equation (2), for pixels with high fusion coefficients, the higher the probability that they are in texture regions, the more gray values ​​corresponding to them are retained in the initial contrast-enhanced image, and the fusion result tends to perform high-intensity contrast enhancement on texture regions, reflecting richer texture details; for pixels with low fusion coefficients, the higher the probability that they are in flat regions, the more gray values ​​corresponding to them are retained in the original image, and the fusion result tends to retain the appearance of the original image.

[0051] Step 1043: Generate a fused contrast-enhanced image based on the weighted sum of gray values.

[0052] Specifically, the weighted summation of the gray values ​​of each pixel can be combined to form a fused grayscale image. Depending on the actual needs, this fused grayscale image can be used as a fused contrast-enhanced image, or the fused grayscale image can be restored to a color image and used as a fused contrast-enhanced image.

[0053] This embodiment uses a fusion coefficient to determine the weights of corresponding pixels in the initial contrast-enhanced image and the original image, and uses the weights to perform grayscale fusion. This allows the fused contrast-enhanced image to reflect high-intensity contrast enhancement in textured areas and to retain the appearance of the original image in flat areas, so that the contrast-enhanced image can more realistically reflect the appearance features of the real scene.

[0054] In some alternative implementations, such as Figure 5 As shown, step 103 above includes: Step 1031: Using a pre-trained image texture estimation model, perform texture probability estimation on the original image to obtain a texture probability estimation map.

[0055] The texture probability estimation map contains texture probability estimates that correspond one-to-one with the pixels in the original image.

[0056] The texture probability estimation map described above includes texture probability estimates that correspond one-to-one with the pixels in the original image. Each texture probability estimate represents the probability that a corresponding pixel is located within a textured region. A larger texture probability estimate indicates a higher likelihood that the corresponding pixel is located within a textured region, while a smaller texture probability estimate indicates a higher likelihood that the corresponding pixel is located within a flat region.

[0057] The image texture estimation model described above can be pre-trained using machine learning methods. For example, a large number of sample images can be collected beforehand, and a texture probability value can be labeled for each pixel in each sample image. Then, the parameters of the initial model are iteratively adjusted to gradually reduce the error between the predicted texture probability estimate map output by the model and the labeled texture probability estimate map. When the error converges, the current initial model is used as the trained image texture estimation model.

[0058] Step 1032: Based on the preset transformation relationship, the texture probability estimation values ​​included in the texture probability estimation map are transformed to obtain the fusion coefficient map.

[0059] The aforementioned transformation relationship can be preset. This transformation relationship can be represented by calculation formulas, correspondence tables, etc. By utilizing this transformation relationship, each texture probability estimate can be converted into a fusion coefficient. The resulting fusion coefficients are then combined into a fusion coefficient map.

[0060] Generally, there is a positive relationship between the texture probability estimate and the fusion coefficient. That is, the larger the texture probability estimate, the greater the probability that the corresponding pixel belongs to the texture region, and the larger the corresponding fusion coefficient. A larger fusion coefficient can obtain gray values ​​from the initial contrast-enhanced image to a greater extent, thus enhancing the contrast of texture details to a greater extent. Conversely, the smaller the texture probability estimate, the smaller the probability that the corresponding pixel belongs to the texture region, and the smaller the corresponding fusion coefficient. A smaller fusion coefficient can obtain gray values ​​from the original image to a greater extent, thus preserving the original appearance features of flat areas to a greater extent.

[0061] This embodiment uses an image texture estimation model to estimate the texture probability of the original image, and then transforms the estimated texture probability values ​​to obtain a fusion coefficient map. This enables the targeted determination of the corresponding fusion coefficient based on the texture probability of each region in the image, which helps to improve the accuracy of generating fusion coefficients.

[0062] In some alternative implementations, step 1032 may include: Using a preset mapping function, the texture probability estimates included in the texture probability estimation map are mapped to fusion coefficients to obtain a fusion coefficient map.

[0063] This mapping function reflects the relationship between the texture probability estimate and the fusion coefficient. The form of the mapping function can be set arbitrarily; typically, the goal is to make the fusion coefficient of textured regions larger and the fusion coefficient of flat regions smaller.

[0064] This embodiment sets a mapping function to specifically map any texture probability estimate to a fusion coefficient, thereby enabling the fusion coefficient corresponding to each pixel to more accurately reflect the texture level of the region where the pixel is located.

[0065] In some alternative implementations, the mapping function is a monotonically increasing mapping function.

[0066] The fusion coefficient map can be calculated by following these steps: For any texture probability estimate included in the texture probability estimation map, a monotonically increasing mapping function is used to map the texture probability estimate as a first fusion coefficient when the texture probability estimate indicates that the corresponding pixel is located in a textured region; and to map the texture probability estimate as a second fusion coefficient when the texture probability estimate indicates that the corresponding pixel is located in a flat region.

[0067] The first fusion coefficient is greater than the second fusion coefficient.

[0068] The form of the monotonically increasing mapping function can be arbitrarily set. As an example, the monotonically increasing mapping function can be α = a + β * p, where a and β are adjustable hyperparameters greater than 0, and p is the texture probability estimate. The significance of this function is: when p for a certain pixel is close to 1, it indicates that the pixel is located in a textured region, and the corresponding α is close to a + β, which increases the contrast enhancement intensity; when p is close to 0, it indicates that the pixel is located in a flat region, and the corresponding α is close to a, which decreases the contrast enhancement intensity.

[0069] This embodiment, by setting a monotonically increasing mapping function, allows textured regions and flat regions to correspond to different fusion coefficients, resulting in more targeted and accurate image contrast enhancement.

[0070] In some alternative implementations, such as Figure 6 As shown, the image texture estimation model includes an image segmentation module 601, a feature extraction module 602, an encoder 603, a low-resolution probability estimation module 604, and an upsampling module 605.

[0071] like Figure 7 As shown, step 1031 includes: Step 10311: Divide the original image into blocks using the image segmentation module to obtain an image block set.

[0072] This embodiment does not limit the method of dividing the image into blocks. For example, a convolutional layer with a kernel size and stride of P (e.g., 16) can be used as an image block dividing module to perform convolution processing on the input original image, thereby dividing the image into multiple non-overlapping image blocks of size P × P.

[0073] Step 10312: Using the feature extraction module, feature extraction is performed on each image block in the image block set to obtain the image token sequence.

[0074] Each image token in the image token sequence corresponds to an image block.

[0075] This feature extraction module can be implemented using a learnable linear projection layer. It flattens each image block into a vector and maps this vector to a fixed-dimensional embedding space through the linear projection layer (typically a fully connected layer). For example, each image token in the image token sequence is a 1×C feature vector, where C is the feature dimension. The image token sequence can form a feature map of size (H / P)×(W / P)×C, where H and W are the height and width of the original image.

[0076] Step 10313: Encode the image token sequence using an encoder to obtain an encoded data sequence.

[0077] Each piece of encoded data in the encoded data sequence is obtained by an encoder encoding the corresponding image token. Optionally, this encoder can be a deformable Transformer encoder. The token sequence is input into a deformable Transformer encoder consisting of multiple (e.g., 2-4) stacked encoded blocks. The encoder's deformable self-attention mechanism allows the model to dynamically focus on and determine other blocks most relevant to the current image block's content, thereby more effectively capturing global contextual information and thus helping to accurately determine whether an image block belongs to a textured region or a flat region.

[0078] Step 10314: Using the low-resolution probability estimation module, perform texture probability prediction on each piece of coded data in the coded data sequence to obtain a low-resolution texture probability map.

[0079] Specifically, each encoded data point in the encoded data sequence can be input into the low-resolution probability estimation module. The low-resolution probability estimation module then performs texture probability prediction on each encoded data point, obtaining a texture probability value corresponding to each encoded data point. These texture probability values ​​form a low-resolution texture probability map. The size of the low-resolution texture probability map is (H / P) × (W / P). The value of each point on this map is between 0 and 1, representing the probability that each image block contained in the original image belongs to a texture region.

[0080] Step 10315: Using the upsampling module, the low-resolution texture probability map is upsampled to obtain a texture probability estimation map with the same resolution as the original image.

[0081] The upsampling module can be implemented based on relevant interpolation algorithms. For example, bilinear interpolation can be used to upsample the low-resolution texture probability map P_low to the full resolution H×W of the original image, resulting in a full-resolution texture probability estimation map P_full. Bilinear interpolation produces a smooth transition, effectively avoiding blockage artifacts in the final fusion coefficient map.

[0082] This embodiment constructs an image texture estimation model by setting up an image segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module. This enables texture probability estimation based on a deep learning model, effectively improving the accuracy of texture probability estimation.

[0083] In some alternative implementations, such as Figure 6 As shown, the low-resolution probability estimation module 604 includes a fully connected layer 6041 and an activation function layer 6042.

[0084] Step 10314 can be performed as follows: First, a fully connected layer is used to map each piece of encoded data in the encoded data sequence to a texture feature estimate.

[0085] Then, by using an activation function layer, the obtained texture feature estimates are converted into low-resolution texture probability values ​​to obtain a low-resolution texture probability map.

[0086] The fully connected layer described above integrates the features extracted from the previous layers of the image texture estimation model and outputs data for classification. The activation function layer typically uses the sigmoid activation function, but other activation functions such as tanh are also possible. The activation function layer performs binary classification on the texture feature estimates, outputting the probability that the corresponding image region is a texture region.

[0087] This embodiment improves the accuracy of generating texture probability estimation maps by setting a fully connected layer and an activation function layer in the low-resolution probability estimation module to accurately predict the texture probability of each encoded data.

[0088] In some alternative implementations, such as Figure 8 As shown, the image texture estimation model is pre-trained according to the following steps: Step 801: Obtain the sample image.

[0089] The sample image can be an image pre-captured of a certain type of scene, and the image contrast enhancement method provided in this embodiment falls into this category. Optionally, the sample image can be a grayscale image or a color image.

[0090] Step 802: Determine the labeled texture probability values ​​corresponding to the pixels in the sample image.

[0091] In this model, the texture probability value is a label between 0 and 1, which is manually assigned to each pixel beforehand. Typically, to improve annotation efficiency, the sample image is divided into P×P blocks, and the annotator assigns a label between 0 and 1 to each block. The final label structure is a (H / P)×(W / P) matrix, where each value represents the probability that the corresponding location belongs to a texture region.

[0092] Step 803: Using the initial image texture estimation model, perform texture probability estimation on the sample image to obtain the predicted texture probability estimates for each pixel in the sample image.

[0093] The structure of the initial image texture estimation model can be constructed using various related neural network models, such as... Figure 6 The structure shown is shown. The initial image texture estimation model can be implemented as follows: Figure 7 The process shown is used to estimate the texture probability of the sample image.

[0094] Step 804: Using a preset loss function, determine the error between the predicted texture probability estimate and the labeled texture probability.

[0095] The loss function can be any of the relevant types, such as the L1 loss function, where the loss value is the mean absolute error between the texture probability value predicted by the network and the true label. This loss function is insensitive to outliers and provides more stable training.

[0096] Step 805: Adjust the parameters of the initial texture estimation model based on the error.

[0097] Typically, gradient descent and backpropagation methods can be used to iteratively adjust the parameters of the initial texture estimation model until the model meets the training termination conditions.

[0098] Step 806: If the initial texture estimation model after parameter adjustment meets the preset training termination condition, the current initial texture estimation model is determined as the trained image texture estimation model.

[0099] Optionally, the above training termination conditions may include at least one of the following: loss value convergence, training iterations reaching a preset number of iterations, training duration reaching a preset duration, etc.

[0100] The image texture estimation model training method provided in this embodiment can adapt the model training process to actual application scenarios, thereby improving the accuracy of the model in texture probability estimation.

[0101] In some alternative implementations, step 802 above can be performed as follows: First, the sample image is divided into blocks to obtain a set of sample block regions.

[0102] Then, for each sample block region in the sample block region set, the proportion of the texture region area in the sample block region to the sample block region is determined, and the proportion is determined as the labeled texture probability value of each pixel included in the sample block region.

[0103] The texture region in any sample block region can be defined manually or automatically by other texture recognition models.

[0104] This embodiment implements the use of the proportion of texture region area as the texture probability value when annotating sample images, which can simplify the annotation process and improve annotation efficiency and model training efficiency.

[0105] In summary, the above embodiments, Figure 9 An exemplary flowchart provided in an embodiment of this disclosure is shown. The flowchart includes two main paths: a contrast enhancement branch and a deep learning texture probability estimation branch, which can be processed in parallel. The contrast enhancement branch generates a variance-weighted grayscale histogram by calculating the local variance of each pixel, and then obtains an initial contrast-enhanced image through histogram equalization mapping (refer to the above). Figure 2 , Figure 3 (Corresponding embodiment). The deep learning texture probability estimation branch sequentially generates a low-resolution texture probability map P_low through image segmentation and embedding, deformable Transformer encoding, a fully connected layer, and a Sigmoid activation function. Then, it generates a full-resolution probability map P_full through bilinear interpolation upsampling, and maps the full-resolution probability map P_full to a fusion coefficient map α_map using a fusion coefficient mapping function. Finally, using the fusion coefficient map α_map, the initial contrast-enhanced image N1 output by the contrast enhancement branch and the original image are weighted and fused to obtain the final contrast-enhanced image.

[0106] Exemplary device Figure 10 This is a schematic diagram of the structure of an image contrast enhancement device provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 10 As shown, the image contrast enhancement device includes: The acquisition module 1001 is used to acquire the original image to be contrast-enhanced; Enhancement module 1002 is used to enhance the contrast of the original image to obtain an initial contrast-enhanced image; The estimation module 1003 is used to process the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map. The fusion coefficients in the fusion coefficient map are used to reduce the contrast enhancement intensity of flat areas in the original image and increase the contrast enhancement intensity of texture areas in the original image. The fusion module 1004 is used to fuse the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map, so as to obtain the fused contrast-enhanced image.

[0107] Reference Figure 11 , Figure 11 This is a schematic diagram of the structure of an image contrast enhancement device provided in another exemplary embodiment of this disclosure.

[0108] In some optional implementations, the enhancement module 1002 includes: a first determining unit 10021, used to determine the gray value variance of a target region where any pixel in the original image is located, wherein the target region is an image region of a preset size with the pixel as the reference point; a statistical unit 10022, used to perform weighted statistics on the number of pixels of different gray levels in the original image based on the gray value variance corresponding to each pixel, to obtain a variance-weighted gray-level histogram; and an enhancement unit 10023, used to perform equalization mapping on the original image based on the variance-weighted gray-level histogram, to obtain an initial contrast-enhanced image.

[0109] In some optional implementations, the statistical unit 10022 includes: a first statistical subunit 100221, used to count the number of pixels corresponding to each gray value of the original image; a determining subunit 100222, used to determine the statistical weight corresponding to each pixel in the original image based on a preset correspondence between the gray value variance and the statistical frequency weight, wherein the statistical frequency weight increases as the gray value variance increases; and a second statistical subunit 100223, used to perform weighted statistics on the number of pixels corresponding to each gray value of the original image based on the statistical frequency weight corresponding to each pixel in the original image, to obtain a variance-weighted gray-level histogram.

[0110] In some optional implementations, the fusion module 1004 includes: a second determining unit 10041, used to determine, based on the fusion coefficients included in the fusion coefficient map, the first weight corresponding to each pixel in the initial contrast-enhanced image and the second weight corresponding to each pixel in the original image; a calculation unit 10042, used to perform a weighted summation of the gray values ​​of corresponding pixels in the initial contrast-enhanced image and the original image based on the first weight and the second weight; and a generation unit 10043, used to generate the fused contrast-enhanced image based on the weighted summation of gray values.

[0111] In some optional implementations, the estimation module 1003 includes: an estimation unit 10031, used to perform texture probability estimation on the original image using a pre-trained image texture estimation model to obtain a texture probability estimation map, wherein the texture probability estimation map includes texture probability estimation values ​​that correspond one-to-one with the pixels included in the original image; and a conversion unit 10032, used to convert the texture probability estimation values ​​included in the texture probability estimation map according to a preset conversion relationship to obtain a fusion coefficient map.

[0112] In some optional implementations, the conversion unit 10032 is further used to: map the texture probability estimates included in the texture probability estimation map to fusion coefficients using a preset mapping function, thereby obtaining a fusion coefficient map.

[0113] In some optional implementations, the mapping function is a monotonically increasing mapping function; the conversion unit 10032 is further configured to: for any texture probability estimate included in the texture probability estimation map, using the monotonically increasing mapping function, when the texture probability estimate indicates that the corresponding pixel is located in a texture region, map the texture probability estimate to a first fusion coefficient; when the texture probability estimate indicates that the corresponding pixel is located in a flat region, map the texture probability estimate to a second fusion coefficient, wherein the first fusion coefficient is greater than the second fusion coefficient.

[0114] In some optional implementations, the image texture estimation model includes an image segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module; the estimation unit 10031 is further configured to: segment the original image into blocks using the image segmentation module to obtain an image block set; extract features from each image block in the image block set using the feature extraction module to obtain an image token sequence, wherein each image token in the image token sequence corresponds to an image block; encode the image token sequence using the encoder to obtain an encoded data sequence; predict the texture probability for each encoded data in the encoded data sequence using the low-resolution probability estimation module to obtain a low-resolution texture probability map; and upsample the low-resolution texture probability map using the upsampling module to obtain a texture probability estimation map with the same resolution as the original image.

[0115] In some optional implementations, the low-resolution probability estimation module includes a fully connected layer and an activation function layer; the estimation unit 10031 is further used to: use the fully connected layer to map each encoded data in the encoded data sequence to a texture feature estimate; and use the activation function layer to convert each texture feature estimate into a low-resolution texture probability value to obtain a low-resolution texture probability map.

[0116] In some optional implementations, the image texture estimation model is pre-trained according to the following steps: acquiring sample images; determining the labeled texture probability values ​​corresponding to the pixels in the sample images; using the initial image texture estimation model to perform texture probability estimation on the sample images, obtaining the predicted texture probability estimates corresponding to the pixels in the sample images; using a preset loss function to determine the error between the predicted texture probability estimates and the labeled texture probability values; adjusting the parameters of the initial texture estimation model based on the error; and if the initial texture estimation model after parameter adjustment meets the preset training termination conditions, the current initial texture estimation model is determined as the trained image texture estimation model.

[0117] In some optional implementations, determining the labeled texture probability value corresponding to each pixel in the sample image includes: dividing the sample image into blocks to obtain a set of sample block regions; for each sample block region in the set of sample block regions, determining the proportion of the texture region area in the sample block region to the sample block region, and determining the proportion as the labeled texture probability value of each pixel included in the sample block region.

[0118] The exemplary embodiments of this device correspond to the exemplary method section described above in terms of implementation. The corresponding content between the two can be referenced, combined, and cited, and will not be repeated here. The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section described above, and will not be repeated here.

[0119] Exemplary electronic devices Figure 12 The present disclosure provides a structural diagram of an electronic device 1200, which includes at least one processor 1201 and a memory 1202.

[0120] The processor 1201 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1200 to perform desired functions.

[0121] The memory 1202 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1201 may execute one or more computer program instructions to implement the image contrast enhancement methods and / or other desired functions of the various embodiments of this disclosure described above.

[0122] In one example, the electronic device 1200 may also include an input device 1203 and an output device 1204, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0123] The input device 1203 may also include, for example, a keyboard, a mouse, etc.

[0124] The output device 1204 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0125] Of course, for the sake of simplicity, Figure 12 Only some of the components of the electronic device 1200 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1200 may include any other suitable components depending on the specific application.

[0126] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image contrast enhancement methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0127] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0128] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the image contrast enhancement methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0129] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0131] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An image contrast enhancement method, comprising: Obtain the original image to be contrast-enhanced; The original image is contrast-enhanced to obtain an initial contrast-enhanced image; The original image is processed using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map. The fusion coefficients in the fusion coefficient map are used to reduce the contrast enhancement intensity of flat regions in the original image and increase the contrast enhancement intensity of texture regions in the original image. Using the fusion coefficients included in the fusion coefficient map, the initial contrast-enhanced image and the original image are fused to obtain the fused contrast-enhanced image.

2. The method according to claim 1, wherein, The step of enhancing the contrast of the original image to obtain an initial contrast-enhanced image includes: For any pixel in the original image, determine the gray value variance of the target region where the pixel is located, wherein the target region is an image region of a preset size with the pixel as the reference point; Based on the variance of the gray value corresponding to each pixel, the number of pixels with different gray values ​​in the original image is weighted and statistically analyzed to obtain a variance-weighted gray-level histogram. Based on the variance-weighted grayscale histogram, the original image is subjected to equalization mapping to obtain an initial contrast-enhanced image.

3. The method according to claim 2, wherein, The step of weighting the number of pixels with different gray levels in the original image based on the variance of the gray value corresponding to each pixel to obtain a variance-weighted gray-level histogram includes: The number of pixels corresponding to each grayscale value of the original image is counted. Based on the preset correspondence between gray value variance and statistical frequency weight, the statistical frequency weight corresponding to each pixel in the original image is determined, wherein the statistical frequency weight increases as the gray value variance increases. Based on the statistical frequency weights corresponding to each pixel in the original image, the number of pixels corresponding to each gray value in the original image is weighted and statistically analyzed to obtain a variance-weighted gray-level histogram.

4. The method according to claim 1, wherein, The step of fusing the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map to obtain the fused contrast-enhanced image includes: Based on the fusion coefficients included in the fusion coefficient map, a first weight corresponding to each pixel in the initial contrast-enhanced image and a second weight corresponding to each pixel in the original image are determined. Based on the first weight and the second weight, the gray values ​​of corresponding pixels in the initial contrast-enhanced image and the original image are weighted and summed. Based on the weighted sum of gray values, a fused contrast-enhanced image is generated.

5. The method according to any one of claims 1-4, wherein, The step of processing the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map includes: Using a pre-trained image texture estimation model, texture probability estimation is performed on the original image to obtain a texture probability estimation map, wherein the texture probability estimation map includes texture probability estimation values ​​that correspond one-to-one with the pixels included in the original image; Based on a preset transformation relationship, the texture probability estimates included in the texture probability estimation map are transformed to obtain a fusion coefficient map.

6. The method according to claim 5, wherein, The process of transforming the texture probability estimates in the texture probability estimation map based on a preset transformation relationship to obtain a fusion coefficient map includes: Using a preset mapping function, the texture probability estimates included in the texture probability estimation map are mapped to fusion coefficients respectively, resulting in a fusion coefficient map.

7. The method according to claim 6, wherein, The mapping function is a monotonically increasing mapping function; The step of using a preset mapping function to map the texture probability estimates included in the texture probability estimation map to fusion coefficients, thereby obtaining a fusion coefficient map, includes: For any texture probability estimate included in the texture probability estimation map, the monotonically increasing mapping function is used to map the texture probability estimate to a first fusion coefficient when the texture probability estimate indicates that the corresponding pixel is located in a texture region; and to map the texture probability estimate to a second fusion coefficient when the texture probability estimate indicates that the corresponding pixel is located in a flat region, wherein the first fusion coefficient is greater than the second fusion coefficient.

8. The method according to claim 5, wherein, The image texture estimation model includes an image segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module; The step of using a pre-trained image texture estimation model to perform texture probability estimation on the original image to obtain a texture probability estimation map includes: The original image is divided into blocks using the image segmentation module to obtain an image block set; Using the feature extraction module, features are extracted from each image block in the image block set to obtain an image token sequence, wherein each image token in the image token sequence corresponds to an image block; The image token sequence is encoded using the encoder to obtain an encoded data sequence; Using the low-resolution probability estimation module, texture probability prediction is performed on each piece of encoded data in the encoded data sequence to obtain a low-resolution texture probability map. The upsampling module is used to upsample the low-resolution texture probability map to obtain a texture probability estimation map with the same resolution as the original image.

9. The method according to claim 8, wherein, The low-resolution probability estimation module includes a fully connected layer and an activation function layer; The step of using the low-resolution probability estimation module to perform texture probability prediction on each piece of coded data in the coded data sequence to obtain a low-resolution texture probability map includes: Using the fully connected layer, each piece of encoded data in the encoded data sequence is mapped to a texture feature estimate; Using the activation function layer, the obtained texture feature estimates are converted into low-resolution texture probability values ​​to obtain the low-resolution texture probability map.

10. The method according to claim 5, wherein, The image texture estimation model is pre-trained according to the following steps: Acquire sample images; Determine the labeled texture probability value corresponding to each pixel in the sample image; Using the initial image texture estimation model, texture probability estimation is performed on the sample image to obtain the predicted texture probability estimates for each pixel in the sample image. Using a preset loss function, the error between the predicted texture probability estimate and the labeled texture probability is determined; Based on the error, adjust the parameters of the initial texture estimation model; If the initial texture estimation model after parameter adjustment meets the preset training termination condition, the current initial texture estimation model is determined as the trained image texture estimation model.

11. The method according to claim 10, wherein, Determining the labeled texture probability values ​​corresponding to the pixels in the sample image includes: The sample image is divided into blocks to obtain a set of sample block regions; For each sample block region in the sample block region set, determine the proportion of the texture region area in the sample block region to the sample block region, and determine the proportion as the labeled texture probability value of each pixel included in the sample block region.

12. An image contrast enhancement device, comprising: The acquisition module is used to acquire the original image to be contrast-enhanced; An enhancement module is used to enhance the contrast of the original image to obtain an initial contrast-enhanced image; An estimation module is used to process the original image using a pre-trained fusion coefficient estimation model to obtain a fusion coefficient map, wherein the fusion coefficients in the fusion coefficient map are used to reduce the contrast enhancement intensity of flat regions in the original image and increase the contrast enhancement intensity of texture regions in the original image. The fusion module is used to fuse the initial contrast-enhanced image and the original image using the fusion coefficients included in the fusion coefficient map, to obtain a fused contrast-enhanced image.

13. A computer-readable storage medium storing a computer program, which, when executed, implements the image contrast enhancement method according to any one of claims 1-11.

14. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image contrast enhancement method according to any one of claims 1-11.

Images