Image noise level estimation method, apparatus, readable storage medium, and electronic device

Abstract

This disclosure provides an image noise level estimation method, apparatus, readable storage medium, and electronic device. [Solution] This method includes the steps of: obtaining a source image to be subjected to noise level estimation; performing noise level estimation on the source image to obtain an initial noise level estimation map; processing the source image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map; and performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map to obtain an adjusted noise level estimation map.

Description

Technical Field

[0005] ,

[0001] The present disclosure relates to an image noise level estimation method, apparatus, readable storage medium, and electronic device.

Background Art

[0002] Noise estimation refers to the process of quantifying the noise level from a single image containing noise. Its purpose is not to directly remove the noise, but to generate a noise level estimation map of the same size as the input image, and each pixel value in this map represents the noise intensity of the pixel at the corresponding position. The noise estimation method provides a basis for important parameters in subsequent processes such as image noise removal.

[0003] Currently, commonly used noise estimation methods have insufficient ability to identify image content and are prone to misidentifying some texture details as noise. Also, in scenes with complex textures or a large amount of noise, it is impossible to effectively distinguish between noise and actual textures, resulting in a low accuracy of noise level estimation.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to solve the above technical problems, the present disclosure provides an image noise level estimation method, apparatus, readable storage medium, and electronic device for solving the problem of reduced accuracy of noise estimation due to insufficient ability to identify image content when performing image noise level estimation.

Means for Solving the Problems

[0005] A first embodiment of the embodiments of the present disclosure provides an image noise level estimation method comprising: acquiring a source image to which noise level estimation is to be performed; performing noise level estimation on the source image to acquire an initial noise level estimation map including initial noise level estimation values ​​corresponding to each pixel point in the source image; processing the source image using a pre-trained image texture estimation model to acquire a noise level adjustment coefficient map, wherein the noise level adjustment coefficients in the noise level adjustment coefficient map reduce the noise level of texture regions in the source image and increase the noise level of flat regions in the source image; and performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map to acquire an adjusted noise level estimation map.

[0006] An image noise level estimation device according to a second embodiment of the embodiments of the present disclosure includes: an acquisition module for acquiring a source image to which noise level estimation is to be performed; a first estimation module for performing noise level estimation on the source image and acquiring an initial noise level estimation map including initial noise level estimation values ​​corresponding to each pixel point in the source image; a second estimation module for processing the source image using a pre-trained image texture estimation model and acquiring a noise level adjustment coefficient map, wherein the noise level adjustment coefficients in the noise level adjustment coefficient map reduce the noise level of texture regions in the source image and increase the noise level of flat regions in the source image; and an adjustment module for performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map and acquiring an adjusted noise level estimation map.

[0007] A computer-readable storage medium according to a third embodiment of the embodiments of this disclosure stores a computer program, and when the computer program is executed by a processor, the processor is instructed to perform the steps of the image noise level estimation method described above.

[0008] An electronic device according to a fourth embodiment of the embodiments of the present disclosure includes a processor and a memory for storing instructions that the processor can execute, and the above-described image noise level estimation method is realized by the processor reading and executing instructions that the processor can execute from the memory. [Effects of the Invention]

[0009] According to the image noise level estimation method, apparatus, readable storage medium, and electronic device of the embodiment of this disclosure, an initial noise level estimation map is obtained by estimating the noise level of the original image, an initial noise level estimation map is obtained by processing the original image using a pre-trained image texture estimation model, and finally, a noise level adjustment is performed on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map to obtain an adjusted noise level estimation map. The adjusted noise level estimation map can be used for noise reduction processing of the image, thereby enabling targeted noise level adjustment to different regions according to the degree of texture intensity in different regions of the image, improving the accuracy of noise level estimation, helping to accurately distinguish between noise in the image and the actual texture, and consequently improving the accuracy of image noise reduction processing. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a flowchart of an image noise level estimation method according to one example of the present disclosure. [Figure 2] Figure 2 is a flowchart of an image noise level estimation method relating to another example of the present disclosure. [Figure 3] Figure 3 is a flowchart of an image noise level estimation method relating to yet another example of the present disclosure. [Figure 4] Figure 4 is a flowchart of an image noise level estimation method relating to yet another example of the present disclosure. [Figure 5] Figure 5 is a structural diagram of an image texture estimation model according to one example of this disclosure. [Figure 6] Figure 6 is a flowchart of an image noise level estimation method relating to yet another example of the present disclosure. [Figure 7] Figure 7 is a training flowchart of an image texture estimation model according to one example of the present disclosure. [Figure 8] Figure 8 is a flowchart of an image noise level estimation method relating to yet another example of the present disclosure. [Figure 9] Figure 9 is a structural diagram of an image noise level estimation device according to one example of the present disclosure. [Figure 10] Figure 10 is a structural diagram of an image noise level estimation device relating to another example of the present disclosure. [Figure 11] Figure 11 is a structural diagram of an electronic device according to one example of the present disclosure. [Modes for carrying out the invention]

[0011] Hereinafter, exemplary embodiments of this disclosure will be described in detail with reference to the drawings in order to interpret this disclosure. The embodiments described are only a selection of embodiments of this disclosure, not all embodiments, and this disclosure is not limited to exemplary embodiments.

[0012] Unless otherwise specified, the relative arrangements of the components and steps, formulas, and numerical values ​​described in these embodiments do not limit the scope of this disclosure.

[0013] [Summary of the application] The core idea of ​​the related image noise estimation method is to divide an image into multiple image blocks, preferentially select "flat blocks" with simple textures and gentle gradient changes, and perform noise statistical analysis on them, based on the fundamental assumption that pixel changes in these regions are mainly due to noise. However, this method has the following drawbacks:

[0014] First, the ability to distinguish image content is insufficient, limiting the accuracy of estimation. The relevant methods heavily rely on correctly selecting "flat regions." However, in complex natural images, textured regions and noise-contaminated flat regions can be very similar in local statistical properties, making accurate distinction difficult. Once a block containing fine texture is misidentified as a flat region, image details are mistaken for noise, leading to an overestimation of the noise level. Conversely, if true flat regions cannot be identified, the noise level will be underestimated. Due to these inherent uncertainties, it is difficult to guarantee the accuracy and stability of noise estimation results.

[0015] Next, there is a lack of consideration for the characteristics of human vision. Related techniques typically assume that noise is uniformly distributed in the image space and aim to estimate a globally unified noise level. However, noise perception by the human visual system is non-uniform across different areas of an image; that is, the human eye is very sensitive to noise in flat areas (e.g., sky, walls) but not so sensitive (or insensitive) to noise in areas with complex textures (e.g., grass, bushes). Related methods fail to reflect this important difference in perception, and their estimation results cannot guide subsequent denoising processes that conform to the subjective quality of the human eye.

[0016] Finally, performance is inconsistent in scenes with severe noise contamination or complex textures. When image noise is heavy, it obscures the original structure and texture information of the image, making it extremely difficult to accurately identify flat areas, and consequently rendering block-based selection methods ineffective. Also, for images with very rich texture details, there are too few available flat areas, resulting in insufficient samples for statistical estimation and ultimately a significant decrease in the reliability of noise estimates.

[0017] In summary, the relevant technologies have failed to effectively resolve the inherent contradictions between the complexity of image content, the characteristics of the human visual system, and the spatial variability of noise.

[0018] Embodiments of the present disclosure aim to solve the above problems. In its noise estimation process, rather than being uniformly applied to the entire image, it is adaptively adjusted according to the semantic content (e.g., texture, edge, flat region) of different regions of the image. By performing texture estimation on the original image, a noise level adjustment coefficient map is obtained, and the initial noise level estimation map is adjusted using the noise level adjustment coefficient map. This process introduces the empirical knowledge of the human visual system, that is, the knowledge that the human eye is more sensitive to noise in flat regions (e.g., sky, wall surface) and less sensitive to noise in texture regions (e.g., lawn, fabric). Therefore, by performing noise removal using the adjusted noise level estimation map, it is possible to determine which regions should suppress noise more aggressively (flat regions) and which regions should be processed conservatively to retain more details (texture regions).

[0019] [Exemplary Method] FIG. 1 is a flowchart of an exemplary image noise level estimation method according to the present disclosure. This embodiment is applicable to various types of electronic devices. As shown in FIG. 1, it includes the following steps 101 to 104.

[0020] In step 101, an original image for which noise level estimation is to be performed is acquired.

[0021] Here, the above original image can be an image taken in any scene, for example, an image taken by a camera on a vehicle, an image taken by a user using a mobile phone, etc.

[0022] In step 102, noise level estimation is performed on the original image to obtain an initial noise level estimation map.

[0023] Here, the initial noise level estimation map includes initial noise level estimation values corresponding to each pixel point in the original image. That is, the size of the initial noise level estimation map is The size is the same as the original image, and each initial noise level estimate within it corresponds to a single pixel point in the original image, representing the noise level of that pixel point. Generally, a higher noise level indicates a greater probability that the corresponding pixel point contains noise.

[0024] Electronic devices performing this method can carry out this step using various relevant noise level estimation methods. Selectively, noise estimation methods based on image block partitioning (e.g., block partitioning methods based on local variance statistics, block partitioning methods based on texture complexity selection) can be employed to estimate the noise level of the original image.

[0025] In step 103, the original image is processed using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map.

[0026] Here, the noise level adjustment coefficients in the noise level adjustment coefficient map are used to reduce the noise level in textured areas of the original image and increase the noise level in flat areas of the original image.

[0027] The image texture estimation model described above is intended to represent the correspondence between the original image and the noise level adjustment coefficient map. This image texture estimation model can be constructed in various ways, such as by creating a correspondence table or by training a neural network model using machine learning methods.

[0028] When selectively training an image texture estimation model using machine learning methods, a large number of sample images can be collected beforehand, and noise level adjustment coefficients can be annotated at each pixel point in each sample image. Then, through iterative adjustment of the initial model parameters, the error between the predicted noise level adjustment coefficient output by the model and the annotated noise level adjustment coefficient is gradually reduced. When the error converges, the current initial model can be used as the trained image texture estimation model.

[0029] Selectively, an image texture estimation model can be used to perform texture probability estimation on the original image, obtain a texture probability estimation map, and then transform the texture probability estimates included in the texture probability estimation map to obtain a noise level adjustment coefficient map.

[0030] In step 104, the noise level adjustment coefficients included in the noise level adjustment coefficient map are used to adjust the noise level of the initial noise level estimation map, and the adjusted noise level estimation map is obtained.

[0031] Since there is a one-to-one correspondence between the noise level adjustment coefficients included in the noise level adjustment coefficient map and the initial noise level estimates included in the initial noise level estimation map, it is possible to perform calculations on any initial noise level estimate based on the corresponding noise level adjustment coefficient to obtain an adjusted noise level estimate.

[0032] The adjusted noise level estimation map can be obtained by selectively multiplying each pixel in the initial noise level estimation map by the initial noise level estimation value corresponding to each pixel in the noise level adjustment coefficient map. Mathematically, this can be expressed as N2 = N1 × α_map, where N1 represents the initial noise level estimation map, N2 represents the adjusted noise level estimation map, and α_map is the noise level adjustment coefficient map. The resolutions of N1, N2, and α_map are the same. By adjusting the initial noise level estimation map using a pixel-by-pixel multiplication method, the adjustment process can be simplified and the efficiency of noise level estimation can be improved.

[0033] Selectively, based on the pixel-by-pixel multiplication results described above, the calculated noise level estimates can be further processed (e.g., through filtering) to obtain an adjusted noise level estimate map.

[0034] Typically, after obtaining a noise level estimation map, noise reduction processing (e.g., noise reduction using an adaptive noise reduction algorithm) can be performed using this noise level estimation map. During the noise reduction processing, noise reduction of the response intensity can be performed based on the noise level estimates included in the noise level estimation map. That is, the larger the noise level estimate, the greater the noise reduction intensity. In the embodiments of this disclosure, since the noise level estimates are related to texture probabilities, i.e., the noise level estimates corresponding to texture regions are lower than in conventional noise reduction methods, the noise reduction intensity in texture regions is small, and the noise reduction intensity in flat regions is large.

[0035] The image noise level estimation method according to the embodiment of this disclosure involves performing noise level estimation on the original image to obtain an initial noise level estimation map, processing the original image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map, and finally performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map to obtain an adjusted noise level estimation map. The adjusted noise level estimation map is used for noise reduction processing of the image, thereby enabling targeted noise level adjustment to different regions according to the degree of texture intensity in different regions of the image, improving the accuracy of noise level estimation, helping to accurately distinguish between noise in the image and the actual texture, and consequently improving the accuracy of image noise reduction processing.

[0036] In several selectable implementations, as shown in Figure 2, step 102 above includes the following steps 1021 to 1023.

[0037] In step 1021, for any single pixel point in the original image, at least two color difference values ​​are determined between that pixel point and other pixel points within the target region.

[0038] Here, the target region is an image region of a predetermined size with the pixel point in question as the reference point. The position of the reference point for each image region can be arbitrarily specified. Typically, the reference point can be the center point of the image region. The above-mentioned predetermined size can be arbitrarily set, for example, an image block consisting of 31 x 31 pixels.

[0039] For any single pixel point in the original image, at least two color difference values ​​are calculated based on that pixel point. For example, the color difference value can be calculated between the pixel point and each pixel point in the target area, or a predetermined number of pixel points can be selected and color difference values ​​can be calculated. The algorithm for calculating the color difference value can be arbitrarily set; for example, the difference in color values ​​between two pixels can be calculated as the color difference value, or the difference in block pixel values ​​can be calculated as the color difference value using the method described in the embodiment below.

[0040] Furthermore, the data used to calculate color difference values ​​is typically the gray value of the pixel. It is also possible to selectively calculate color difference values ​​based on each color component contained in the pixel (for example, by using the average value of the RGB color components).

[0041] Note that the pre-set size is the maximum size of each image region. In the boundary and corner regions of the original image, the number of pixels around the reference point does not meet the number of pixels specified by the pre-set size. Therefore, the size of the image region corresponding to this reference point can be smaller than the pre-set size.

[0042] In step 1022, the initial noise level estimate corresponding to the pixel point is determined based on at least two color difference values.

[0043] After obtaining at least two color difference values ​​corresponding to each pixel, the initial noise level estimate for that pixel point can be determined by various methods. For example, the minimum or average value of the at least two color difference values ​​can be used as the initial noise level estimate.

[0044] In step 1023, the acquired initial noise level estimates are combined to form an initial noise level estimation map.

[0045] This embodiment employs a search window method to calculate the color difference value between each pixel in the original image and other surrounding pixels, enabling the rapid acquisition of an initial noise level estimate based on the color difference value. The acquired initial noise level estimate map can rudimentarily reflect the noise characteristics of the original image, which helps in efficiently performing subsequent noise level adjustments.

[0046] In several selectable implementations, as shown in Figure 3, step 1021 above includes the following steps 10211 to 10213.

[0047] In step 10211, the target region is divided into blocks according to a predetermined block division size to obtain at least two block regions.

[0048] Here, the size of the block division can be set arbitrarily, for example, 9x9 pixels. In this embodiment, the target area can be divided into at least two non-overlapping block areas, or it can be divided into block areas having at least two overlapping areas in the form of a sliding window.

[0049] In step 10212, the block region where the pixel point is located is designated as the central block region, and the difference in block pixel values ​​corresponding to the central block region and the other block regions in at least two other block regions is determined.

[0050] The block pixel value difference can be calculated according to various algorithms for calculating the similarity between image blocks. For example, the block pixel value difference can be the L1 distance, L2 distance, etc., between a central block region and other block regions.

[0051] In step 10213, the differences between at least two obtained block pixel values ​​are determined as at least two color difference values.

[0052] This embodiment divides the target region where each pixel point in the original image is located into blocks, and calculates the difference in block pixel values ​​between block regions as a color difference value. This allows the color difference value to reflect the local texture features of the image, which helps improve the adaptability of noise level estimation, reduces the amount of data to be calculated, and improves computational efficiency.

[0053] In several selectable implementations, as shown in Figure 4, step 103 above includes the following steps 1031 to 1032.

[0054] In step 1031, a pre-trained image texture estimation model is used to perform texture probability estimation on the original image and obtain a texture probability estimation map.

[0055] Here, the texture probability estimates included in the texture probability estimation map correspond one-to-one with the pixel points in the original image. The texture probability estimate represents the probability that the corresponding pixel point is located in a textured area. A larger texture probability estimate indicates a higher probability that the corresponding pixel point is located in a textured area, while a smaller texture probability estimate indicates a higher probability that the corresponding pixel point is located in a flat area.

[0056] The image texture estimation model described above can be obtained by pre-training it using machine learning methods. For example, a large number of sample images can be collected in advance, and a texture probability value can be annotated for each pixel point in each sample image. Then, by iteratively adjusting the parameters of the initial model, the error between the predicted texture probability estimation map output by the model and the annotated texture probability estimation map is gradually reduced. When the error converges, the current initial model can be used as the trained image texture estimation model.

[0057] In step 1032, the texture probability estimates included in the texture probability estimation map are transformed based on a pre-configured transformation relationship to obtain a noise level adjustment coefficient map.

[0058] Here, the above conversion relationships can be pre-configured. These conversion relationships can be represented in the form of calculation formulas, correspondence tables, etc. By using these conversion relationships, each texture probability estimate can be converted into a noise level adjustment coefficient. The obtained noise level adjustment coefficients are combined to form a single noise level adjustment coefficient map.

[0059] Typically, there can be an inverse relationship between the texture probability estimate and the noise level adjustment coefficient. That is, the larger the texture probability estimate, the greater the probability that the corresponding pixel belongs to the texture region, and the smaller the corresponding noise level adjustment coefficient, which weakens the noise level and prevents the actual texture from being mistakenly identified as noise and removed in subsequent noise reduction processing. Conversely, the smaller the texture probability estimate, the smaller the probability that the corresponding pixel belongs to the texture region, and the larger the corresponding noise level adjustment coefficient, which strengthens the noise level, allowing for high-intensity noise reduction on flat areas in subsequent noise reduction processing.

[0060] This embodiment utilizes an image texture estimation model to estimate texture probabilities for the original image, and then transforms the estimated texture probabilities to obtain a noise level adjustment coefficient map. This allows for the targeted determination of the corresponding noise level adjustment coefficients according to the texture probabilities of each region in the image, thereby improving the accuracy of generating the noise level adjustment coefficient map.

[0061] In several selectable implementations, step 1032 described above is performed The procedure may include a step to obtain a noise level adjustment coefficient map by using a pre-configured mapping function to map the texture probability estimates included in the texture probability estimation map to noise level adjustment coefficients.

[0062] This mapping function can reflect the relationship between texture probability estimates and noise level adjustment coefficients. The form of the mapping function can be set arbitrarily, and typically, the goal of setting the mapping function is to reduce the noise level adjustment coefficient in textured areas and increase it in flat areas.

[0063] This embodiment allows for the targeted mapping of arbitrary texture probability estimates to noise level adjustment coefficients by setting a mapping function, thereby making the noise level adjustment coefficient corresponding to each pixel more precise.

[0064] In several possible implementations, the mapping function is a monotonically decreasing mapping function.

[0065] The noise level adjustment coefficient map can be calculated by following these steps.

[0066] For any texture probability estimate included in the texture probability estimation map, a monotonically decreasing mapping function is used to map the texture probability estimate to a first noise level adjustment coefficient if the texture probability estimate indicates that the corresponding pixel point is located in the texture region, and to a second noise level adjustment coefficient if the texture probability estimate indicates that the corresponding pixel point is located in the flat region.

[0067] Here, the first noise level adjustment coefficient is smaller than the second noise level adjustment coefficient.

[0068] The form of the monotonically decreasing mapping function can be arbitrarily set. For example, this monotonically decreasing mapping function can be given as α = a - β × p, where a and β are tunable hyperparameters greater than 0, and p is the texture probability estimate. The significance of this function is that when p corresponding to a pixel is close to 1, it indicates that the pixel is located in the texture region, and the corresponding α is close to a - β, which has the effect of attenuating the noise level. When p is close to 0, it indicates that the pixel is located in the flat region, and the corresponding α is close to a, which has the effect of maintaining or enhancing the noise level.

[0069] This embodiment improves the accuracy of image noise level estimation by setting a monotonically decreasing mapping function, which allows textured and flat regions to be associated with different noise level adjustment coefficients. This helps to perform denoising of different intensities on textured and flat regions, further enhancing the accuracy of image denoising.

[0070] In several selectable implementations, as shown in Figure 5, the image texture estimation model includes an image block segmentation module 501, a feature extraction module 502, an encoder 503, a low-resolution probability estimation module 504, and an upsampling module 505.

[0071] As shown in Figure 6, step 1031 includes the following steps 10311 to 10315.

[0072] In step 10311, the image block division module is used to divide the original image into blocks and obtain a set of image blocks.

[0073] This embodiment does not limit the method used to perform block division on an image. For example, by using a convolutional layer with a kernel size and stride of P (e.g., 16) as an image block division module and performing a convolution operation on the input original image, the image can be divided into multiple non-overlapping P×P size image blocks.

[0074] In step 10312, the feature extraction module is used to extract features from each image block in the image block set to obtain an image token sequence.

[0075] Here, each image token in the image token sequence corresponds to one image block.

[0076] This feature extraction module can be implemented using a learnable linear projection layer. This feature extraction module can unpack each image block into a single vector, and then map this vector to a fixed-dimensional embedding space using a linear projection layer (usually a fully connected layer). For example, each image token in an image token sequence is a 1 × C size feature vector, where C is the feature dimension. The image token sequence can constitute a feature map with size (H / P) × (W / P) × C, where H and W are the height and width of the original image.

[0077] In step 10313, an encoder is used to encode the image token sequence and obtain the encoded data sequence.

[0078] Each encoded data in the encoded data sequence is obtained by encoding the corresponding image token using an encoder. Selectively, this encoder can be a deformable Transformer encoder. The above token sequence is input to a deformable Transformer encoder with multiple (e.g., 2-4) encoded blocks stacked together. The encoder's deformable self-attention mechanism allows the model to dynamically focus on other blocks most relevant to the content of the current image block, thereby more effectively capturing global contextual information and helping to accurately determine whether a single image block belongs to a textured region or a flat region.

[0079] In step 10314, a low-resolution probability estimation module is used to perform texture probability prediction for each encoded data in the encoded data sequence to obtain a low-resolution texture probability map.

[0080] Specifically, each encoded data in the encoded data sequence is input to a low-resolution probability estimation module, which then performs a texture probability prediction for each encoded data to obtain a corresponding texture probability value. Each texture probability value constitutes a low-resolution texture probability map. The size of the low-resolution texture probability map is (H / P) × (W / P), and the value of each point on this map is between 0 and 1, representing the probability that each image block contained in the original image belongs to the texture region.

[0081] In step 10315, the upsampling module is used to upsample the low-resolution texture probability map to obtain a texture probability estimation map with the same resolution as the original image.

[0082] The upsampling module can be implemented based on the relevant interpolation algorithm. For example, a bilinear interpolation algorithm can be used to upsample a low-resolution texture probability map P_low to the full resolution H×W of the original image to obtain a full-resolution texture probability estimation map P_full. The bilinear interpolation algorithm can generate smooth transitions, effectively avoiding block artifacts in the final noise level adjustment coefficient map.

[0083] This embodiment enables texture probability estimation based on a deep learning model by configuring an image texture estimation model with a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module, thereby effectively improving the accuracy of texture probability estimation.

[0084] In several selectable implementations, as shown in Figure 5, the low-resolution probability estimation module 504 includes a fully connected layer 5041 and an activation function layer 5042.

[0085] Step 10314 above can be performed as follows:

[0086] First, a fully connected layer is used to map each encoded data point in the encoded data sequence to a texture feature estimate.

[0087] Next, an activation function layer is used to convert each of the acquired texture feature estimates into low-resolution texture probabilities, thereby obtaining a low-resolution texture probability map.

[0088] The role of the fully connected layer described above is to integrate the features extracted in the previous layers of the image texture estimation model and output classification data. The type of activation function used by the activation function layer described above can usually be a sigmoid activation function, but it may selectively be another activation function, such as a tanh activation function. The role of the activation function layer is to perform binary classification on the texture feature estimates and output the probability that the corresponding image region is a texture region.

[0089] This embodiment achieves accurate texture probability prediction for each encoded data by providing a fully connected layer and an activation function layer in the low-resolution probability estimation module, thereby improving the accuracy of generating the texture probability estimation map.

[0090] In several selectable implementations, as shown in Figure 7, the image texture estimation model is obtained by pre-training according to the following steps 701 to 706.

[0091] Step 701 involves obtaining a sample image.

[0092] Here, the sample image may be an image obtained in advance by capturing a specific type of scene, and the application scene of the image noise level estimation method according to the embodiment of this disclosure belongs to that type of scene.

[0093] In step 702, the annotation texture probability value corresponding to each pixel point in the sample image is determined.

[0094] Here, the annotation texture probability value is a label between 0 and 1 that has been artificially annotated to each pixel point beforehand. Typically, to improve annotation efficiency, the sample image is partitioned, and the size of each partitioned image block is P×P, and the annotator can annotate each image block with a label between 0 and 1. The final label consists of a (H / P)×(W / P) matrix, where each value in the matrix represents the probability that the region at the corresponding position belongs to the texture region.

[0095] In step 703, an initial image texture estimation model is used to perform texture probability estimation on a sample image, and a predicted texture probability estimate corresponding to each pixel point in the sample image is obtained.

[0096] The structure of the initial image texture estimation model can be constructed using various relevant neural network models, for example, as shown in Figure 5. The initial image texture estimation model can perform texture probability estimation on a sample image according to the flow shown in Figure 6.

[0097] In step 704, a pre-configured loss function is used to determine the error between the predicted texture probability estimate and the annotation texture probability value.

[0098] This loss function can be any of the relevant types of loss functions, such as the L1 loss function. The loss value is the mean absolute error between the texture probability value predicted by the network and the actual label. This loss function is insensitive to outliers and makes training more stable.

[0099] In step 705, the parameters of the initial image texture estimation model are adjusted based on the error.

[0100] Typically, gradient descent and backpropagation can be used to iteratively adjust the parameters of the initial image texture estimation model until the model meets the training completion conditions.

[0101] In step 706, if the initial image texture estimation model after parameter adjustment satisfies the pre-set training completion conditions, the current initial image texture estimation model is confirmed as the post-training image texture estimation model.

[0102] Selectively, the above training termination conditions may include at least one of the following: the loss value has converged, the number of training iterations has reached a predetermined number, or the training time has reached a predetermined time.

[0103] The image texture estimation model training method according to this embodiment can adapt the model training process to actual application scenarios, thereby improving the accuracy of texture probability estimation by the model.

[0104] In several possible implementations, step 702 above can be performed as follows:

[0105] First, the sample image is divided into blocks to obtain a set of sample block regions.

[0106] Next, for any sample block region in the sample block region set, the proportion of the sample block region occupied by the texture region is determined, and this proportion is determined as the annotation texture probability value for the pixel points included in that sample block region.

[0107] Here, the texture region within any sample block region may be artificially defined or automatically defined by another texture recognition model.

[0108] This embodiment simplifies the annotation process and improves both annotation efficiency and model training efficiency by using the proportion of texture area occupied as the annotation texture probability value when performing annotation on a sample image.

[0109] Combining the above embodiments, Figure 8 shows an exemplary flowchart relating to an embodiment of the present disclosure. This flowchart includes two main paths: a block-partitioning noise level estimation path and a deep learning texture probability estimation path. The block-partitioning noise level estimation path generates an initial noise level estimation map N1 by calculating the minimum L1 distance between image blocks in the search window (see the corresponding embodiments in Figures 3 and 4 above). The deep learning texture probability estimation path generates a low-resolution texture probability map P_low in the following order: image block partitioning and embedding, deformable Transformer coding, a fully connected layer, and a Sigmoid activation function. Subsequently, a full-resolution probability map P_full is generated by bilinear interpolation upsampling, and the full-resolution probability map P_full is mapped to a noise level adjustment coefficient map α_map using an adjustment coefficient mapping function. Finally, the initial noise level estimation map N1 output by the block-partitioning noise level estimation path and the noise level adjustment coefficient map α_map output by the deep learning texture probability estimation path are multiplied pixel by pixel to obtain the final adjusted noise level estimation map N2.

[0110] [Example device] Figure 9 is a schematic diagram of the structure of an image noise level estimation device according to one example of the present disclosure. This embodiment is applicable to electronic devices, and as shown in Figure 9, the image noise level estimation device is An acquisition module 901 for obtaining the source image on which noise level estimation should be performed, A first estimation module 902 for performing noise level estimation on the original image and obtaining an initial noise level estimation map that includes initial noise level estimation values ​​corresponding to each pixel point in the original image, A second estimation module 903 for processing the original image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map, wherein the noise level adjustment coefficients in the noise level adjustment coefficient map reduce the noise level of texture regions in the original image and increase the noise level of flat regions in the original image. The system includes an adjustment module 904 for performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map, and obtaining an adjusted noise level estimation map.

[0111] Referring to Figure 10, Figure 10 is a schematic diagram of the structure of an image noise level estimation device relating to another example of the present disclosure.

[0112] In several selectable implementations, the second estimation module 903 is an estimation unit 9031 for obtaining a texture probability estimation map by performing texture probability estimation on the original image using a pre-trained image texture estimation model, wherein the texture probability estimates included in the texture probability estimation map correspond one-to-one with the pixel points included in the original image, and the conversion unit 9032 for obtaining a noise level adjustment coefficient map by converting the texture probability estimates included in the texture probability estimation map based on a pre-set conversion relationship.

[0113] In several selectable implementations, the first estimation module 902 includes a first determination unit 9021 for determining at least two color difference values ​​between any single pixel point in the original image and other pixel points in the target region, wherein the target region is an image region of a predetermined size with the pixel point as the reference point, the first determination unit 9021 includes a second determination unit 9022 for determining an initial noise level estimate corresponding to the pixel point based on the at least two color difference values, and a generation unit 9023 for combining the acquired initial noise level estimates to form an initial noise level estimation map.

[0114] In several selectable implementations, the first determination unit 9021 includes a block division subunit 90211 for dividing a target region into blocks according to a preset block division size to obtain at least two block regions, a first determination subunit 90212 for determining the block pixel value difference between the central block region and the other block regions among the at least two block regions, with the block region in which the pixel point is located being the central block region, and a second determination subunit 90213 for determining the obtained at least two block pixel value differences as at least two color difference values.

[0115] In several selectable implementations, the conversion unit 9032 further uses a pre-configured mapping function to map the texture probability estimates included in the texture probability estimation map to noise level adjustment coefficients, thereby obtaining a noise level adjustment coefficient map.

[0116] In several selectable implementations, the mapping function is a monotonically decreasing mapping function, and the transformation unit 9032 further uses the monotonically decreasing mapping function to map any texture probability estimate included in the texture probability estimation map to a first noise level adjustment coefficient if the texture probability estimate indicates that the corresponding pixel point is located in a texture region, and to a second noise level adjustment coefficient if the texture probability estimate indicates that the corresponding pixel point is located in a flat region, wherein the first noise level adjustment coefficient is smaller than the second noise level adjustment coefficient.

[0117] In several selectable implementations, the image texture estimation model includes an image block partitioning module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module. The estimation unit 9031 further performs block partitioning on the original image using the image block partitioning module to obtain a set of image blocks, performs feature extraction on each image block in the set of image blocks using the feature extraction module to obtain an image token sequence, where each image token in the image token sequence corresponds to one image block, encodes the image token sequence using the encoder to obtain an encoded data sequence, predicts the texture probability for each encoded data in the encoded data sequence using the low-resolution probability estimation module to obtain a low-resolution texture probability map, and upsamples the low-resolution texture probability map using the upsampling module to obtain a texture probability estimation map having the same resolution as the original image.

[0118] In several selectable implementations, the low-resolution probability estimation module includes a fully connected layer and an activation function layer, and the estimation unit 9031 further uses the fully connected layer to map each encoded data in the encoded data sequence to a texture feature estimate, and uses the activation function layer to convert each obtained texture feature estimate into a low-resolution texture probability to obtain a low-resolution texture probability map.

[0119] In several selectable implementations, the adjustment module 904 further multiplies the initial noise level estimate corresponding to each pixel point in the initial noise level estimation map by the noise level adjustment coefficient corresponding to each pixel point in the noise level adjustment coefficient map, pixel by pixel, to obtain an adjusted noise level estimation map.

[0120] In several selectable implementations, the image texture estimation model is obtained by pre-training according to the following steps: acquiring a sample image; determining the annotation texture probability value corresponding to each pixel point in the sample image; performing texture probability estimation on the sample image using the initial image texture estimation model to obtain a predicted texture probability estimate corresponding to each pixel point in the sample image; determining the error between the predicted texture probability estimate and the annotation texture probability value using a pre-set loss function; adjusting the parameters of the initial image texture estimation model based on the error; and determining the current initial image texture estimation model as the trained image texture estimation model if the parameter-adjusted initial image texture estimation model satisfies a pre-set training termination condition.

[0121] In several selectable implementations, the step of determining the annotation texture probability value corresponding to each pixel point in the sample image includes the steps of: dividing the sample image into blocks to obtain a set of sample block regions; and determining the ratio of the area of ​​the texture region within a given sample block region to the sample block region within the set of sample block regions, and determining this ratio as the annotation texture probability value of the pixel point included in that sample block region.

[0122] The exemplary embodiments of this apparatus correspond to the exemplary method described above in terms of implementation, and the corresponding content between the two can be referenced, combined, and cited; a detailed explanation is omitted here. The beneficial technical effects corresponding to the exemplary embodiments of this apparatus can be found by referring to the corresponding beneficial technical effects of the exemplary method described above; a detailed explanation is omitted here.

[0123] [Example electronic device] Figure 11 is a structural diagram of an electronic device 1100 according to an embodiment of the present disclosure, which includes at least one processor 1101 and a memory 1102.

[0124] The processor 1101 can be a central processing unit (CPU) or another form of processing unit having data processing capability and / or instruction execution capability, and can control other components in the electronic device 1100 to perform a desired function.

[0125] The memory 1102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. The computer-readable storage media may store one or more computer program instructions, and the processor 1101 can implement the image noise level estimation method and / or other desired functions of each embodiment of the present disclosure described above by executing one or more computer program instructions.

[0126] As an example, the electronic device 1100 may further include an input device 1103 and an output device 1104 connected to each other via a bus system and / or other form of connection mechanism (not shown).

[0127] The input device 1103 may include, for example, a keyboard, a mouse, etc.

[0128] The output device 1104 can output various types of information to the outside. This output device 1104 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0129] For simplicity, Figure 11 shows only some of the components relating to this disclosure in the electronic device 1100, omitting components such as buses and input / output interfaces. Otherwise, the electronic device 1100 may further include any other suitable components depending on the specific application.

[0130] [Examples of computer program products and computer-readable storage media] Embodiments of this disclosure provide a computer program product that includes computer program instructions in addition to the methods and apparatus described above. When the computer program instructions are executed by a processor, the processor is caused to perform the steps in the image noise level estimation method of the various embodiments of this disclosure described in the “Exemplary Methods” portion above.

[0131] Computer program products can be created using one or any combination of programming languages ​​to produce program code for performing the operations of the embodiments of this disclosure, including object-oriented programming languages ​​such as Java® and C++, and conventional procedural programming languages ​​such as the C language or similar programming languages. The program code may run entirely on a user computing device, partially on a user device, run as a standalone software package, run partially on a user computing device and partially on a remote computing device, or run entirely on a remote computing device or a server.

[0132] Furthermore, embodiments of the present disclosure further provide a computer-readable storage medium in which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor is caused to perform the steps in the image noise level estimation method of the various embodiments of the present disclosure described in the “Exemplary Methods” portion above.

[0133] Any combination of one or more readable media can be used as a computer-readable storage medium. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (non-exclusive list) of readable storage media include electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0134] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, merits, and effects mentioned in this disclosure are merely illustrative and not limiting, and these advantages, merits, and effects are not necessarily present in every example of this disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand effects and are not limiting, and the above details do not necessarily limit this disclosure to being realized by the above specific details.

[0135] Those skilled in the art can make various modifications and alterations to the present disclosure without departing from the spirit and scope of the present application. Thus, if such modifications and alterations of the present application fall within the claims of the present disclosure and the equivalent art thereto, the present disclosure also includes such modifications and alterations.

Claims

1. An image noise level estimation method in which each step is performed by an image noise level estimation device, The steps include: obtaining the original image to be used for noise level estimation, The steps include: performing noise level estimation on the original image to obtain an initial noise level estimation map that includes initial noise level estimation values ​​corresponding to each pixel point in the original image; A step of processing the original image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map, wherein the noise level adjustment coefficients in the noise level adjustment coefficient map reduce the noise level of texture regions in the original image and increase the noise level of flat regions in the original image. An image noise level estimation method characterized by comprising the step of performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map, and obtaining an adjusted noise level estimation map.

2. The step of performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map to obtain the adjusted noise level estimation map is: The image noise level estimation method according to claim 1, characterized in that it includes the step of multiplying an initial noise level estimate corresponding to each pixel point in the initial noise level estimation map by a noise level adjustment coefficient corresponding to each pixel point in the noise level adjustment coefficient map to obtain the adjusted noise level estimation map.

3. The step of performing noise level estimation on the aforementioned original image and obtaining an initial noise level estimation map is: A step of determining at least two color difference values ​​between any single pixel point in the original image and other pixel points within a target region, wherein the target region is an image region of a predetermined size with the single pixel point as the reference point. The steps include determining an estimated initial noise level corresponding to the pixel point based on the at least two color difference values, The image noise level estimation method according to claim 1, comprising the step of combining each acquired initial noise level estimate to form the initial noise level estimation map.

4. The step of determining at least two color difference values ​​between the pixel point and other pixel points within the target region is: The steps include: performing block division on the target region according to a predetermined block division size to obtain at least two block regions; The step of determining the difference in block pixel values ​​between the central block region and the other block regions among the at least two block regions, with the block region in which the pixel point is located being designated as the central block region, The image noise level estimation method according to claim 3, comprising the step of determining the difference between at least two acquired block pixel values ​​as the at least two color difference values.

5. The step of processing the original image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map is: A step of obtaining a texture probability estimation map by performing texture probability estimation on the original image using a pre-trained image texture estimation model, wherein the texture probability estimates included in the texture probability estimation map correspond one-to-one with the pixel points included in the original image. The image noise level estimation method according to claim 1, comprising the step of obtaining a noise level adjustment coefficient map by converting the texture probability estimates included in the texture probability estimation map based on a pre-set conversion relationship.

6. The step of obtaining a noise level adjustment coefficient map by transforming the texture probability estimates included in the texture probability estimation map based on a pre-set transformation relationship is: The image noise level estimation method according to claim 5, characterized in that it includes the step of mapping the texture probability estimates included in the texture probability estimation map to noise level adjustment coefficients using a pre-set mapping function to obtain a noise level adjustment coefficient map.

7. The aforementioned mapping function is a monotonically decreasing mapping function, The step of obtaining a noise level adjustment coefficient map by mapping the texture probability estimates included in the texture probability estimation map to noise level adjustment coefficients using a pre-configured mapping function is as follows: The process includes the steps of mapping any texture probability estimate included in the texture probability estimation map to a first noise level adjustment coefficient using the monotonically decreasing mapping function if the texture probability estimate indicates that the corresponding pixel point is located in a texture region, and mapping the texture probability estimate to a second noise level adjustment coefficient if the texture probability estimate indicates that the corresponding pixel point is located in a flat region. The image noise level estimation method according to claim 6, characterized in that the first noise level adjustment coefficient is smaller than the second noise level adjustment coefficient.

8. The aforementioned image texture estimation model includes an image block segmentation module, a feature extraction module, an encoder, a low-resolution probability estimation module, and an upsampling module. The step of obtaining a texture probability estimation map by performing texture probability estimation on the original image using a pre-trained image texture estimation model is: The steps include: using the aforementioned image block division module to divide the original image into blocks and obtain a set of image blocks; A step of obtaining an image token sequence by performing feature extraction on each image block in the image block set using the feature extraction module, wherein each image token in the image token sequence corresponds to one image block. The steps include: using the encoder to encode the image token sequence and obtaining an encoded data sequence; The steps include: using the low-resolution probability estimation module to perform texture probability prediction for each encoded data in the encoded data sequence to obtain a low-resolution texture probability map; The image noise level estimation method according to claim 5, comprising the step of upsampling the low-resolution texture probability map using the upsampling module to obtain the texture probability estimation map having the same resolution as the original image.

9. The low-resolution probability estimation module includes a fully connected layer and an activation function layer, The step of using the low-resolution probability estimation module to perform texture probability prediction for each encoded data in the encoded data sequence and obtain a low-resolution texture probability map is: The steps include: using the fully connected layer to map each encoded data in the encoded data sequence to a texture feature estimate; The image noise level estimation method according to claim 8, comprising the step of using the activation function layer to convert each acquired texture feature estimate into a low-resolution texture probability to obtain the low-resolution texture probability map.

10. The aforementioned image texture estimation model is Steps to obtain a sample image, The steps include determining the annotation texture probability value corresponding to each pixel point in the aforementioned sample image, The steps include: performing texture probability estimation on the sample image using an initial image texture estimation model to obtain predicted texture probability estimates corresponding to each pixel point in the sample image; A step of determining the error between the predicted texture probability estimate and the annotation texture probability value using a pre-set loss function, The steps include adjusting the parameters of the initial image texture estimation model based on the aforementioned error, The steps include: confirming the current initial image texture estimation model as the post-training image texture estimation model when the initial image texture estimation model after parameter adjustment satisfies the pre-set training completion conditions; The image noise level estimation method according to claim 5, characterized in that it is obtained by being pre-trained according to the following.

11. The step of determining the annotation texture probability value corresponding to each pixel point in the aforementioned sample image is as follows: The steps include: performing block division on the aforementioned sample image to obtain a set of sample block regions, The image noise level estimation method according to claim 10, comprising the steps of: determining the ratio of the area of ​​the texture region within a sample block region to the total area of ​​a sample block region for any sample block region in the set of sample block regions; and determining the ratio as the annotation texture probability value of the pixel points included in the sample block region.

12. An acquisition module for obtaining the original image of the target to be subjected to noise level estimation, A first estimation module for performing noise level estimation on the original image and obtaining an initial noise level estimation map that includes initial noise level estimation values ​​corresponding to each pixel point in the original image, A second estimation module for processing the original image using a pre-trained image texture estimation model to obtain a noise level adjustment coefficient map, wherein the noise level adjustment coefficients in the noise level adjustment coefficient map reduce the noise level of texture regions in the original image and increase the noise level of flat regions in the original image. An image noise level estimation device characterized by including an adjustment module for performing noise level adjustment on the initial noise level estimation map using the noise level adjustment coefficients included in the noise level adjustment coefficient map, and obtaining an adjusted noise level estimation map.

13. A computer-readable storage medium on which computer programs are stored, A computer-readable storage medium characterized in that, when the computer program is executed by the processor, the processor is instructed to perform the steps of the image noise level estimation method described in any one of claims 1 to 11.

14. An electronic device including a processor and a memory for storing instructions that the processor can execute, The electronic device is characterized in that the processor reads and executes the executable instructions from the memory to realize the image noise level estimation method according to any one of claims 1 to 11.

Images