Image denoising method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides an image denoising method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-processed image, and extracting low-frequency features and high-frequency features of the to-be-processed image; determining a first parameter corresponding to a preset scale according to the high-frequency feature and the preset scale; the first parameter represents the position of an object in the to-be-processed image represented in the high-frequency feature; determining a second parameter and a third parameter corresponding to a preset scale according to the low-frequency feature and the preset scale; the second parameter represents the position of an object in the to-be-processed image represented in the low-frequency feature; the third parameter represents image details of an object in the to-be-processed image represented in the low-frequency feature; determining a target image according to the high-frequency features and the first parameter, the second parameter and the third parameter corresponding to each preset scale; the target image is an image obtained after the to-be-processed image is subjected to noise removal. The method is used for improving the image denoising precision.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image denoising method, apparatus, device, storage medium, and program product. Background Technology

[0002] In monitoring scenarios, the monitoring results are determined by analyzing the acquired images. The less noise in the images, the more accurate the monitoring results. However, the image acquisition process, especially the acquisition of infrared images, is easily affected by equipment and environmental interference, resulting in acquired images containing a lot of noise, which in turn affects the monitoring results.

[0003] In related technologies, prior models based on median filtering and wavelet thresholding are used to denoise images. Although median filtering can suppress Gaussian noise, it will smooth the local gradient of the image. Wavelet thresholding, due to its fixed threshold, easily removes non-noise information, resulting in low image denoising accuracy.

[0004] Based on the above analysis, it can be seen that improving the accuracy of image denoising has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides image denoising methods, apparatus, devices, storage media, and program products to achieve the technical effect of improving the accuracy of image denoising.

[0006] In a first aspect, embodiments of this application provide an image denoising method, including:

[0007] The image to be processed is acquired, and its low-frequency and high-frequency features are extracted. The low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0008] Based on the high-frequency features and the preset scale, a first parameter corresponding to the preset scale is determined; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0009] Based on the low-frequency features and the preset scale, a second parameter and a third parameter corresponding to the preset scale are determined; the second parameter represents the position of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features.

[0010] The target image is determined based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image of the image to be processed after removing noise.

[0011] In one possible implementation, determining the first parameter corresponding to the preset scale based on the high-frequency features and the preset scale includes:

[0012] The high-frequency feature is determined to correspond to the feature at a preset scale, which is called the scale feature; the preset scale represents the resolution size, and there are multiple preset scales;

[0013] Based on the scale features corresponding to the preset scale, a first parameter corresponding to the preset scale is determined; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0014] In one possible implementation, determining the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale includes:

[0015] Based on the first parameter, second parameter, and third parameter corresponding to each preset scale, a first fusion feature is determined; the first fusion feature represents the fusion of information from high-frequency features and low-frequency features at each preset scale;

[0016] The target image is determined based on the high-frequency features and the first fusion feature.

[0017] In one possible implementation, determining the target image based on the high-frequency features and the first fused features includes:

[0018] The high-frequency feature is determined to correspond to the feature at a preset scale, which is called the scale feature; the preset scale represents the resolution size, and there are multiple preset scales;

[0019] The target image is determined based on the features at each scale and the first fusion feature.

[0020] In one possible implementation, determining the target image based on the scale features and the first fusion feature includes:

[0021] Based on the size of each preset scale, the features of each scale are sorted to obtain the sorting result;

[0022] Based on the sorting results, features at each scale are sequentially fused to obtain the second fused feature;

[0023] The target image is determined based on the first fusion feature and the second fusion feature.

[0024] In one possible implementation, determining the first fusion feature based on the first parameter, second parameter, and third parameter corresponding to each preset scale includes:

[0025] Based on the first parameter, the second parameter, and the third parameter at the preset scale, the fused feature components at the preset scale are determined; the fused feature components represent the information of high-frequency features and low-frequency features fused at the preset scale.

[0026] The first fusion feature is determined based on each fusion feature component.

[0027] In one possible implementation, the first fusion feature includes multiple feature values; the feature values ​​characterize features at preset locations in the image to be processed;

[0028] Determining the target image based on the high-frequency features and the first fused features includes:

[0029] For each feature value, based on the first fused feature and a preset range, multiple target values ​​corresponding to the feature value are determined.

[0030] For each target value, a weight is determined based on the feature value and its corresponding preset position, as well as the target value and its corresponding preset position; the weight represents the magnitude of the influence of the target value on the feature value.

[0031] The feature values ​​are adjusted according to each target value and its corresponding weight to obtain new feature values;

[0032] Based on each new feature value, a new first fusion feature is determined;

[0033] The target image is determined based on the high-frequency features and the new first fusion features.

[0034] In one possible implementation, extracting the low-frequency features of the image to be processed includes:

[0035] Extract initial features from the image to be processed; the initial features characterize the initial low-frequency information of the image to be processed;

[0036] Based on the initial features, a first feature is determined; the first feature characterizes the contrast information of the image to be processed.

[0037] Based on the initial features, scaling parameters and offset parameters are determined; the scaling parameters represent the adjustment factor of the first feature, and the offset parameters represent the degree of offset of the first feature.

[0038] The low-frequency feature is determined based on the initial feature, the first feature, the scaling parameter, and the offset parameter.

[0039] In one possible implementation, the method further includes:

[0040] The image to be trained and the preset labels are input into the initial model to obtain the output image corresponding to the image to be trained; wherein, the initial model is a neural network model to be trained, the initial model is used to perform image denoising processing, and the preset labels represent the image to be trained without noise;

[0041] Based on the preset label and the output image, frequency band difference information is determined using a preset first loss function; wherein, the frequency band difference information characterizes the magnitude of the difference between the preset label and the output image within a preset frequency band;

[0042] Based on the preset label and the output image, texture difference information is determined using a preset second loss function; wherein, the texture difference information characterizes the magnitude of the texture difference between the preset label and the output image;

[0043] The initial model is trained based on the frequency band difference information and the texture difference information to obtain a trained image denoising model.

[0044] The image denoising model is used to perform the image denoising method as described in the first embodiment.

[0045] Secondly, embodiments of this application provide an image denoising apparatus, comprising:

[0046] The acquisition module is used to acquire the image to be processed and extract the low-frequency features and high-frequency features of the image to be processed; the low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0047] The first determining module is used to determine a first parameter corresponding to the preset scale based on the high-frequency features and the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0048] The second determining module is used to determine a second parameter and a third parameter corresponding to the preset scale based on the low-frequency features and the preset scale; the second parameter represents the position of the object in the image to be processed represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed represented by the low-frequency features.

[0049] The third determining module is used to determine the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image of the image to be processed after removing noise.

[0050] Thirdly, embodiments of this application provide an image denoising device, including: a memory and a processor;

[0051] The memory stores computer-executed instructions;

[0052] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0053] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0054] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0055] The image denoising method, apparatus, device, storage medium, and program product provided in this application acquire an image to be processed, extracts low-frequency and high-frequency features from the image, where low-frequency features reflect the global structural information of the image and high-frequency features reflect the local detail information, facilitating subsequent image data processing by combining low-frequency and high-frequency features to simultaneously analyze the global structural information and local detail information of the image; by determining a first parameter corresponding to the preset scale based on the high-frequency features and the preset scale, the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features; and by determining a second parameter and a third parameter corresponding to the preset scale based on the low-frequency features and the preset scale, the second... The first parameter represents the location of the object in the image to be processed as represented by the low-frequency features, and the second parameter represents the image details of the object in the image to be processed as represented by the low-frequency features. This can further obtain cross-scale feature information, improve the accuracy of subsequent image denoising, and generate a high-quality denoised image. By determining the target image after noise removal based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale, the global structural information, local detail information, and cross-scale feature information of the image can be combined to accurately distinguish noise information and non-noise information in the image to be processed, thereby removing noise information from the image to be processed and obtaining a denoised image, thus improving the accuracy of image denoising. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0057] Figure 1 Flowchart of the image detection method provided in this application Figure 1 ;

[0058] Figure 2Flowchart of the image detection method provided in this application Figure 2 ;

[0059] Figure 3 Flowchart of the image detection method provided in this application Figure 3 ;

[0060] Figure 4 A schematic diagram of the process for extracting low-frequency features from an image to be processed, provided in this application;

[0061] Figure 5 A schematic diagram of the image processing flow in the image denoising method provided in this application;

[0062] Figure 6 Flowchart of the image denoising method provided in this application Figure 4 ;

[0063] Figure 7 A schematic diagram of the training framework for the initial model provided in this application;

[0064] Figure 8 A schematic diagram of the image denoising device provided in this application;

[0065] Figure 9 A schematic diagram of the image denoising device provided in this application.

[0066] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0068] The following examples illustrate specific application scenarios of this application. For instance, a specific application scenario could be the denoising of infrared images of forests acquired by infrared sensors, thereby enabling wildfire detection using the denoised images. For example, since infrared sensors can penetrate smoke, infrared images of wildfire scenarios acquired by these sensors can provide crucial information for disaster prevention and control. However, infrared images of wildfire scenarios face complex noise interference, such as random thermal radiation from surface vegetation, atmospheric turbulence fluctuations, and equipment hardware noise, all of which can introduce noise interference, easily leading to misjudgments in disaster prevention and control. Therefore, it is necessary to denoise the acquired infrared images of wildfire scenarios to reduce noise interference and improve the accuracy of disaster prevention and control. However, using prior models based on median filtering and wavelet thresholding to denoise infrared images makes it difficult to distinguish between noise and fire point features, especially in nighttime or long-distance monitoring scenarios, where the mixing of weak thermal signals and noise is more prominent, making it even more difficult to distinguish between noise and fire point features. For example, median filtering smooths the local gradient of fire points, while fixed thresholds can easily lead to fire points being misjudged as noise and filtered out, severely limiting the accuracy of wildfire detection.

[0069] A specific application scenario of this application can also be the denoising of infrared images of power equipment acquired by infrared thermal imagers, thereby using the denoised images to detect thermal faults in the equipment. For example, in power system operation and maintenance, infrared thermal imagers are widely used to acquire infrared images of equipment such as transformers and transmission lines to capture abnormal temperature rises on the equipment surface and detect potential equipment hazards such as insulation aging and poor contact in advance. However, infrared images are often subject to various noise interferences: such as sunlight reflection, random thermal radiation fluctuations caused by rain and snow; equipment noise caused by sensor dark current; and the mixing of reflections from the metal surface of the equipment with actual heat points. Median filtering can easily blur small fault points (such as locally overheated solder joints), while fixed thresholds can easily misjudge noise as faults, leading to missed detections or false alarms.

[0070] The image denoising method provided in this application is used to solve the above-mentioned technical problems.

[0071] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0072] Figure 1 Flowchart of the image detection method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0073] S101. Obtain the image to be processed and extract the low-frequency and high-frequency features of the image to be processed; the low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0074] The entity executing this application may be an electronic device with data processing capabilities. The electronic device may provide an image upload interface to acquire user-uploaded images to be processed. The image to be processed may be an infrared image; for example, it may be an infrared image of a wildfire scene captured by an infrared sensor and uploaded via the image upload interface.

[0075] For example, after acquiring an image to be processed, the electronic device extracts low-frequency and high-frequency features from the image. Low-frequency features represent the low-frequency information of the image, while high-frequency features represent the high-frequency information. Low-frequency information includes global structural information in the image; for example, in an infrared image of a wildfire scene, low-frequency information includes the overall thermal radiation distribution of the forest, unburned data, and temperature distribution information of large targets such as buildings. High-frequency information includes local detail information in the image; for example, in an infrared image of a wildfire scene, high-frequency information includes the flickering edges of flames and transient thermal signals of small moving targets such as birds and fallen leaves.

[0076] High-frequency information in an image to be processed can be filtered out using a filter, leaving low-frequency information; conversely, low-frequency information can be filtered out using a filter, leaving high-frequency information. This allows for the determination of high-frequency features based on the high-frequency information and low-frequency features based on the low-frequency information. For example, a Gaussian filter can be used to filter out high-frequency information in an image to be processed, leaving low-frequency information. The filtered high-frequency information can be used as high-frequency features, and the remaining low-frequency information can be used as low-frequency features.

[0077] Neural networks can also be used to decompose the image to be processed, thereby extracting low-frequency and high-frequency features. For example, the image to be processed can be decomposed according to the following formulas (1)-(2):

[0078] (1);

[0079] (2);

[0080] In the formula, This represents an image to be processed with a length of H, a width of W, and a channel count of 1. This represents a low-frequency component with a length of H, a width of W, and a number of channels of C. This represents a high-frequency component with a length of H, a width of W, and 3C channels. Indicates the core size is Deformable convolution, where i can be equal to 1, 2, or 3, is an improved convolution operation that can adaptively adjust the sampling position. Indicates the core size is Depthwise separable convolution is an efficient convolution operation that significantly reduces computation and parameter count by decomposing the convolution process while maintaining good feature extraction capabilities; ⊙ represents element-wise multiplication. represents the activation function; MLP stands for Multilayer Perceptron, a fully connected feedforward neural network operation used to generate weights for each channel; GAP stands for Global Average Pooling. Low-frequency features are determined based on the low-frequency components obtained from the decomposition, and high-frequency features are determined based on the high-frequency components obtained from the decomposition.

[0081] S102. Determine the first parameter corresponding to the preset scale based on the high-frequency features and the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0082] For example, after acquiring high-frequency features, a first parameter corresponding to the preset scale can be determined based on the high-frequency features and the preset scale. The first parameter represents the location of the object in the image to be processed as represented by the high-frequency features. For example, for an infrared image of a wildfire scene, the first parameter may include the location of a high-frequency region, such as the region formed by the edge of the wildfire representing the location of the wildfire object, and the region formed by the edge of the smoke representing the location of the smoke object.

[0083] It can include multiple preset scales, and the preset scales may be unequal. The preset scale represents the size of the image. For any preset scale, a first parameter corresponding to that preset scale can be determined based on the high-frequency features and the preset scale. This yields the first parameters corresponding to each preset scale, thus obtaining multiple first parameters representing the location of the object in the image to be processed, as represented by the high-frequency features. This enriches the subsequent processing data and improves the accuracy of subsequent image data processing.

[0084] Data processing based on high-frequency features and a preset scale can be performed using convolution operations to obtain the first parameter corresponding to the preset scale. For example, a corresponding convolution kernel can be set according to the preset scale, and the first parameter corresponding to the preset scale can be obtained by using the convolution calculation result of the high-frequency features and the convolution kernel.

[0085] Alternatively, a first parameter corresponding to a preset scale can be obtained by downsampling or upsampling the high-frequency features. For example, if the preset scale is smaller than the scale corresponding to the high-frequency features, the high-frequency features can be downsampled to obtain the first parameter corresponding to the preset scale. If the preset scale is larger than the scale corresponding to the high-frequency features, the high-frequency features can be upsampled to obtain the first parameter corresponding to the preset scale.

[0086] S103. Based on the low-frequency features and the preset scale, determine the second parameter and the third parameter corresponding to the preset scale; the second parameter represents the position of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features.

[0087] For example, after acquiring low-frequency features, a second parameter and a third parameter corresponding to the preset scale can be determined based on the low-frequency features and the preset scale. The second parameter represents the location of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features. For example, for an infrared image in a wildfire scene, the second parameter may include the location of the low-frequency background, such as the location of the geographical area formed by the edge of the terrain contour, and the third parameter may include noise information in the terrain contour, etc.

[0088] It can include multiple preset scales, and the sizes of each preset scale can be unequal. For any preset scale, a second parameter and a third parameter corresponding to that preset scale can be determined based on the low-frequency features and the preset scale. This yields the second and third parameters corresponding to each preset scale, resulting in multiple second parameters representing the location of the object in the image to be processed as indicated by the low-frequency features, and multiple third parameters representing the image details of the object in the image to be processed as indicated by the low-frequency features. This enriches the subsequent processing data and improves the accuracy of subsequent image data processing.

[0089] Data processing based on low-frequency features and a preset scale can be performed using convolution operations to obtain the second and third parameters corresponding to the preset scale. For example, a corresponding convolution kernel can be set according to the preset scale, and the second and third parameters corresponding to the preset scale can be obtained by using the convolution calculation result of the low-frequency features and the convolution kernel. The convolution kernels corresponding to the second and third parameters are not equal.

[0090] Alternatively, based on a preset scale, the second and third parameters corresponding to the preset scale can be obtained by downsampling or upsampling the low-frequency features. For example, if the preset scale is smaller than the scale corresponding to the low-frequency features, the low-frequency features can be downsampled to obtain the second and third parameters corresponding to the preset scale, where the second and third parameters correspond to different downsampling algorithms. If the preset scale is larger than the scale corresponding to the low-frequency features, the low-frequency features can be upsampled to obtain the second and third parameters corresponding to the preset scale, where the second and third parameters correspond to different upsampling algorithms.

[0091] S104. Determine the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image to be processed after removing noise.

[0092] For example, after obtaining the first parameter, second parameter, and third parameter corresponding to each preset scale, the target image can be determined based on the high-frequency features, combined with the first parameter, second parameter, and third parameter corresponding to each preset scale. The target image is the image to be processed after noise removal.

[0093] For example, by analyzing the first, second, and third parameters corresponding to each preset scale, noise and non-noise information in the image to be processed can be obtained. Based on the obtained noise and non-noise information, high-frequency features are reconstructed, and the corresponding noise information in the high-frequency information is removed to obtain the noise-removed target image.

[0094] For example, since low-frequency information includes global structural information in the image, the first parameter is determined based on the low-frequency features that characterize low-frequency information. Therefore, the first parameter corresponds to the multi-scale global structural information of the image to be processed. High-frequency information includes local detail information of the image. The second and third parameters are determined based on the high-frequency features that characterize high-frequency information. Therefore, the second and third parameters correspond to the multi-scale local detail information of the image to be processed. Based on the first, second, and third parameters corresponding to each preset scale, the neural network can learn the multi-scale global structural information, the multi-scale local detail information, and the noise information in the global structural information and local detail information of the image. This allows for the differentiation of noise and non-noise information in the image to be processed. Then, based on the differentiated noise and non-noise information in the image to be processed, the high-frequency features are reconstructed to obtain the target image after noise removal.

[0095] The image denoising method provided in this application acquires an image to be processed and extracts its low-frequency and high-frequency features. Low-frequency features reflect the global structural information of the image, while high-frequency features reflect its local detail information. This facilitates subsequent image data processing by combining low-frequency and high-frequency features to simultaneously analyze both the global structural information and local detail information. Based on the high-frequency features and a preset scale, a first parameter corresponding to the preset scale is determined. The first parameter represents the position of the object in the image to be processed as represented by the high-frequency features. Based on the low-frequency features and the preset scale, a second parameter and a third parameter corresponding to the preset scale are determined. The second parameter represents the low-frequency features. The first parameter represents the location of the object in the image to be processed, and the third parameter represents the image details of the object in the image to be processed, which are represented by the low-frequency features. This can further obtain cross-scale feature information, improve the accuracy of subsequent image denoising, and generate a high-quality denoised image. By determining the target image after noise removal based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale, the global structural information, local detail information, and cross-scale feature information of the image can be combined to accurately distinguish noise information and non-noise information in the image to be processed, thereby removing noise information in the image to be processed and obtaining a denoised image, thus improving the accuracy of image denoising.

[0096] In one specific embodiment, S102 above includes the following sub-steps:

[0097] First, determine the features corresponding to the high-frequency features at the preset scale, which are called scale features; the preset scale represents the resolution size, and there are multiple preset scales.

[0098] Second, determine the first parameter corresponding to the preset scale based on the scale features corresponding to the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0099] Figure 2 Flowchart of the image denoising method provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, S102 will be described in detail. The method includes:

[0100] S201. Obtain the image to be processed, and extract the low-frequency features and high-frequency features of the image to be processed; the low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0101] The execution process of S201 is the same as that of S101, and will not be described again here.

[0102] S202. Determine the features corresponding to the high-frequency features at the preset scale, which are called scale features; the preset scale represents the resolution size, and there are multiple preset scales.

[0103] For example, after obtaining high-frequency features, the features corresponding to the high-frequency features at a preset scale can be determined, which are called scale features. Here, the preset scale represents the resolution size, and there are multiple preset scales.

[0104] For example, after obtaining high-frequency and low-frequency features, a multi-scale pyramid feature structure can be constructed based on these features. The multi-scale pyramid feature structure includes features corresponding to multiple preset scales. The features corresponding to the high-frequency features at preset scales can be determined based on the multi-scale pyramid feature structure; these are the scale features. There are multiple preset scales, and each preset scale represents the resolution.

[0105] Specifically, high-frequency features can be upsampled to obtain high-resolution scale features corresponding to the high-frequency features. High-frequency features can be downsampled to obtain low-resolution scale features corresponding to the high-frequency features. By combining high-resolution scale features, low-resolution scale features, high-frequency features with unchanged resolution, and low-frequency features with unchanged resolution, a multi-scale pyramid feature structure can be constructed.

[0106] For example, multi-scale pyramid features can be constructed according to the following formula (3):

[0107] (3);

[0108] In the formula, Represents multi-scale pyramid features; This represents a low-resolution scale feature obtained by performing a 2x bicubic downsampling on the high-frequency features; This represents a high-resolution scale feature obtained by doubly upsampling the high-frequency features; This indicates that a channel splicing operation is being performed.

[0109] For high-frequency features, in multi-scale pyramid features, high-frequency features correspond to scale features at three preset scales, including the scale feature at the first preset scale. Scale characteristics of the second preset scale and the scale characteristics of the third preset scale. Among them, the scale features of the first preset scale. In the features of multi-scale pyramids Correspondingly; the scale characteristics of the second preset scale In the features of multi-scale pyramids Correspondingly; the scale characteristics of the third preset scale In the features of multi-scale pyramids Correspondingly, to enhance the correlation between multi-scale features, multi-scale pyramid features... , , It can be a scale feature that incorporates low-frequency characteristics.

[0110] S203. Determine the first parameter corresponding to the preset scale based on the scale features corresponding to the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0111] For example, after determining the scale features corresponding to each preset scale, the first parameter corresponding to the preset scale can be determined based on the scale features corresponding to the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0112] For example, for each preset scale, the first parameter corresponding to the preset scale can be determined based on the scale characteristics corresponding to the preset scale.

[0113] Specifically, convolution can be used to perform a convolution operation on the scale features corresponding to the preset scale to obtain the first parameter corresponding to the preset scale. Specifically, the first parameter corresponding to the preset scale can be calculated according to the following formula (4):

[0114] (4);

[0115] In the formula, This represents the first parameter corresponding to the k-th preset scale, where k can be equal to 1, 2, or 3; The size of the convolution kernel is . The weights of the convolution kernels corresponding to different preset scales are not equal, and the convolution kernels corresponding to different preset scales can be convolution kernels determined through training.

[0116] S204. Based on the low-frequency features and the preset scale, determine the second parameter and the third parameter corresponding to the preset scale; the second parameter represents the position of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features.

[0117] For example, for any preset scale, corresponding to the first parameter, a corresponding convolution kernel can be set according to the preset scale. The second and third parameters corresponding to the preset scale are obtained by using the convolution calculation result of the low-frequency features and the convolution kernel. Specifically, the second parameter corresponding to the preset scale can be calculated according to the following formula (5):

[0118] (5);

[0119] In the formula, This represents the second parameter corresponding to the k-th preset scale; Indicates the kernel size as The convolution operation can use a trained convolution kernel; it should be noted that the calculation formula for the third parameter corresponding to the preset scale is similar to formula (5), only the convolution kernels used are not equal, such as calculating the third parameter corresponding to the k-th preset scale. Sizes can be used The convolution kernel has the same size as the kernel corresponding to the second parameter, but its weights are not equal to those of the kernel corresponding to the second parameter; d represents the scaling factor.

[0120] S205. Determine the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image to be processed after removing noise.

[0121] The execution process of S205 is the same as that of S104, and will not be described again here.

[0122] In this embodiment, by determining the high-frequency features corresponding to a preset scale as scale features, and the preset scale representing the resolution, multiple preset scales can be obtained, resulting in scale features at multiple scales (resolutions) corresponding to the high-frequency features. These scale features at multiple scales exhibit scale correlation, thereby improving the accuracy of subsequent data processing. By determining the first parameter corresponding to the preset scale based on the scale features, the location information of objects in the image to be processed, represented by the high-frequency features, can be further extracted from the scale features. The information in the first parameter is more advanced than that in the scale features, providing more effective image information. Furthermore, the dimension C of the first parameter is lower than the dimension d of the scale features, which improves data processing efficiency.

[0123] In one specific implementation, S104 above includes the following sub-steps:

[0124] First, the first fusion feature is determined based on the first parameter, second parameter, and third parameter corresponding to each preset scale; the first fusion feature represents the information of high-frequency features and low-frequency features at each preset scale.

[0125] Second, the target image is determined based on the high-frequency features and the first fusion feature.

[0126] Figure 3 Flowchart of the image denoising method provided in this application Figure 3 ,like Figure 3 As shown, in this embodiment... Figure 1 Based on the embodiments, S104 will be described in detail. The method includes:

[0127] S301. Obtain the image to be processed and extract the low-frequency and high-frequency features of the image to be processed; the low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0128] The execution process of S301 is the same as that of S101, and will not be described again here.

[0129] S302. Based on the high-frequency features and the preset scale, determine the first parameter corresponding to the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0130] The execution process of S302 is the same as that of S102, and will not be described again here.

[0131] S303. Based on the low-frequency features and the preset scale, determine the second parameter and the third parameter corresponding to the preset scale; the second parameter represents the position of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features.

[0132] The execution process of S303 is the same as that of S103, and will not be described again here.

[0133] S304. Determine the first fusion feature based on the first parameter, second parameter, and third parameter corresponding to each preset scale; the first fusion feature represents the information of high-frequency features and low-frequency features fused at each preset scale.

[0134] For example, a first fusion feature can be determined based on the first parameter, second parameter, and third parameter corresponding to each preset scale. The first fusion feature represents the fusion of information from high-frequency and low-frequency features at each preset scale.

[0135] For example, the second and third parameters corresponding to each preset scale of low-frequency features can be used to guide the first parameter corresponding to each preset scale of high-frequency features for high-frequency reconstruction.

[0136] Specifically, the first combination of the first parameters corresponding to each preset scale is used as the Query in the attention mechanism. The first combination can be the result of concatenating multiple first parameters. The second combination of the second parameters corresponding to each preset scale is used as the Key in the attention mechanism. The second combination can be the result of concatenating multiple second parameters. The third combination of the third parameters corresponding to each preset scale is used as the Value in the attention mechanism. The third combination can be the result of concatenating multiple third parameters. Based on the attention mechanism, the fused data is calculated according to the Query, Key, and Value. Based on the calculated fused data, the first fused feature is determined.

[0137] S305. Determine the target image based on the high-frequency features and the first fusion feature; the target image is the image of the image to be processed after removing noise.

[0138] For example, after determining the first fusion feature, the target image can be determined based on the high-frequency feature and the first fusion feature; the target image is the image of the image to be processed after removing noise.

[0139] For example, noise and non-noise information in the image to be processed can be obtained by analyzing the first fusion feature. Based on the obtained noise and non-noise information, the high-frequency features are reconstructed to obtain the target image after noise removal.

[0140] For example, since the first fusion feature integrates information from high-frequency and low-frequency features at each preset scale, the neural network can be used to learn information from high-frequency and low-frequency features at each preset scale, as well as noise information from high-frequency and low-frequency features at each preset scale, based on the first fusion feature. This allows for the differentiation of noise and non-noise information in the image to be processed, and the high-frequency features can then be reconstructed to remove the corresponding noise information from the high-frequency information, resulting in a noise-removed target image.

[0141] In this embodiment, a first fusion parameter is determined based on a first parameter, a second parameter, and a third parameter corresponding to each preset scale. This determined first fusion parameter characterizes the fusion of high-frequency and low-frequency features at each preset scale and can be used for subsequent analysis of the correspondence between the information of high-frequency and low-frequency features at each preset scale and image noise. By determining the target image based on the high-frequency features and the first fusion feature, the first fusion feature can guide high-frequency features in image reconstruction, thereby removing noise information from the high-frequency features and obtaining a more accurate denoised image.

[0142] In some specific implementations, S304 above includes the following sub-steps:

[0143] S3041. Determine the fusion feature components at the preset scale based on the first parameter, the second parameter, and the third parameter at the preset scale; the fusion feature components represent the information of high-frequency features and low-frequency features fused at the preset scale.

[0144] For example, for each preset scale, the fused feature components at the preset scale can be determined based on the first parameter, the second parameter, and the third parameter at the preset scale; the fused feature components represent the information of the fused high-frequency features and low-frequency features at the preset scale.

[0145] For example, for any preset scale, the second and third parameters corresponding to the preset scale for the low-frequency components can be used to guide the first parameter corresponding to the preset scale for the high-frequency components in high-frequency reconstruction. That is, based on the first, second, and third parameters at the preset scale, the fused feature components at that preset scale are determined. These fused feature components represent the information of fused high-frequency and low-frequency features at that preset scale.

[0146] Specifically, the first parameter corresponding to the preset scale is used as the Query in the attention mechanism, the second parameter corresponding to the preset scale is used as the Key in the attention mechanism, and the third parameter corresponding to the preset scale is used as the Value in the attention mechanism. Based on the attention mechanism, the fused data is calculated according to the Query, Key, and Value. The first fused feature is determined based on the calculated fused data. For example, the fused feature components can be determined according to the following formula (6):

[0147] (6);

[0148] In the formula, This represents the fused feature component at the k-th preset scale; T represents the activation function; T represents the transpose. The mask corresponding to the k-th preset scale is used to dynamically reduce the attention weights in the gradient abrupt region of the feature components (which may contain noise). It can be calculated according to the following formula (7). :

[0149] (7);

[0150] In the formula, express The gradient; Indicates to Find the 1-norm; exp represents the natural exponential function.

[0151] S3042. Determine the first fusion feature based on each fusion feature component.

[0152] For example, after obtaining the fusion feature components corresponding to each preset scale, the first fusion feature can be determined based on each fusion feature component.

[0153] For example, the cumulative sum of each fusion feature component can be calculated, and the first fusion feature can be determined based on the cumulative sum. Specifically, the first fusion feature can be determined according to the following formula (8):

[0154] (8);

[0155] In the formula, This represents the first fusion feature.

[0156] In these implementations, based on an attention mechanism, for any preset scale, the fusion feature components at that preset scale are determined according to the first, second, and third parameters. High-frequency reconstruction can then be constrained using the keys and values ​​corresponding to the low-frequency features at that preset scale, ensuring consistency between the details and the overall structure of the reconstructed image. By accumulating and summing each fusion feature component to determine the first fusion feature, fusion feature components from multiple scales can be fused, thereby ensuring consistency between the details and the overall structure of the reconstructed image at each preset scale.

[0157] In some specific implementations, the first fusion feature includes multiple feature values; the feature values ​​characterize features at preset locations in the image to be processed, and S305 above includes the following sub-steps:

[0158] First, for each feature value, based on the first fusion feature and a preset range, determine multiple target values ​​corresponding to the feature value.

[0159] For example, the first fusion feature includes multiple feature values, each representing a feature at a preset location in the image to be processed. For instance, for any pixel location in the image to be processed, this pixel location is a preset location, and the value in the first fusion feature corresponding to this preset location is the feature value corresponding to that preset location. For each feature value, multiple target values ​​corresponding to the feature value can be determined based on the first fusion feature and a preset range.

[0160] For example, a local window of a preset size can be defined with the preset position corresponding to the feature value as the center, and this defined local window can be used as a preset range. This preset range includes multiple preset positions, and the multiple feature values ​​corresponding to these multiple preset positions in the first fusion feature are determined as multiple target values ​​corresponding to the feature values.

[0161] Specifically, for the first fusion feature, let the i-th preset position in the first fusion feature be the i-th pixel position. The feature value corresponding to the i-th preset position is The preset range corresponding to this preset position is as follows: local window centered , such as Centered The window, the partial window includes The multiple feature values ​​corresponding to multiple pixel positions within the preset range are determined in the first fusion feature. Multiple corresponding target values.

[0162] Second, for each target value, the weight is determined based on the feature value and its corresponding preset position, as well as the target value and its corresponding preset position; the weight represents the magnitude of the influence of the target value on the feature value.

[0163] For example, after determining the target value, for each target value, the weight can be determined based on the feature value and the preset position corresponding to the feature value, as well as the target value and the preset position corresponding to the target value; the weight represents the magnitude of the influence of the target value on the feature value.

[0164] For example, if the preset position The preset position corresponding to a target value is The preset position The corresponding target value is The weights can be determined according to the following formula (9):

[0165] (9);

[0166] In the formula, Indicates the preset position is The target value relative to the preset position is The weights of the feature values ​​are given by the equation, where the first term on the right-hand side is used to calculate feature similarity and the second term on the right-hand side is used to calculate spatial distance. Indicates to Find the square of the 2-norm; cos represents the cosine function; express and The distance between them; The adaptive bandwidth parameter can be expressed by the following formula (10):

[0167] (10);

[0168] It can be expressed by the following formula (11):

[0169] (11);

[0170] In the formula, max represents the maximum value function.

[0171] Third, adjust the feature values ​​according to each target value and its corresponding weight to obtain new feature values.

[0172] For example, after obtaining the weights corresponding to each target value, the feature values ​​can be adjusted according to each target value and its corresponding weight to obtain new feature values.

[0173] For example, the eigenvalues ​​can be adjusted by accumulating the products of each target value and its corresponding weight to obtain new eigenvalues. For instance, the new eigenvalues ​​can be calculated using the following formula (12):

[0174] (12);

[0175] In the formula, This represents a new eigenvalue.

[0176] Fourth, based on each new eigenvalue, determine the new first fusion feature.

[0177] For example, after obtaining the new feature values ​​corresponding to each preset position, a new first fusion feature can be determined based on each new feature value. For instance, for any preset position, the feature value corresponding to that preset position can be replaced with the new feature value corresponding to that preset position to obtain a new first fusion feature.

[0178] Fifth, the target image is determined based on the high-frequency features and the new first fusion features.

[0179] For example, after obtaining the new first fusion feature, the target image can be determined based on the high-frequency features and the new first fusion feature. For instance, the new first fusion feature can replace the original first fusion feature, and the target image can be determined based on the high-frequency features and the original first fusion feature.

[0180] In these embodiments, for each feature value in the first fusion feature, multiple target values ​​corresponding to the feature value are determined based on the first fusion feature and a preset range. For each target value, a weight is determined based on the feature value and its corresponding preset position, as well as the target value and its corresponding preset position. This weight represents the magnitude of the influence of the target value on the feature value and can be used to subsequently adjust the feature value to obtain a more accurate feature value. By adjusting the feature value according to each target value and its corresponding weight, new feature values ​​are obtained. When adjusting the feature value, the feature similarity and spatial neighborhood similarity between the feature value and the target value are calculated, which can enhance the continuity of image edges. By determining new first fusion features based on each new feature value, and by determining the target image based on high-frequency features and the new first fusion features, a more accurate target image can be obtained.

[0181] In some specific implementations, S305 above includes the following sub-steps:

[0182] S3051. Determine the features corresponding to the high-frequency features at the preset scale, which are called scale features; the preset scale represents the resolution size, and there are multiple preset scales.

[0183] The execution process of S3051 is the same as that of S202, and will not be described again here.

[0184] S3052. Determine the target image based on the features at each scale and the first fusion feature.

[0185] For example, after determining the features at each scale, the target image can be determined based on the features at each scale and the first fusion feature.

[0186] For example, noise and non-noise information in the image to be processed can be obtained by analyzing the first fusion feature. Based on the noise and non-noise information obtained from the analysis, features at each scale are reconstructed to obtain the target image after noise removal.

[0187] For example, since the first fusion feature integrates information from high-frequency and low-frequency features at each preset scale, the neural network can be used to learn information from high-frequency and low-frequency features at each preset scale, as well as noise information from high-frequency and low-frequency features at each preset scale, based on the first fusion feature. This allows for the differentiation of noise and non-noise information in the image to be processed, reconstruction of high-frequency features, removal of corresponding noise information from features at each scale, and the resulting noise-removed target image.

[0188] Multi-scale features can be fused to obtain fused multi-scale features. Based on the noise and non-noise information in the image to be processed, the corresponding noise information in the fused multi-scale features is removed to obtain the target image after noise removal.

[0189] In these implementations, by determining the features corresponding to high-frequency features at preset scales (which are called scale features), and since there are multiple preset scales, cross-scale feature information can be obtained for subsequent generation of denoised images. By determining the target image based on each scale feature and a first fusion feature, the first fusion feature can guide image reconstruction at each scale feature, thereby removing noise information from the high-frequency features. Image reconstruction using multiple scale features can gradually restore the high-frequency details of the image, avoiding distortion caused by direct reconstruction, thus obtaining a more accurate denoised image.

[0190] In some specific implementations, S3052 above includes the following sub-steps:

[0191] First, based on the size of each preset scale, the features of each scale are sorted to obtain the sorting results.

[0192] For example, features at each scale can be sorted according to the size of each preset scale to obtain a sorting result.

[0193] For example, since the preset scale represents the resolution, the features of each scale can be sorted in descending order of resolution based on the resolution corresponding to each preset scale, and the sorting result can be obtained.

[0194] Specifically, if the scale features include the scale features of the first preset scale... Scale characteristics of the second preset scale and the scale characteristics of the third preset scale. The three scale features are sorted in descending order of resolution, and the sorting result is as follows: .

[0195] Second, based on the ranking results, features at each scale are fused sequentially to obtain the second fused feature.

[0196] For example, after obtaining the sorting result, features at each scale can be fused sequentially based on the sorting result to obtain the second fused feature. Features at each scale can be fused sequentially based on the residual blocks. The second fused feature represents the fused feature that incorporates features from each scale.

[0197] Specifically, if the sorting result is The second fused feature can be obtained by fusing features at each scale sequentially according to the following formulas (13)-(15):

[0198] (13);

[0199] (14);

[0200] (15);

[0201] In the formula, This indicates the result of the first fusion. This indicates residual calculation; This indicates the result of the second fusion; This indicates that the first fusion result is upsampled. For example, a three-stage upsampling method can be used to upsample the first sampling result. This indicates the result of the third fusion; This indicates upsampling of the second fusion result. It can be based on the final fusion result, i.e., the third fusion result. This was identified as the second fusion feature.

[0202] Third, the target image is determined based on the first fusion feature and the second fusion feature.

[0203] For example, after obtaining the second fusion feature, the target image can be determined based on the first fusion feature and the second fusion feature.

[0204] For example, noise and non-noise information in the image to be processed can be obtained by analyzing the first fusion feature. Based on the noise and non-noise information obtained from the analysis, the second fusion feature is reconstructed to obtain the target image after noise removal.

[0205] For example, if the second fusion feature is The target image can be determined according to the following formula (16):

[0206] (16);

[0207] In the formula, Represents the target image; The first fusion feature is represented by MLP, which stands for Multilayer Perceptron Operation. The Multilayer Perceptron is a trained fully connected feedforward neural network used to generate the weights of each channel in the first fusion feature. Noise information and non-noise information can be distinguished by the weights. For example, the more noise information there is, the smaller the corresponding channel weight is, and the less noise information there is, the larger the corresponding channel weight is. This indicates a convolution operation, used to compress channels; This represents the hyperbolic tangent function, used to constrain the pixel values ​​of the target image to the range [-1, 1].

[0208] These embodiments provide a more specific method for generating target images. By sorting features at each preset scale, a sorting result is obtained. This sorting result provides a fusion order for subsequent feature fusion at each scale. Based on the sorting result, features at each scale are fused sequentially to obtain a second fused feature. This allows for the gradual recovery of high-frequency details according to increasing resolution, avoiding image distortion caused by direct reconstruction. By determining the target image based on the first and second fused features, residual connections can be used to gradually fuse features at different scales, enhancing the recovery of high-frequency details and thus removing more image noise.

[0209] In a specific embodiment, the extraction of low-frequency features from the image to be processed in S101 includes the following sub-steps:

[0210] First, extract the initial features of the image to be processed; the initial features represent the initial low-frequency information of the image to be processed.

[0211] Second, based on the initial features, determine the first feature; the first feature represents the contrast information of the image to be processed.

[0212] Third, based on the initial features, determine the scaling parameter and the offset parameter; the scaling parameter represents the adjustment factor of the first feature, and the offset parameter represents the degree of offset of the first feature.

[0213] Fourth, determine the low-frequency features based on the initial features, the first feature, the scaling parameters, and the offset parameters.

[0214] Figure 4 This application provides a flowchart illustrating the process of extracting low-frequency features from an image to be processed. Figure 4 As shown, in this embodiment... Figure 1 Based on the embodiments, S101 will be described in detail. The process of extracting low-frequency features of the image to be processed includes:

[0215] S401. Extract the initial features of the image to be processed; the initial features represent the initial low-frequency information of the image to be processed.

[0216] For example, after acquiring the image to be processed, the initial features of the image to be processed can be extracted; the initial features characterize the initial low-frequency information of the image to be processed.

[0217] For example, filters or convolution operations can be used to extract initial features of the image to be processed, which represent the low-frequency information of the image.

[0218] S402. Based on the initial features, determine the first feature; the first feature represents the contrast information of the image to be processed.

[0219] For example, after obtaining the initial features, a first feature can be determined based on the initial features; the first feature represents the contrast information of the image to be processed.

[0220] For example, the first feature can be calculated according to the following formula (17):

[0221] (17);

[0222] In the formula, Indicates pixel position The feature value in the first feature represents the pixel position. Corresponding image contrast; Represents the pixel position in the initial feature Eigenvalues ​​at; In the initial features, the pixel position is used as the basis. Centered The mean of the window; In the initial features, the pixel position is used as the basis. Centered Variance of the window; A number greater than 0 is used to prevent division by zero in the formula. The feature values ​​corresponding to all pixel positions are combined to form the first feature value.

[0223] S403. Based on the initial features, determine the scaling parameter and the offset parameter; the scaling parameter represents the adjustment factor of the first feature, and the offset parameter represents the degree of offset of the first feature.

[0224] For example, scaling parameters and offset parameters can be determined based on initial features; the scaling parameter represents the adjustment factor of the first feature, and the offset parameter represents the degree of offset of the first feature. The scaling parameter is used to adjust the first feature by a factor, and the offset parameter is used to adjust the value of the first feature according to the magnitude of the offset.

[0225] For example, global average pooling can be performed on the initial features to compress spatial information and obtain global features. A trained multilayer perceptron then processes these global features to generate scaling and offset parameters. Since these parameters are obtained through global feature processing, they are global and can be dynamically adjusted to adjust the overall image content.

[0226] S404. Determine the low-frequency features based on the initial features, the first features, the scaling parameters, and the offset parameters.

[0227] For example, after obtaining the initial feature, the first feature, the scaling parameter, and the offset parameter, the low-frequency feature can be determined based on the initial feature, the first feature, the scaling parameter, and the offset parameter.

[0228] For example, the enhancement coefficient can be calculated based on the first feature, the scaling parameter, and the offset parameter. For instance, the enhancement coefficient can be calculated using the following formula (18):

[0229] (18);

[0230] In the formula, Indicates the enhancement coefficient; The function is used to limit the enhancement magnitude within the range of [-1, 1] to avoid extreme values ​​and ensure a natural enhancement. Indicates the scaling parameter; Indicates the offset parameter; This indicates the first characteristic.

[0231] After obtaining the enhancement coefficient, the low-frequency features can be determined based on the initial features and the enhancement coefficient. For example, the low-frequency features can be calculated according to the following formula (19):

[0232] (19);

[0233] In this embodiment, initial features of the image to be processed are extracted; the initial features represent the initial low-frequency information of the image to be processed; a first feature is determined based on the initial features; the first feature represents the contrast information of the image to be processed; if the contrast approaches zero, it represents a flat area of ​​the image; if the contrast is large, it represents an edge or texture area of ​​the image, thereby distinguishing between flat areas and edge areas of the image; scaling parameters and offset parameters are determined based on the initial features; the scaling parameter represents the adjustment factor of the first feature, and the offset parameter represents the offset degree of the first feature; low-frequency features are determined based on the initial features, the first feature, the scaling parameters, and the offset parameters, which can non-linearly enhance the initial features and further improve the recognition of objects in the image.

[0234] Figure 5 The image processing flow diagram in the image denoising method provided in this application is shown below. Figure 5 As shown, the image processing flow in the image denoising method includes:

[0235] After obtaining the image to be processed, the initial features and high-frequency features of the image to be processed are extracted, wherein the initial features are the initial low-frequency features.

[0236] The extracted initial features can be enhanced to obtain low-frequency features.

[0237] After obtaining the low-frequency features, the high-frequency features and low-frequency features are fused to obtain the first fused feature.

[0238] After obtaining the first fusion feature, a new first fusion feature can be obtained based on the first fusion feature.

[0239] After obtaining the new first fusion feature, the target image can be generated based on the new first fusion feature and the high-frequency feature.

[0240] In one specific embodiment, the above method further includes:

[0241] First, the image to be trained and the preset labels are input into the initial model to obtain the output image corresponding to the image to be trained; wherein, the initial model is the neural network model to be trained, the initial model is used for image denoising processing, and the preset labels represent the image to be trained without noise.

[0242] Second, based on the preset labels and the output image, and using a preset first loss function, determine the frequency band difference information; wherein, the frequency band difference information characterizes the magnitude of the difference between the preset labels and the output image within the preset frequency band.

[0243] Third, based on the preset labels and the output image, and using a preset second loss function, determine the texture difference information; whereby the texture difference information characterizes the magnitude of the texture difference between the preset labels and the output image.

[0244] Fourth, based on the frequency band difference information and texture difference information, the initial model is trained to obtain a trained image denoising model. This image denoising model is used to execute the method provided in the first embodiment above.

[0245] Figure 6 Flowchart of the image denoising method provided in this application Figure 4 ,like Figure 6 As shown, the method includes:

[0246] S601. Input the image to be trained and the preset label into the initial model to obtain the output image corresponding to the image to be trained; wherein, the initial model is the neural network model to be trained, the initial model is used for image denoising processing, and the preset label represents the image to be trained without noise.

[0247] For example, multiple training images and their corresponding preset labels can be obtained, where the training images represent noisy training images, and the preset labels represent noise-free training images. For any training image and its corresponding preset label, the training image and the preset label can be input into an initial model to obtain the output image corresponding to the training image; wherein, the initial model is a neural network model to be trained, and the initial model is used for image denoising processing.

[0248] The initial model may include a generator and a discriminator to be trained. The generator takes the image to be trained as input and outputs the corresponding output image. The discriminator distinguishes between the output image corresponding to the image with a preset label.

[0249] S602. Based on the preset label and the output image, determine the frequency band difference information according to the preset first loss function; wherein, the frequency band difference information represents the magnitude of the difference between the preset label and the output image within the preset frequency band.

[0250] For example, after obtaining the output image, frequency band difference information can be determined based on the preset label and the output image, and a preset first loss function; wherein, the frequency band difference information characterizes the magnitude of the difference between the preset label and the output image within the preset frequency band.

[0251] For example, if the initial model includes a generator and a discriminator to be trained, after obtaining the output image, the generator can determine the frequency band difference information based on the preset labels and the output image, and on the preset first loss function.

[0252] The frequency band difference information can be determined based on the following formula (20) of the preset first loss function:

[0253] (20);

[0254] In the formula, This represents the function value of the preset first loss function. The larger the function value of the preset first loss function, the greater the frequency band difference; the smaller the function value of the preset first loss function, the smaller the frequency band difference. Indicates the output image; Indicates the preset label; This represents a bandpass filter, where, The cutoff frequency is defined as [0.1, 0.9]. It means retaining 0.1-0.9 times the image frequency in the image and eliminating extreme frequencies outside the range. These extreme frequencies correspond to image noise and image artifacts.

[0255] S603. Based on the preset label and the output image, determine the texture difference information according to the preset second loss function; wherein, the texture difference information represents the magnitude of the texture difference between the preset label and the output image.

[0256] For example, after obtaining the output image, texture difference information can be determined based on the preset label and the output image, using a preset second loss function; wherein, the texture difference information characterizes the magnitude of the texture difference between the preset label and the output image.

[0257] For example, if the initial model includes a generator and a discriminator to be trained, after obtaining the output image, the generator and the discriminator can determine the texture difference information based on the preset labels and the output image, respectively, according to the preset second loss function.

[0258] Texture difference information can be determined based on the following formula (21) of the preset second loss function:

[0259] (twenty one);

[0260] In the formula, This represents the value of the preset second loss function. The larger the value of the preset second loss function, the greater the texture difference; the smaller the value of the preset second loss function, the smaller the texture difference. Indicates the discriminator; Represents the mathematical expectation; Represents a logarithmic function.

[0261] S604. Based on the frequency band difference information and texture difference information, train the initial model to obtain the trained image denoising model.

[0262] For example, after obtaining the frequency band difference information and texture difference information, the initial model can be trained to obtain a trained image denoising model.

[0263] Specifically, if the initial model includes a generator and a discriminator to be trained, the optimization objective for the generator can be determined based on the sum of the frequency band difference information and the texture difference information. For example, the optimization objective for the generator can be determined according to the following formula (22):

[0264] (twenty two);

[0265] In the formula, This represents the optimization target value for training the generator; Indicates the frequency band difference weight; This represents the texture difference weight.

[0266] For the discriminator, the optimization target for training can be determined based on texture differences.

[0267] During training, the generator minimizes the optimization objective determined by formula (22), while the discriminator maximizes the optimization objective determined by formula (21). The two are trained alternately to achieve dynamic game theory.

[0268] S605. Obtain the image to be processed, input the image to be processed into the image denoising model, and the image denoising model outputs the target image according to the method provided in the first embodiment. The target image is the image of the image to be processed after removing noise.

[0269] In this embodiment, by inputting the image to be trained and preset labels into the initial model, an output image corresponding to the image to be trained is obtained. Based on the preset labels and the output image, frequency band difference information is determined using a preset first loss function. This forces the output image to have a spectrum consistent with the preset labels within the effective frequency band, thereby suppressing high-frequency noise and low-frequency artifacts. Based on the preset labels and the output image, texture difference information is determined using a preset second loss function. This forces the generator to output an image with natural texture through a discriminator, avoiding blurring or artifacts. By training the initial model based on the frequency band difference information and texture difference information, a trained image denoising model is obtained. Stable adversarial training between the generator and the adversary can be achieved. The preset first loss function and the preset second loss function form a joint cooperative constraint, achieving a balance between noise suppression and detail preservation, thereby improving the image denoising effect of the trained image denoising model.

[0270] Figure 7 A schematic diagram of the training framework for the initial model provided in this application is shown below. Figure 7 As shown, the image to be trained is input into a dynamic scale-aware convolutional network. The dynamic scale-aware convolutional network outputs initial features and high-frequency features, where the initial features are the initial low-frequency features.

[0271] After outputting the initial features, the generated initial features are input into the dynamic contrast modulation network, which outputs low-frequency features.

[0272] After outputting the low-frequency features, the low-frequency features and high-frequency features are input into a multi-scale cross-attention network, which outputs the first fused feature.

[0273] After outputting the low-frequency features and the first fused features, the low-frequency features and the first fused features are input into the feature aggregation and joint constraint network. The feature aggregation and joint constraint network outputs the denoised image and adjusts the model parameters according to the value of the loss function.

[0274] Figure 8 A schematic diagram of the image denoising apparatus provided in this application is shown below. Figure 8 As shown, the image denoising device 80 provided in this embodiment includes:

[0275] The acquisition module 801 is used to acquire the image to be processed and extract the low-frequency features and high-frequency features of the image to be processed; the low-frequency features represent the low-frequency information of the image to be processed, and the high-frequency features represent the high-frequency information of the image to be processed.

[0276] The first determining module 802 is used to determine a first parameter corresponding to the preset scale based on the high-frequency features and the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0277] The second determining module 803 is used to determine a second parameter and a third parameter corresponding to the preset scale based on the low-frequency features and the preset scale; the second parameter represents the position of the object in the image to be processed represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed represented by the low-frequency features.

[0278] The third determining module 804 is used to determine the target image based on high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image to be processed after removing noise.

[0279] In one possible implementation, the first determining module 802 is further configured to:

[0280] The features corresponding to high-frequency features at a preset scale are called scale features; the preset scale represents the resolution size, and there are multiple preset scales.

[0281] Based on the scale features corresponding to the preset scale, the first parameter corresponding to the preset scale is determined; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

[0282] In one possible implementation, the third determining module 804 is further configured to:

[0283] The features corresponding to high-frequency features at a preset scale are called scale features; the preset scale represents the resolution size, and there are multiple preset scales.

[0284] The target image is determined based on the features at each scale and the first fusion feature.

[0285] In one possible implementation, the third determining module 804 is further configured to:

[0286] Based on the size of each preset scale, the features of each scale are sorted to obtain the sorting result;

[0287] Based on the ranking results, features at each scale are sequentially fused to obtain the second fused feature;

[0288] The target image is determined based on the first fusion feature and the second fusion feature.

[0289] In one possible implementation, the third determining module 804 is further configured to:

[0290] Based on the first parameter, the second parameter, and the third parameter at the preset scale, the fusion feature components at the preset scale are determined; the fusion feature components represent the information of high-frequency features and low-frequency features fused at the preset scale.

[0291] The first fusion feature is determined based on each fusion feature component.

[0292] In one possible implementation, the first fusion feature includes multiple feature values; the feature values ​​characterize features at preset locations in the image to be processed; the third determining module 804 is further configured to:

[0293] For each feature value, based on the first fused feature and a preset range, multiple target values ​​corresponding to the feature value are determined;

[0294] For each target value, a weight is determined based on the feature value and its corresponding preset position, as well as the target value and its corresponding preset position; the weight represents the magnitude of the influence of the target value on the feature value.

[0295] Based on each target value and its corresponding weight, the feature values ​​are adjusted to obtain new feature values.

[0296] Based on each new feature value, a new first fusion feature is determined;

[0297] The target image is determined based on high-frequency features and the new first fusion features.

[0298] In one possible implementation, the acquisition module 801 is also used for:

[0299] Extract initial features from the image to be processed; the initial features represent the initial low-frequency information of the image to be processed.

[0300] Based on the initial features, the first feature is determined; the first feature represents the contrast information of the image to be processed.

[0301] Based on the initial features, determine the scaling parameter and the offset parameter; the scaling parameter represents the adjustment factor of the first feature, and the offset parameter represents the degree of offset of the first feature;

[0302] Low-frequency features are determined based on initial features, first features, scaling parameters, and offset parameters.

[0303] In one possible implementation, the image denoising device 80 further includes a training module for:

[0304] The image to be trained and the preset labels are input into the initial model to obtain the output image corresponding to the image to be trained; wherein, the initial model is the neural network model to be trained, and the initial model is used for image denoising processing, and the preset labels represent the image to be trained without noise;

[0305] Based on the preset labels and the output image, frequency band difference information is determined using a preset first loss function; wherein, the frequency band difference information characterizes the magnitude of the difference between the preset labels and the output image within the preset frequency band;

[0306] Based on the preset labels and the output image, texture difference information is determined using a preset second loss function; wherein, texture difference information characterizes the magnitude of texture difference between the preset labels and the output image;

[0307] The initial model is trained based on the frequency band difference information and texture difference information to obtain the trained image denoising model;

[0308] The image denoising model is used to perform the image denoising method as described in the first embodiment.

[0309] The image denoising device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0310] Figure 9 This is a schematic diagram of the image denoising device provided in this application. Figure 9 As shown, the image denoising device 90 provided in this embodiment includes at least one processor 901 and a memory 902. Optionally, the image denoising device 90 further includes a communication component 903. The processor 901, memory 902, and communication component 903 are connected via a bus.

[0311] In a specific implementation, at least one processor 901 executes computer execution instructions stored in memory 902, causing at least one processor 901 to perform the above-described method.

[0312] The specific implementation process of processor 901 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0313] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0314] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0315] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0316] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0317] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0318] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0319] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0320] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0321] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0322] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0323] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0324] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0325] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image denoising method, characterized in that, include: Acquire the image to be processed, and extract the low-frequency and high-frequency features of the image to be processed; The low-frequency features characterize the low-frequency information of the image to be processed, and the high-frequency features characterize the high-frequency information of the image to be processed. Based on the high-frequency characteristics and the preset scale, determine the first parameter corresponding to the preset scale; The first parameter represents the location of the object in the image to be processed as represented by the high-frequency features; Based on the low-frequency features and the preset scale, a second parameter and a third parameter corresponding to the preset scale are determined; the second parameter represents the position of the object in the image to be processed as represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed as represented by the low-frequency features. The target image is determined based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image of the image to be processed after removing noise.

2. The method according to claim 1, characterized in that, The step of determining the first parameter corresponding to the preset scale based on the high-frequency features and the preset scale includes: The high-frequency feature is determined to correspond to the feature at a preset scale, which is called the scale feature; the preset scale represents the resolution size, and there are multiple preset scales; Based on the scale features corresponding to the preset scale, a first parameter corresponding to the preset scale is determined; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features.

3. The method according to claim 1, characterized in that, The step of determining the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale includes: Based on the first parameter, second parameter, and third parameter corresponding to each preset scale, a first fusion feature is determined; the first fusion feature represents the fusion of information from high-frequency features and low-frequency features at each preset scale; The target image is determined based on the high-frequency features and the first fusion feature.

4. The method according to claim 3, characterized in that, Determining the target image based on the high-frequency features and the first fused features includes: The high-frequency feature is determined to correspond to the feature at a preset scale, which is called the scale feature; the preset scale represents the resolution size, and there are multiple preset scales; The target image is determined based on the features at each scale and the first fusion feature.

5. The method according to claim 4, characterized in that, The step of determining the target image based on the features at each scale and the first fused feature includes: Based on the size of each preset scale, the features of each scale are sorted to obtain the sorting result; Based on the sorting results, features at each scale are sequentially fused to obtain the second fused feature; The target image is determined based on the first fusion feature and the second fusion feature.

6. The method according to claim 3, characterized in that, The step of determining the first fusion feature based on the first parameter, second parameter, and third parameter corresponding to each preset scale includes: Based on the first parameter, the second parameter, and the third parameter at the preset scale, the fused feature components at the preset scale are determined; the fused feature components represent the information of high-frequency features and low-frequency features fused at the preset scale. The first fusion feature is determined based on each fusion feature component.

7. The method according to claim 3, characterized in that, The first fusion feature includes multiple feature values; The feature value represents the feature at a preset location in the image to be processed; Determining the target image based on the high-frequency features and the first fused features includes: For each feature value, based on the first fused feature and a preset range, multiple target values ​​corresponding to the feature value are determined. For each target value, a weight is determined based on the feature value and its corresponding preset position, as well as the target value and its corresponding preset position; the weight represents the magnitude of the influence of the target value on the feature value. The feature values ​​are adjusted according to each target value and the weight corresponding to the target value to obtain new feature values; Based on each new feature value, a new first fusion feature is determined; The target image is determined based on the high-frequency features and the new first fusion features.

8. The method according to claim 1, characterized in that, The extraction of low-frequency features from the image to be processed includes: Extract initial features from the image to be processed; the initial features characterize the initial low-frequency information of the image to be processed; Based on the initial features, a first feature is determined; the first feature characterizes the contrast information of the image to be processed. Based on the initial features, scaling parameters and offset parameters are determined; the scaling parameters represent the adjustment factor of the first feature, and the offset parameters represent the degree of offset of the first feature. The low-frequency feature is determined based on the initial feature, the first feature, the scaling parameter, and the offset parameter.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: The image to be trained and the preset labels are input into the initial model to obtain the output image corresponding to the image to be trained; wherein, the initial model is a neural network model to be trained, the initial model is used to perform image denoising processing, and the preset labels represent the image to be trained without noise; Based on the preset label and the output image, frequency band difference information is determined using a preset first loss function; wherein, the frequency band difference information characterizes the magnitude of the difference between the preset label and the output image within a preset frequency band; Based on the preset label and the output image, texture difference information is determined using a preset second loss function; wherein, the texture difference information characterizes the magnitude of the texture difference between the preset label and the output image; The initial model is trained based on the frequency band difference information and the texture difference information to obtain a trained image denoising model. The image denoising model is used to perform the image denoising method as described in claim 1.

10. An image denoising apparatus, characterized in that, include: The acquisition module is used to acquire the image to be processed and extract the low-frequency and high-frequency features of the image to be processed. The low-frequency features characterize the low-frequency information of the image to be processed, and the high-frequency features characterize the high-frequency information of the image to be processed. The first determining module is used to determine a first parameter corresponding to the preset scale based on the high-frequency features and the preset scale; the first parameter represents the position of the object in the image to be processed as represented by the high-frequency features. The second determining module is used to determine a second parameter and a third parameter corresponding to the preset scale based on the low-frequency features and the preset scale; the second parameter represents the position of the object in the image to be processed represented by the low-frequency features; the third parameter represents the image details of the object in the image to be processed represented by the low-frequency features. The third determining module is used to determine the target image based on the high-frequency features and the first, second, and third parameters corresponding to each preset scale; the target image is the image of the image to be processed after removing noise.

11. An image denoising device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images