Noise image generation model training, denoising model training and denoising method

Abstract

The present invention relates to the field of image processing, and provides a noise image generation model training, denoising model training and denoising method. Among them, the noise image generation model training method includes: obtaining random data and actual noise images; inputting the random data into a first generator to obtain simulated noise; inputting the actual noise image into a denoiser to obtain a denoised image; inputting the simulated noise and the denoised image into a synthesis module to obtain a synthetic noise image; inputting the synthetic noise image and the actual noise image into a first discriminator to obtain a first discrimination result; according to the first discrimination result, the first generator, denoiser and first discriminator in the first noise image generation model are trained to obtain a second noise image generation model, and the second noise image generation model is used to synthesize noise images. Through the present invention, a training noise image in an N2N model is generated to enhance the denoising effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a noise image generation model training, a denoising model training and a denoising method. Background Art

[0002] In recent years, inspection robots have been gradually adopted for data center inspections, effectively addressing the low efficiency and high costs of traditional manual inspections and improving intelligent operations and maintenance management. However, electromagnetic interference generated by the large number of electronic devices and high and low voltage power transmission lines within data centers can cause varying degrees of noise in the images captured by the robot's imaging equipment, resulting in loss of image edges and details. This can reduce the recognition rate and positioning accuracy of subsequent visual recognition and positioning tasks. Therefore, to enhance the interference resistance of the robot's vision system, it is particularly important to implement image denoising preprocessing to improve image quality.

[0003] Existing techniques use the N2N (Noise2Noise) model to denoise noisy images. However, during training, the N2N model requires input of paired noisy images of inspected objects. These two images must meet two requirements: they must originate from the same clean image, and the noise in the noisy images must be independently distributed. However, in data center semiconductor packaging scenarios, clean images are difficult to obtain, making it difficult for training images to meet these two requirements. Consequently, the trained N2N model performs poorly in denoising. Summary of the Invention

[0004] In order to generate a training noise image for the N2N model and enhance the denoising effect, the present invention proposes a noise image generation model training, a denoising model training and a denoising method.

[0005] In a first aspect, the present invention provides a method for training a noise image generation model, wherein the first noise image generation model includes a first generator, a denoiser, a first discriminator and a synthesis module, and the method includes:

[0006] Get random data and actual noise images;

[0007] Inputting random data into the first generator to obtain simulated noise;

[0008] Input the actual noisy image into the denoiser to obtain a denoised image;

[0009] Inputting the simulated noise and denoised image into a synthesis module to obtain a synthetic noise image;

[0010] Inputting the synthetic noise image and the actual noise image into a first discriminator to obtain a first discrimination result;

[0011] According to the first discrimination result, the first generator, the denoiser and the first discriminator in the first noise image generation model are trained to obtain a second noise image generation model, and the second noise image generation model is used to synthesize the noise image.

[0012] Through the above method, the simulated noise generated by the first generator is generated based on random data. Therefore, the obtained simulated noise is independently distributed. At the same time, the denoised image obtained by the denoiser can be generated by the same actual noise image, which is equivalent to coming from the same clean image. Therefore, the synthetic noise image obtained based on the simulated noise and the denoised image can meet the requirements of the N2N model for training images.

[0013] In an optional embodiment, the first noise image generation model further includes a second generator and a second discriminator. According to the first discrimination result, the first generator, denoiser and first discriminator in the first noise image generation model are trained to obtain a second noise image generation model, including:

[0014] According to the actual noise image, the actual noise is obtained;

[0015] Inputting the simulated noise and the actual noise into the second discriminator to obtain a second discrimination result;

[0016] Input the denoised image into the second generator to obtain a simulated noise image;

[0017] Calculating a first loss function value according to the first discrimination result, the second discrimination result, the simulated noise image, and the actual noise image;

[0018] Adjust the parameters in the first generator, the parameters in the denoiser, the parameters in the second generator, the parameters in the first discriminator and the parameters in the second discriminator, and recalculate the first loss function value until the first loss function value no longer decreases, thereby obtaining a second noisy image generation model.

[0019] In an optional embodiment, calculating a first loss function value according to the first discrimination result, the second discrimination result, the simulated noise image, and the actual noise image includes:

[0020] Calculate the second loss function value according to the first discrimination result;

[0021] Calculate the third loss function value according to the second discrimination result;

[0022] Calculating a fourth loss function value according to the actual noise image and the simulated noise image;

[0023] Calculate the first loss function value based on the second loss function value, the third loss function value, and the fourth loss function value.

[0024] Through the above implementation, the second loss function value is calculated based on the first discrimination result, and the denoiser, the second generator and the first discriminator can be trained according to the second loss function value. The third loss function value is calculated based on the second discrimination result, and the first generator and the second discriminator can be trained according to the third loss function value. The fourth loss function value is calculated based on the actual noise image and the simulated noise image, and the fourth loss function value can be used to train the denoiser and the second generator.

[0025] In an optional embodiment, calculating the first loss function value according to the second loss function value, the third loss function value, and the fourth loss function value includes:

[0026] The second loss function value, the third loss function value and the fourth loss function value are weighted and summed to obtain the first loss function value, and the weight value of the second loss function value, the weight value of the third loss function value and the weight value of the fourth loss function value decrease in sequence.

[0027] Through the above implementation, since the denoiser, the second generator and the first discriminator are trained according to the second loss function value, and the denoiser, the second generator and the first discriminator are the main network components of the noise image generation model, the weight of the second loss function value is the largest. At the same time, considering that the fourth loss function value simultaneously trains the denoiser and the second generator, and the denoiser and the second generator are equivalent to the encoder and the decoder, training the denoiser and the second generator at the same time may cause model instability, so the weight of the fourth loss function value is the smallest.

[0028] In an optional implementation, obtaining actual noise according to the actual noise image includes:

[0029] According to the actual noise image, multiple image blocks are obtained;

[0030] Performing a two-dimensional discrete wavelet transform on each image block to obtain a scale coefficient image and a wavelet coefficient image corresponding to each image block;

[0031] The image blocks where the scale coefficient image and the wavelet coefficient image meet the preset conditions are regarded as noise areas;

[0032] The actual noise is obtained by subtracting each pixel in the noise area from the average pixel in the noise area.

[0033] Through the above implementation, since the two-dimensional discrete wavelet transform belongs to the space-frequency transform, the wavelet transform of the signal can obtain the time-frequency spectrum, and the multi-scale decomposition of the image can be achieved by translation and scaling, etc., which can accurately separate the effective signal and noise, so that the actual noise extracted can more accurately reflect the real spatial distribution of the noise. In addition, the Gibbs effect will not occur when processing sudden changes such as sharp edges that are more common in the images of inspected objects.

[0034] In an optional embodiment, the denoiser includes at least one inverted bottleneck block, the inverted bottleneck block includes a plastic convolution layer, the plastic convolution layer includes a depthwise separable convolution layer, and the actual noisy image is input into the denoiser to obtain a denoised image, including:

[0035] According to the actual noise image and the depth-separable convolution layer, the first feature map is obtained;

[0036] A denoised image is obtained according to the first feature map and the actual noise image.

[0037] Through the above implementation, different convolution kernels in the depthwise separable convolution in the plastic convolution layer are used to capture the contextual information in the image, which is more suitable for processing images containing spatially heterogeneous patterns.

[0038] In an optional embodiment, the plastic convolution layer further includes a pooling layer, a predictor network, an upsampling layer, and a dynamic convolution layer, and a first feature map is obtained according to the actual noise image and the depth-separable convolution layer, including:

[0039] Performing a shuffling operation on the actual noise image to obtain an actual noise image after the operation;

[0040] The actual noise image after the operation is input into the pooling layer to obtain the second feature map;

[0041] Input the second feature map into the predictor network to obtain the third feature map;

[0042] Inputting the third feature map into a depth-wise separable convolutional layer to obtain a plurality of first filters;

[0043] Inputting each first filter into an upsampling layer to obtain a plurality of second filters;

[0044] Each second filter and the actual noise image after the operation are input into the dynamic convolution layer to obtain the first feature map.

[0045] Through the above implementation, the second feature obtained by the pooling layer is used as the input of the predictor network, which expands the receptive field of the predictor network, while reducing the computational cost and storage cost and improving the denoising speed.

[0046] In a second aspect, the present invention further provides a denoising model training method, the method comprising:

[0047] Get random data and actual noise images;

[0048] Generate a first noise image pair according to the random data and the actual noise image, and a first noise generation model, where the first noise generation model is obtained by the noise image generation model training method according to the first aspect or any embodiment of the first aspect;

[0049] The pre-built initial N2N denoising model is trained according to the first noisy image pair to obtain an N2N denoising model.

[0050] Through the above method, the first noise image pair generated by the first noise generation model comes from the same clean image, and the noise in the first noise image pair is independently distributed. That is, the first noise image pair meets the requirements of the N2N model for training images. Training the N2N model based on the first noise image pair can make the denoising performance of the trained N2N model better.

[0051] In an optional embodiment, the initial N2N denoising model includes a plastic convolution layer, which includes a pooling layer, a predictor network, a depth-wise separable convolution layer, an upsampling layer, and a dynamic convolution layer.

[0052] In an optional embodiment, a pre-built initial N2N denoising model is trained according to the first noisy image pair to obtain an N2N denoising model, including:

[0053] Training a pre-built initial N2N denoising model according to the first noisy image pair to obtain a first N2N denoising model;

[0054] The denoiser in the first noise generation model is replaced by the first N2N denoising model to obtain a third noise image generation model;

[0055] generating a second noise image pair according to the random data, the actual noise image, and a third noise image generation model;

[0056] Calculating a fifth loss function value based on the first noise image pair and the second noise image pair;

[0057] The parameters in the first N2N denoising model are adjusted according to the fifth loss function value, and the fifth loss function value is recalculated until the fifth loss function value no longer decreases, thereby obtaining the N2N denoising model.

[0058] Through the above implementation, the N2N denoising model is adjusted and trained according to the calculated fifth loss function value to improve the denoising performance of the N2N denoising model.

[0059] In a third aspect, the present invention further provides a denoising method, the method comprising:

[0060] Obtain the image to be denoised;

[0061] The image to be denoised is input into the N2N denoising model to obtain an image after denoising, and the N2N denoising model is trained by the denoising model training method of the second aspect or any embodiment of the second aspect.

[0062] Through the above implementation, the N2N denoising model is obtained by training the first noise image pair corresponding to the same clean image, and the noise in the first noise image pair is independently distributed, that is, the first noise image pair meets the requirements of the N2N model for training images, and the denoising performance of the N2N model trained based on the first noise image pair is better.

[0063] In a fourth aspect, the present invention further provides a noise image generation model training device, the device comprising:

[0064] A first acquisition module is used to acquire random data and actual noise images;

[0065] A first generating module is used to input random data into a first generator to obtain simulated noise;

[0066] A denoising module is used to input the actual noisy image into the denoiser to obtain a denoised image;

[0067] A synthesis module, configured to input the simulated noise and the denoised image into the synthesis module to obtain a synthesized noise image;

[0068] a discrimination module, configured to input the synthetic noise image and the actual noise image into a first discriminator to obtain a first discrimination result;

[0069] The first training module is used to train the first generator, denoiser and first discriminator in the first noise image generation model according to the first discrimination result to obtain a second noise image generation model, and the second noise image generation model is used to synthesize the noise image.

[0070] Through the above device, the simulated noise generated by the first generator is generated based on random data. Therefore, the obtained simulated noise is independently distributed. At the same time, the denoised image obtained by the denoiser can be generated by the same actual noise image, which is equivalent to coming from the same clean image. Therefore, the synthetic noise image obtained based on the simulated noise and the denoised image can meet the requirements of the N2N model for training images.

[0071] In a fifth aspect, the present invention further provides a denoising model training device, comprising:

[0072] A second acquisition module is used to acquire random data and actual noise images;

[0073] a second generation module, configured to generate a first noise image pair based on the random data and the actual noise image, and a first noise generation model, where the first noise generation model is obtained by the noise image generation model training method according to the first aspect or any embodiment of the first aspect;

[0074] The second training module is used to train the pre-built initial N2N denoising model according to the first noise image pair to obtain the N2N denoising model.

[0075] Through the above-mentioned device, the first noise image pair generated by the first noise generation model comes from the same clean image, and the noise in the first noise image pair is independently distributed. That is, the first noise image pair meets the requirements of the N2N model for training images. Training the N2N model based on the first noise image pair can make the denoising performance of the trained N2N model better.

[0076] In a sixth aspect, the present invention also provides a computer device comprising a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the computer instructions to thereby execute the noise image generation model training method of the first aspect or any embodiment of the first aspect, or the denoising model training method of the second aspect or any embodiment of the second aspect, or the steps of the denoising method of the third aspect or any embodiment of the third aspect.

[0077] In the seventh aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the hydrological model correction method of the first aspect or any embodiment of the first aspect, or the denoising model training method of the second aspect or any embodiment of the second aspect, or the steps of the denoising method of the third aspect or any embodiment of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0079] Figure 1 is a flowchart of a noise image generation model training method proposed according to an exemplary embodiment;

[0080] Figure 2 is a network structure diagram of a first noise image generation model in an example;

[0081] Figure 3This is a schematic diagram of the structure of a plastic convolutional layer in an example;

[0082] Figure 4 is a schematic diagram of the structure of a denoiser in an example;

[0083] Figure 5 is a two-dimensional discrete wavelet transform flow chart in one example;

[0084] Figure 6 is a flowchart of a denoising model training method proposed according to an exemplary embodiment;

[0085] Figure 7 is a diagram showing the working process of a denoising model training method in an example;

[0086] Figure 8 is a flow chart of a denoising method proposed according to an exemplary embodiment;

[0087] Figure 9 is a visualization of a noisy chip image in one example;

[0088] Figure 10 is an enlarged image of a noise chip image in one example;

[0089] Figure 11 This is the visualization result of training once in an example;

[0090] Figure 12 This is the amplification result of training once in an example;

[0091] Figure 13 This is the visualization result of training 6 times in one example;

[0092] Figure 14 This is the amplified result when training 6 times in one example;

[0093] Figure 15 In one example, the zoomed-in result of Canny edge detection on a noisy chip image is shown.

[0094] Figure 16 This is the enlarged result of Canny edge detection when training once in an example;

[0095] Figure 17 This is an example of a zoomed-in Canny edge detector when trained 6 times.

[0096] Figure 18 Schematic diagram of a noise image generation model training device according to an exemplary embodiment;

[0097] Figure 191 is a structural diagram of a denoising model training device proposed according to an exemplary embodiment;

[0098] Figure 20 1 is a schematic structural diagram of a denoising device according to an exemplary embodiment;

[0099] Figure 21 The figure is a schematic diagram of the hardware structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0100] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0101] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0102] The N2N model is based on the principle that multiple sets of noisy observations can predict real signals. It exploits the principle that the expected value of the noise input equals the clean signal to learn the mapping between pairs of independently degraded versions of the same training image. The N2N model attempts to learn the mapping between pairs of independently degraded versions of the same clean image. Training requires input of pairs of noisy images (x+N1, x+N2) of inspected objects. These two noisy images must simultaneously meet two requirements: they must originate from the same clean image x, and the noise N1 and N2 in the noisy images must follow independent distributions. However, in semiconductor packaging scenarios, it is difficult to obtain such pairs with a (quasi-) constant x, making it impossible to obtain image pairs that meet these requirements. This results in poor denoising performance for the N2N model.

[0103] In order to generate training noise images in the N2N model and enhance the denoising effect, the present invention proposes a noise image generation model training, a denoising model training and a denoising method.

[0104] Figure 1 The present invention is a flowchart of a noise image generation model training method proposed according to an exemplary embodiment. Figure 2 This is the network structure diagram of the first noise image generation model. The first noise image generation model includes a first generator 1, a denoiser 2, a first discriminator 3 and a synthesis module 4. Figure 1 As shown, the noise image generation model training method includes the following steps S101 to S106.

[0105] Step S101: Acquire random data and actual noise image.

[0106] In an optional embodiment, the actual noise image can be obtained by randomly cropping the acquired inspection image.

[0107] In an optional embodiment, the random data may be data that obeys a normal distribution. For example, the random data

[0108]

[0109] Step S102: inputting random data into the first generator to obtain simulated noise.

[0110] In an optional embodiment, the first generator is composed of a deconvolution layer, a normalization layer (Batch Norm, BN) and a ReLU activation function.

[0111] Step S103: Input the actual noisy image into the denoiser to obtain a denoised image.

[0112] In an optional embodiment, the denoiser is composed of an inverted bottleneck block, wherein the convolutional layer in the inverted bottleneck block is a depthwise separable convolution.

[0113] In an optional embodiment, the activation function in the denoiser may be a sigmoid activation function.

[0114] Step S104: inputting the simulated noise and the denoised image into a synthesis module to obtain a synthesized noise image.

[0115] In an optional embodiment, the simulated noise and the denoised image are added pixel by pixel to obtain a synthetic noise image.

[0116] Step S105: Input the synthetic noise image and the actual noise image into the first discriminator to obtain a first discrimination result.

[0117] In an optional embodiment, the first discriminator is used to determine whether the input image is a synthetic noise image or an actual noise image, thereby improving the authenticity of the synthetic noise image.

[0118] In an optional embodiment, the first discriminator is composed of a standard convolutional layer, a normalization layer (Batch Norm, BN) and an LReLU activation function.

[0119] Step S106: According to the first discrimination result, the first generator, the denoiser and the first discriminator in the first noise image generation model are trained to obtain a second noise image generation model, and the second noise image generation model is used to synthesize the noise image.

[0120] Through the above method, the simulated noise generated by the first generator is generated based on random data. Therefore, the obtained simulated noise is independently distributed. At the same time, the denoised image obtained by the denoiser can be generated by the same actual noise image, which is equivalent to coming from the same clean image. Therefore, the synthetic noise image obtained based on the simulated noise and the denoised image can meet the requirements of the N2N model for training images.

[0121] The related image denoising methods use standard convolutional layers to extract image features. The implementation process of the standard convolutional layer is to convert the weights The convolution kernel is applied to the two-dimensional input feature map Where H and W represent the height and width of the feature map, C in with C out Represents the number of input and output channels, and K represents the size of the convolution kernel. At each output position, a C-sized image around the position is collected from the input feature map. in ×K 2 The local block is then multiplied by the convolution kernel weight matrix W. The disadvantage of this method is that all output pixels share the same convolution kernel, which makes it difficult to capture contextual information and cannot adapt to target images containing spatially heterogeneous patterns.

[0122] In one example, a denoiser includes at least one inverted bottleneck block, which includes a plastic convolutional layer, which includes a depthwise separable convolutional layer. The depthwise separable convolutions in the plastic convolutional layer capture contextual information in the image using different convolution kernels, making it more suitable for processing images containing spatially heterogeneous patterns.

[0123] In the above step S103, the denoised image is obtained by the following steps:

[0124] Step a1: Obtain the first feature map based on the actual noise image and the depth-separable convolutional layer.

[0125] Step a2: Obtain a denoised image based on the first feature map and the actual noise image.

[0126] like Figure 3 As shown in Figure 2, in addition to the depthwise separable convolution, the plastic convolution layer also includes a pooling layer, a predictor network, an upsampling layer, and a dynamic convolution layer. In step a1 above, the first feature map is obtained by the following steps:

[0127] Step b1: Perform a shuffling operation on the actual noise image to obtain the actual noise image after the operation. For example, a 2*shuffling operation from space to channel can be performed to obtain the actual noise image after the operation.

[0128] Step b2: Input the actual noise image after the operation into the pooling layer to obtain the second feature map. In this embodiment of the present invention, in the pooling layer, the actual noise image Perform 4*4 average pooling operation and downsample to Through downsampling operations, the computational cost is reduced and the receptive field is increased.

[0129] Step b3: Input the second feature map into the predictor network to obtain a third feature map. In an embodiment of the present invention, the second feature map obtained in step b2 is input into a lightweight predictor network composed of a residual network and a maximum pooling operation, and the predictor network outputs a feature two-dimensional discrete wavelet transform map (the third feature map)

[0130] Step b4: Input the third feature map into the depthwise separable convolution layer to obtain a plurality of first filters. In this embodiment of the present invention, the third feature map is transformed into a series of filters by the depthwise separable convolution layer. where i∈{1,2,…,H / 8}, j∈{1,2,…,W / 8}, that is, by using depthwise separable convolution, each convolution kernel in Y is transformed into C channels.

[0131] Step b5: Input each first filter into the upsampling layer to obtain multiple second filters. In the embodiment of the present invention, the first filter is upsampled by bilinear interpolation to obtain a pixel-by-pixel filter (second filter) Among them, i∈{1,2,…,H}, j∈{1,2,…,W}.

[0132] Step b6: Input each second filter and the actual noise image after the operation into the dynamic convolution layer to obtain the first feature map.

[0133] In an embodiment of the present invention, the second feature map obtained by the pooling layer is used as the input of the predictor network, which expands the receptive field of the predictor network, reduces the computational cost and storage cost, and improves the denoising speed.

[0134] Figure 4 It is a schematic diagram of the denoiser structure. Figure 4In this denoiser, the spatial pyramid method and the inverted bottleneck block are combined. The inverted bottleneck block includes the plastic convolution layer in the above embodiment. In this denoiser, a space-to-channel 2* shuffle operation is first used to construct a pyramid with a preset number of layers; then, in each layer of the pyramid, several inverted bottleneck blocks are stacked in a preset ratio, which are composed of multiple standard convolutions and depthwise convolutions, and a 1×1 plastic convolution layer or a 3×3 plastic convolution layer is inserted between them to extract the feature information of the image; at the end of the bottom pyramid, the feature map is upsampled and connected to the input of the previous stage; in the top pyramid, the features extracted from the next layer of the pyramid are fused as the final network output. Compared with the traditional encoder-decoder structure, the structure based on the spatial pyramid reuses the features extracted from each scale, which can achieve faster model inference speed. Table 1 shows the specific network structure of the denoiser.

[0135] Table 1 Specific network structure of denoiser

[0136]

[0137] exist Figure 2 In the first noisy image generation model, the second generator and the second discriminator are also included. In this embodiment of the present invention, the second generator is a generator based on a plastic convolutional layer, and the network structure of the second generator is the same as that of the denoiser. The activation function in the second generator is a linear activation function. The specific network structures of the second discriminator and the first discriminator are shown in Table 2.

[0138] Table 2 Specific network structures of the second discriminator and the first discriminator

[0139]

[0140] In the above step S106, the second noise image generation model is obtained through the following steps:

[0141] Step c1: Obtain actual noise according to the actual noise image.

[0142] In an optional embodiment, two-dimensional discrete wavelet transform may be used to obtain actual noise from the actual noise image.

[0143] Step c2: Input the simulated noise and the actual noise into the second discriminator to obtain a second discrimination result.

[0144] In an optional embodiment, the second discriminator is composed of a standard convolutional layer, a normalization layer (Batch Norm, BN) and an LReLU activation function.

[0145] In an optional embodiment, the second discriminator is used to discriminate whether the input image is simulated noise or actual noise, thereby improving the realism of the simulated noise generated by the first generator.

[0146] Step c3: Input the denoised image into the second generator to obtain a simulated noise image.

[0147] Step c4: Calculate a first loss function value based on the first discrimination result, the second discrimination result, the simulated noise image, and the actual noise image.

[0148] Step c5: Adjust the parameters in the first generator, the parameters in the denoiser, the parameters in the second generator, the parameters in the first discriminator and the parameters in the second discriminator, and recalculate the first loss function value until the first loss function value no longer decreases, thereby obtaining a second noisy image generation model.

[0149] In an optional embodiment, in the above step c1, obtaining specific content of actual noise according to the actual noise image includes:

[0150] First, a plurality of image blocks are obtained based on the actual noise image. In the embodiment of the present invention, a sliding window with a length of d×d and a step size of S1 is used to traverse the actual noise image to obtain an image block set P = {p1, p2, p3, ..., p m}.

[0151] Secondly, a two-dimensional discrete wavelet transform is performed on each image block to obtain a scale coefficient image and a wavelet coefficient image corresponding to each image block.

[0152] Again, the image blocks whose scale coefficient image and wavelet coefficient image meet the preset conditions are regarded as noise areas. In the embodiment of the present invention, the preset conditions are set as:

[0153]

[0154] Among them, p i is the i-th image block, W k are the scale coefficient image and wavelet coefficient image, Represents the standard deviation of the wavelet coefficients, λ∈(0,1). An image block that meets the above preset conditions can indicate that the standard deviation of each sub-image coefficient in the image block is not far from their average value, so p can be determined i The noise area.

[0155] Finally, the actual noise is obtained by subtracting each pixel in the noise region from the average pixel in the noise region. Since most noise has an additive zero mean, the actual noise can be obtained by subtracting each pixel in the image block that meets the above pre-set conditions from the average pixel in the image block.

[0156] Through the above implementation, since the two-dimensional discrete wavelet transform is a space-frequency transform, the time-frequency spectrum can be obtained by performing wavelet transform on the signal, and multi-scale decomposition of the image can be achieved by translation and scaling, etc., which can accurately separate the effective signal and noise, so that the actual noise extracted can more accurately reflect the real spatial distribution of the noise, and avoid the Gibbs effect when processing sudden changes such as sharp edges.

[0157] Since the scaling parameter a, translation parameter b, and time t in the Continuous Wavelet Transform (CWT) are all continuous, for the convenience of calculation, the above parameters are discretized, namely discrete wavelet transform (DWT). Therefore, the image can be represented as:

[0158]

[0159]

[0160]

[0161] ψ D (x,y)=ψ(x)ψ(y)

[0162] Among them, x and y represent the coordinates of the x-axis and y-axis in a point continuous image respectively. is the scaling function, ψ is the conversion basis function, ψ V Corresponding to changes along the line (e.g. vertical edge), ψ H Corresponding to changes along the column (e.g. horizontal edges), ψ D Corresponding to the change of diagonal line.

[0163] The scaling function and conversion basis function are defined as follows:

[0164]

[0165]

[0166] Wherein, index superscript i = {V, H, D}, j is the scale factor, and m and n are the length and width respectively.

[0167] Therefore, for an image f(x,y) of size M×N, its two-dimensional discrete wavelet transform is defined as:

[0168]

[0169]

[0170] Where f(x,y) is an image of size M×N, j0 is an arbitrary starting scale that is set to 0 by default, The coefficient is an approximation of f(x,y) at scale j0. The coefficient j≥j0 adds horizontal, vertical and diagonal details. The original two-dimensional image can be decomposed into four sub-images: 1 scale coefficient containing low-pass information and 3 wavelet coefficients corresponding to the vertical, horizontal and diagonal details respectively and Figure 5 is a two-dimensional discrete wavelet transform flow chart, where h ψ represents low-pass filtering, For high-pass filtering. Represents a convolution operation.

[0171] In an optional embodiment, in the above step c4, the first loss function value is calculated as follows:

[0172] First, based on the first discrimination result, the second loss function value is calculated. In an embodiment of the present invention, the second loss function value is calculated based on the first discrimination result, and the denoiser, the first generator, and the first discriminator can be trained based on the second loss function value. The formula for calculating the second loss function value is expressed as follows:

[0173]

[0174] in, is the second loss function value; is the first discriminator; is a denoiser; is the first generator; Z is the actual noise image; r is random data.

[0175] Secondly, based on the second discrimination result, the third loss function value is calculated. In the embodiment of the present invention, the third loss function value is calculated based on the second discrimination result, and the first generator and the second discriminator can be trained based on the third loss function value. The formula for calculating the third loss function value is expressed as follows:

[0176]

[0177] in, is the value of the third loss function; n is the actual noise; is the second discriminator; is the first generator; For distribution obey The sampling in , λ is the preset weight.

[0178] Again, a fourth loss function value is calculated based on the actual noise image and the simulated noise image. In an embodiment of the present invention, the fourth loss function value is calculated based on the actual noise image and the simulated noise image, and the fourth loss function value can be used to train the denoiser and the second generator. The formula for calculating the fourth loss function value is as follows:

[0179]

[0180] in, is the value of the fourth loss function; Z is the actual noise image; is a denoiser; This loss function can effectively avoid the problem of multiple source domain data with different contents being mapped into target domain data with the same content. Therefore, it can effectively remove some redundant noise in the actual noisy image and improve the denoising performance.

[0181] Finally, the first loss function value is calculated based on the second loss function value, the third loss function value, and the fourth loss function value. In an embodiment of the present invention, the calculation formula of the first loss function value is expressed as follows:

[0182]

[0183] Among them, α, β, and γ are the weights of the third loss function value, the second loss function value, and the fourth loss function value, respectively.

[0184] In an optional embodiment, the second loss function value, the third loss function value, and the fourth loss function value are weighted and summed to obtain the first loss function value, and the weight values ​​of the second loss function value, the third loss function value, and the fourth loss function value decrease in sequence. Since the denoiser, the second generator, and the first discriminator are trained according to the second loss function value, and the denoiser, the second generator, and the first discriminator are the main network components of the noise image generation model, the weight of the second loss function value is the largest. At the same time, considering that the fourth loss function value simultaneously trains the denoiser and the second generator, and the denoiser and the second generator are equivalent to the encoder and the decoder, training the denoiser and the second generator at the same time may cause model instability, so the weight of the fourth loss function value is the smallest.

[0185] Figure 6 4 is a flowchart of a denoising model training method according to an exemplary embodiment. The method includes the following steps S601 to S603.

[0186] Step S601: Acquire random data and actual noise image.

[0187] Step S602: Generate a first noise image pair according to the random data, the actual noise image, and a first noise generation model. The first noise generation model is obtained by the noise image generation model training method in the above embodiment.

[0188] Step S603: training the pre-built initial N2N denoising model according to the first noisy image pair to obtain an N2N denoising model.

[0189] Through the above method, the first noise image pair generated by the first noise generation model comes from the same clean image, and the noise in the first noise image pair is independently distributed. That is, the first noise image pair meets the requirements of the N2N model for training images. Training the N2N model based on the first noise image pair can make the denoising performance of the trained N2N model better.

[0190] In one example, the initial N2N denoising model includes a plastic convolution layer, which includes a pooling layer, a predictor network, a depth-wise separable convolution layer, an upsampling layer, and a dynamic convolution layer.

[0191] In one example, in the above step S603, the N2N denoising model is trained in the following manner:

[0192] Step d1: According to the first noisy image pair, the pre-built initial N2N denoising model is trained to obtain a first N2N denoising model.

[0193] Step d2: Replace the denoiser in the first noise generation model with the first N2N denoising model to obtain a third noise image generation model.

[0194] Step d3: Generate a second noise image pair according to the random data, the actual noise image, and the third noise image generation model.

[0195] Step d4: Calculate a fifth loss function value based on the first noise image pair and the second noise image pair.

[0196] Step d5: Adjust the parameters in the first N2N denoising model according to the fifth loss function value, and recalculate the fifth loss function value until the fifth loss function value no longer decreases, thereby obtaining the N2N denoising model.

[0197] In an embodiment of the present invention, the N2N denoising model is adjusted and trained according to the calculated fifth loss function value to improve the denoising performance of the N2N denoising model.

[0198] Through the embodiment of the present invention, based on the N2N model, a denoising method based on plastic convolution and fast iterative training strategy (Fast Iteration Noise2Noise, FIN2N) is proposed.

[0199] The following is a specific example to illustrate the working process of a denoising model training method. Figure 7 As shown, the process includes the following:

[0200] First, for a real noisy image dataset made by random cropping Each actual noise image Input the denoiser in the first noisy image generation model Will obey Random vectors from a distribution Input the first generator in the first noise image generation model Then The output results are added to get the Z-based (i) The first noise image pair The expression of this process is as follows:

[0201]

[0202] Then, the first noise image pairs of all inspected objects are expressed as N2N denoising model The first N2N denoising model can be obtained by training by minimizing the following formula:

[0203]

[0204] In order to achieve iterative training, the first N2N denoising model obtained based on the first round of training needs to be trained for the second round. The first noise image pair of the inspection object is generated in the jth round (j≥2). The expression is as follows:

[0205]

[0206] The first noise image pair of all inspected objects in round j is expressed as The first N2N denoising model in the jth round can be trained by minimizing the following formula.

[0207]

[0208] In order to prove that the denoising effect of the N2N denoising model trained based on the FIN2N denoising method is better than the denoising effect of the model trained only once, the following mathematical analysis of the process is performed.

[0209] Let x represent a clean image, subject to N represents zero-mean noise independent of x, obeying x′=x+N0,N0 obeys Then Z′1=x′+N1 and Z′2=x′+N2 can be expressed as a noise observation pair based on the noise image, which is the first noise image pair in the above embodiment. The N2N denoising model obtained after training is expressed as follows:

[0210]

[0211] Among them, f Noisy N2N To complete the trained N2N model,

[0212]

[0213] For convenience, let g(y) represent the molecular part of the above formula:

[0214]

[0215] When y=0, y=1:

[0216]

[0217]

[0218] Since the derivative of g(y) Function g(y) is a monotonically increasing function, which means that g(y) has zero root in the interval (0,1). Therefore, it can be proved that for Further prove

[0219] The above analysis can be summarized as follows: Noisy N2N (Z,y), assuming Then there exists y0 such that for all Satisfy constraints This conclusion proves the following two points:

[0220] (1) When using f Noisy N2N As a denoiser, the N2N denoiser trained using the first noise image pair performs better than the N2N denoiser trained using the training images in the related art.

[0221] (2) The N2N denoiser obtained by the denoising model training method provided by the embodiment of the present invention is better than the N2N denoiser trained only once.

[0222] Figure 8 8 is a flowchart of a denoising method according to an exemplary embodiment. The method includes the following steps S801 to S802.

[0223] Step S801: Acquire an image to be denoised.

[0224] Step S802: inputting the image to be denoised into the N2N denoising model to obtain the image after denoising. The N2N denoising model is trained by the denoising model training method in the above embodiment.

[0225] In an embodiment of the present invention, the N2N denoising model is obtained by training a first noise image pair corresponding to the same clean image, and the noise in the first noise image pair is independently distributed, that is, the first noise image pair meets the requirements of the N2N model for training images, and the denoising performance of the N2N model obtained by training based on the first noise image pair is better.

[0226] To verify the effectiveness of the denoising method proposed in an embodiment of the present invention, a real noise chip dataset RN-CD (Real Noise Chip Dataset, RN-CD) was produced. This dataset contains 1,000 noisy chip images collected under different external lighting conditions and other packaging conditions, and is randomly divided into training set: validation set: test set = 6:2:2 ratio.

[0227] In terms of quantitative analysis, Tables 3 and 4 list and compare the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the FIN2N denoising method provided by the embodiment of the present invention with the benchmark methods BM3D, DnCNN, N2N, and N2V on a chip's real noise dataset. For example, the denoising method provided by the embodiment of the present invention improves the PSNR value by 2.05dB and the SSIM value by 0.1405 compared to the suboptimal method BM3D, and improves the PSNR value by 3.01dB and the SSIM value by 0.1959 compared to the suboptimal method N2V based on deep learning. The results show that the performance of the denoising method provided by the embodiment of the present invention is significantly better than that based on traditional denoising methods and denoising methods based on deep learning. The denoising method provided by the embodiment of the present invention can achieve better denoising effects on real noisy images. Therefore, the denoising method provided by the embodiment of the present invention is very effective in removing real noise in quantitative analysis.

[0228] Table 3. Average PSNR results of different methods on the RN-CD test set

[0229]

[0230] Table 4 Average SSIM results of different methods on the RN-CD test set

[0231]

[0232] In terms of qualitative analysis, Figure 9 、 Figure 10 They are respectively the visualization image and the magnified image of the noise chip image. Figure 11 、 Figure 12 They are the visualization results and magnification results after training once, Figure 13 、 Figure 14 The visualization and magnification results of the training cycle are shown below. As can be seen from the figure, as the number of iterations increases, sharper image edges and clearer characters are obtained. Figure 15 This is the enlarged result of Canny edge detection on the noisy chip image. Figure 16 This is the enlarged result of Canny edge detection after training once. Figure 17 This is the enlarged result of Canny edge detection after training 6 times. Figure 15 It can be seen that a lot of noise is mistakenly judged as edges. This false edge will interfere with the application of subsequent image recognition algorithms, directly leading to a decrease in the recognition rate and accuracy of the recognition algorithm. Figure 16 and Figure 17 It can be seen that with the continuous iterative training of the N2N denoising model, the denoising effect of the model gradually increases, the edges and characters become clearer, the interference edges become fewer, the edge contours are enhanced and gradually connected into complete straight lines, and the clear edges help the recognition algorithm accurately and completely identify each chip and the characters on its surface. The above visualization results show that the actual visual effect improves with the increase of the iteration cycle. Therefore, the denoising method provided by the embodiment of the present invention is very effective in removing real noise in qualitative analysis.

[0233] Figure 18 This is a schematic diagram of the structure of a noise image generation model training device proposed according to an exemplary embodiment, the device comprising:

[0234] The first acquisition module 1801 is used to acquire random data and an actual noise image; for details, please refer to the description of step S101 in the above embodiment, which will not be repeated here.

[0235] The first generating module 1802 is used to input random data into the first generator to obtain simulated noise; for details, please refer to the description of step S102 in the above embodiment, which will not be repeated here.

[0236] The denoising module 1803 is used to input the actual noisy image into the denoiser to obtain a denoised image. For details, please refer to the description of step S103 in the above embodiment and will not be repeated here.

[0237] The synthesis module 1804 is used to input the simulated noise and the denoised image into the synthesis module to obtain a synthesized noise image; for details, please refer to the description of step S104 in the above embodiment, which will not be repeated here.

[0238] The discrimination module 1805 is used to input the synthetic noise image and the actual noise image into the first discriminator to obtain a first discrimination result; for details, please refer to the description of step S105 in the above embodiment, which will not be repeated here.

[0239] First training module 1806 is configured to train the first generator, denoiser, and first discriminator in the first noise image generation model based on the first discrimination result to obtain a second noise image generation model. The second noise image generation model is used to synthesize the noise image. For details, refer to the description of step S106 in the above embodiment and will not be repeated here.

[0240] Through the above device, the simulated noise generated by the first generator is generated based on random data. Therefore, the obtained simulated noise is independently distributed. At the same time, the denoised image obtained by the denoiser can be generated by the same actual noise image, which is equivalent to coming from the same clean image. Therefore, the synthetic noise image obtained based on the simulated noise and the denoised image can meet the requirements of the N2N model for training images.

[0241] Figure 19 1 is a structural diagram of a denoising model training device proposed according to an exemplary embodiment, the device comprising:

[0242] The second acquisition module 1901 is used to acquire random data and an actual noise image; for details, please refer to the description of step S601 in the above embodiment, which will not be repeated here.

[0243] The second generation module 1902 is used to generate a first noise image pair based on the random data and the actual noise image, and the first noise generation model. The first noise generation model is obtained by the noise image generation model training method of the above embodiment; for details, please refer to the description of step S602 in the above embodiment, which will not be repeated here.

[0244] The second training module 1903 is used to train the pre-built initial N2N denoising model based on the first noise image pair to obtain an N2N denoising model. For details, please refer to the description of step S603 in the above embodiment, which will not be repeated here.

[0245] Through the above-mentioned device, the first noise image pair generated by the first noise generation model comes from the same clean image, and the noise in the first noise image pair is independently distributed. That is, the first noise image pair meets the requirements of the N2N model for training images. Training the N2N model based on the first noise image pair can make the denoising performance of the trained N2N model better.

[0246] Figure 20 1 is a schematic structural diagram of a denoising device according to an exemplary embodiment, the device comprising:

[0247] The third acquisition module 2001 is used to acquire the image to be denoised; for details, please refer to the description of step S801 in the above embodiment, which will not be repeated here.

[0248] Denoising module 2002 is configured to input the image to be denoised into an N2N denoising model to obtain a denoised image. The N2N denoising model is trained using the denoising model training method described in the above embodiment. For details, see the description of step S802 in the above embodiment and will not be repeated here.

[0249] The specific limitations and beneficial effects of the above-mentioned device can be found in the above-mentioned limitations on the noise image generation model training, denoising model training and denoising method, which will not be repeated here. The above-mentioned modules can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0250] Figure 21 FIG. 1 is a schematic diagram of the hardware structure of a computer device according to an exemplary embodiment. Figure 21 As shown, the device includes one or more processors 2110 and a memory 2120, and the memory 2120 includes a persistent memory, a volatile memory, and a hard disk. Figure 21 The apparatus may further include an input device 2130 and an output device 2140 .

[0251] The processor 2110, the memory 2120, the input device 2130 and the output device 2140 may be connected via a bus or other means. Figure 21 The bus connection is taken as an example.

[0252] The processor 2110 may be a central processing unit (CPU). The processor 2110 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. The general-purpose processor may be a microprocessor or any conventional processor.

[0253] Memory 2120, as a non-transitory computer-readable storage medium, includes persistent memory, volatile memory, and a hard disk, and can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the noise image generation model training, denoising model training, and denoising methods in the embodiments of the present application. Processor 2110 executes the non-transitory software programs, instructions, and modules stored in memory 2120 to execute various functional applications and data processing of the server, thereby implementing any of the aforementioned noise image generation model training, denoising model training, and denoising methods.

[0254] The memory 2120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data required for use, etc. In addition, the memory 2120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 2120 may optionally include a memory remotely arranged relative to the processor 2110, and these remote memories may be connected to the data processing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0255] The input device 2130 can receive input digital or character information and generate signal input related to user settings and function control. The output device 2140 can include a display device such as a display screen.

[0256] One or more modules are stored in the memory 2120 and when executed by one or more processors 2110, the execution is as follows: Figure 1 The method shown.

[0257] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. Figure 1 Related description of the embodiment shown.

[0258] An embodiment of the present invention further provides a non-transitory computer storage medium storing computer-executable instructions that can execute the method of any of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage medium can also include a combination of the above types of memory.

[0259] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0260] The foregoing is merely a detailed description of the present invention, intended to enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features claimed herein.

Claims

1. A noise image generation model training method, characterized in that: The first noise image generation model includes a first generator, a denoiser, a first discriminator and a synthesis module, and the method includes: Get random data and real noise images; Inputting the random data into the first generator to obtain simulated noise; Inputting the actual noise image into the denoiser to obtain a denoised image; Inputting the simulated noise and the denoised image into the synthesis module to obtain a synthesized noise image; Inputting the synthetic noise image and the actual noise image into the first discriminator to obtain a first discrimination result; According to the first discrimination result, the first generator, the denoiser and the first discriminator in the first noise image generation model are trained to obtain a second noise image generation model, wherein the second noise image generation model is used to synthesize a noise image; The denoiser includes at least one inverted bottleneck block, the inverted bottleneck block includes a plastic convolution layer, the plastic convolution layer includes a depth-separable convolution layer, and the actual noise image is input into the denoiser to obtain a denoised image, including: Obtaining a first feature map according to the actual noise image and the depth-separable convolutional layer; Obtaining the denoised image according to the first feature map and the actual noise image; The plastic convolution layer also includes a pooling layer, a predictor network, an upsampling layer and a dynamic convolution layer. The first feature map is obtained according to the actual noise image and the depth-separable convolution layer, including: Performing a shuffling operation on the actual noise image to obtain an actual noise image after the operation; Inputting the actual noise image after the operation into the pooling layer to obtain a second feature map; Inputting the second feature map into the predictor network to obtain a third feature map; Inputting the third feature map into the depthwise separable convolutional layer to obtain a plurality of first filters; Inputting each of the first filters into the upsampling layer to obtain a plurality of second filters; Each of the second filters and the actual noise image after the operation is input into the dynamic convolution layer to obtain the first feature map.

2. The method according to claim 1, characterized in that The first noise image generation model further includes a second generator and a second discriminator. According to the first discrimination result, the first generator, the denoiser and the first discriminator in the first noise image generation model are trained to obtain the second noise image generation model, including: Acquire actual noise according to the actual noise image; Inputting the simulated noise and the actual noise into the second discriminator to obtain a second discrimination result; Inputting the denoised image into the second generator to obtain a simulated noise image; Calculating a first loss function value according to the first discrimination result, the second discrimination result, the simulated noise image, and the actual noise image; Adjust the parameters in the first generator, the parameters in the denoiser, the parameters in the second generator, the parameters in the first discriminator and the parameters in the second discriminator, and recalculate the first loss function value until the first loss function value no longer decreases, thereby obtaining a second noisy image generation model.

3. The method according to claim 2, characterized in that Calculating a first loss function value according to the first discrimination result, the second discrimination result, the simulated noise image, and the actual noise image includes: Calculating a second loss function value according to the first discrimination result; Calculating a third loss function value according to the second discrimination result; Calculating a fourth loss function value according to the actual noise image and the simulated noise image; The first loss function value is calculated based on the second loss function value, the third loss function value and the fourth loss function value.

4. The method according to claim 3, characterized in that: Calculating the first loss function value according to the second loss function value, the third loss function value, and the fourth loss function value includes: The second loss function value, the third loss function value and the fourth loss function value are weighted and summed to obtain the first loss function value, and the weight value of the second loss function value, the weight value of the third loss function value and the weight value of the fourth loss function value decrease in sequence.

5. The method according to claim 2, characterized in that: Acquiring actual noise according to the actual noise image includes: According to the actual noise image, a plurality of image blocks are obtained; Performing a two-dimensional discrete wavelet transform on each of the image blocks to obtain a scale coefficient image and a wavelet coefficient image corresponding to each of the image blocks; The image blocks where the scale coefficient image and the wavelet coefficient image meet the preset conditions are regarded as the noise area; The actual noise is obtained by subtracting each pixel in the noise area from the average pixel in the noise area.

6. A denoising model training method, characterized in that: The method comprises: Get random data and real noise images; Generate a first noise image pair according to the random data and the actual noise image, and a first noise generation model, wherein the first noise generation model is obtained by the noise image generation model training method according to any one of claims 1 to 5; The pre-constructed initial N2N denoising model is trained according to the first noise image pair to obtain an N2N denoising model.

7. The method according to claim 6, characterized in that The initial N2N denoising model includes a plastic convolution layer, which includes a pooling layer, a predictor network, a depth-separable convolution layer, an upsampling layer, and a dynamic convolution layer.

8. The method according to claim 6, characterized in that According to the first noise image pair, a pre-built initial N2N denoising model is trained to obtain an N2N denoising model, including: According to the first noise image pair, training a pre-constructed initial N2N denoising model to obtain a first N2N denoising model; Replacing the denoiser in the first noise generation model with the first N2N denoising model to obtain a third noise image generation model; Generate a second noise image pair according to the random data, the actual noise image, and the third noise image generation model; Calculating a fifth loss function value according to the first noise image pair and the second noise image pair; The parameters in the first N2N denoising model are adjusted according to the fifth loss function value, and the fifth loss function value is recalculated until the fifth loss function value no longer decreases, thereby obtaining the N2N denoising model.

9. A denoising method, characterized in that: The method comprises: Obtain an image to be denoised; The image to be denoised is input into an N2N denoising model to obtain an image after denoising, wherein the N2N denoising model is trained by the denoising model training method according to any one of claims 6 to 8.

10. A noise image generation model training device, characterized in that: The first noise image generation model includes a first generator, a denoiser, a first discriminator and a synthesis module, and the device includes: A first acquisition module, used for acquiring random data and actual noise images; A first generating module, used for inputting the random data into the first generator to obtain simulated noise; A denoising module, used for inputting the actual noise image into the denoiser to obtain a denoised image; A synthesis module, used for inputting the simulated noise and the denoised image into the synthesis module to obtain a synthesized noise image; A discrimination module, used for inputting the synthetic noise image and the actual noise image into the first discriminator to obtain a first discrimination result; A first training module is used to train the first generator, the denoiser and the first discriminator in the first noise image generation model according to the first discrimination result to obtain a second noise image generation model, wherein the second noise image generation model is used to synthesize a noise image; The denoiser includes at least one inverted bottleneck block, the inverted bottleneck block includes a plastic convolution layer, the plastic convolution layer includes a depth-separable convolution layer, and the actual noise image is input into the denoiser to obtain a denoised image, including: Obtaining a first feature map according to the actual noise image and the depth-separable convolutional layer; Obtaining the denoised image according to the first feature map and the actual noise image; The plastic convolution layer also includes a pooling layer, a predictor network, an upsampling layer and a dynamic convolution layer. The first feature map is obtained according to the actual noise image and the depth-separable convolution layer, including: Performing a shuffling operation on the actual noise image to obtain an actual noise image after the operation; Inputting the actual noise image after the operation into the pooling layer to obtain a second feature map; Inputting the second feature map into the predictor network to obtain a third feature map; Inputting the third feature map into the depthwise separable convolutional layer to obtain a plurality of first filters; Inputting each of the first filters into the upsampling layer to obtain a plurality of second filters; Each of the second filters and the actual noise image after the operation is input into the dynamic convolution layer to obtain the first feature map.

11. A denoising model training device, characterized in that: The device comprises: A second acquisition module is used to acquire random data and actual noise images; A second generating module, configured to generate a first noise image pair according to the random data and the actual noise image, and a first noise generating model, wherein the first noise generating model is obtained by the noise image generating model training method according to any one of claims 1 to 5; The second training module is used to train the pre-constructed initial N2N denoising model according to the first noise image pair to obtain the N2N denoising model.

12. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the noise image generation model training method described in any one of claims 1 to 5, or the denoising model training method described in claims 8 to 10, or the steps of the denoising method described in claim 11 by executing the computer instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the noise image generation model construction training method as described in any one of claims 1 to 5, or the denoising model training method as described in claims 6 to 8, or the steps of the denoising method as described in claim 9.

Images