Image global amplitude-preserving denoising and completion model construction method, device and storage medium

Abstract

The application discloses a method and device for constructing an image global amplitude-preserving denoising and completion model and a storage medium, comprising: acquiring a two-dimensional image pair composed of an original sample image and a corresponding noisy missing image; iteratively training a primary image reconstruction model of a preset two-stage multi-scale model according to the two-dimensional image pair and a target loss function to obtain model reconstruction parameters and a model reconstruction image, wherein the target loss function is composed of an image reconstruction model loss function and an image segmentation model loss function; updating the two-dimensional image pair according to the model reconstruction image to obtain a target image pair; and iteratively training a secondary image segmentation model of the preset two-stage multi-scale model and the primary image reconstruction model according to the model reconstruction parameters, the target loss function and the target image pair to obtain a target image interpretation model. The application solves the technical problem of poor interpretability of the denoised and completed image.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method for constructing a global amplitude-preserving denoising and completion model for images, an electronic device, and a readable storage medium. Background Technology

[0002] Images, with their large information content, long transmission distance, and fast transmission speed, have become an important source and means for humans to obtain information. However, during the generation, transmission, and shooting process, images are inevitably subject to noise pollution, which can easily lead to incomplete, partially damaged, or partially destroyed images.

[0003] Currently, for images with compromised image quality, deep learning models are typically used for denoising and incomplete image processing to improve the signal-to-noise ratio. However, current deep learning-based denoising and incomplete methods only focus on reducing the difference between the recovered data and the original data during training. Furthermore, when there are large gaps in the image, the deep learning model can only capture local relationships, which leads to the inability to obtain reasonable interpolation results for low-amplitude missing components. In other words, it is not conducive to subsequent processing and interpretation of the denoised and incomplete image. Therefore, the interpretability of the denoised and incomplete image is currently poor. Summary of the Invention

[0004] The main objective of this application is to provide a method for constructing a global amplitude-preserving denoising and completion model for images, an electronic device, and a readable storage medium, aiming to solve the technical problem of poor interpretability of denoised and completed images in the prior art.

[0005] To achieve the above objectives, this application provides a method for constructing a global image amplitude-preserving denoising and completion model, the method comprising:

[0006] Obtain two-dimensional image pairs consisting of the original sample image and the corresponding noisy, missing image;

[0007] Based on the two-dimensional image pairs and the target loss function, a primary image reconstruction model with a preset two-level multi-scale model is iteratively trained to obtain model reconstruction parameters and model reconstruction images. The target loss function consists of an image reconstruction model loss function and an image segmentation model loss function.

[0008] The image is reconstructed based on the model, and the two-dimensional image pair is updated to obtain the target image pair;

[0009] Based on the model reconstruction parameters, the target loss function, and the target image pair, the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model are iteratively trained to obtain the target image interpretation model.

[0010] To achieve the above objectives, this application also provides an image global amplitude-preserving denoising and completion model construction apparatus, the image global amplitude-preserving denoising and completion model construction apparatus comprising:

[0011] The acquisition module is used to acquire two-dimensional image pairs consisting of the original sample image and the corresponding noisy, missing image;

[0012] The first training module is used to iteratively train a primary image reconstruction model with a preset two-level multi-scale model based on the two-dimensional image pairs and the target loss function, so as to obtain the model reconstruction parameters and the model reconstruction image. The target loss function is composed of the image reconstruction model loss function and the image segmentation model loss function.

[0013] The update module is used to reconstruct the image based on the model, update the two-dimensional image pair, and obtain the target image pair;

[0014] The second training module is used to iteratively train the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model based on the model reconstruction parameters, the target loss function, and the target image pair, so as to obtain the target image interpretation model.

[0015] This application also provides an electronic device, the electronic device comprising: a memory, a processor, and a program of the image global amplitude preservation denoising and completion model construction method stored in the memory and executable on the processor, wherein when the program of the image global amplitude preservation denoising and completion model construction method is executed by the processor, the steps of the image global amplitude preservation denoising and completion model construction method as described above can be implemented.

[0016] This application also provides a computer-readable storage medium storing a program that implements a method for constructing a global image amplitude-preserving denoising and completion model. When the program for constructing a global image amplitude-preserving denoising and completion model is executed by a processor, it implements the steps of the method described above.

[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image global amplitude preservation denoising and completion model construction method described above.

[0018] This application provides a method, electronic device, and readable storage medium for constructing a global amplitude-preserving denoising and completion model for images. Specifically, it involves: acquiring a two-dimensional image pair consisting of an original sample image and its corresponding noisy, missing image; iteratively training a primary image reconstruction model with a preset two-level multi-scale model based on the two-dimensional image pair and a target loss function to obtain model reconstruction parameters and a model reconstruction image; wherein the target loss function consists of an image reconstruction model loss function and an image segmentation model loss function; updating the two-dimensional image pair based on the model reconstruction image to obtain a target image pair; and iteratively training a secondary image segmentation model with the preset two-level multi-scale model and the primary image reconstruction model based on the model reconstruction parameters, the target loss function, and the target image pair to obtain a target image interpretation model. Because the target loss function relied upon during iterative training of the pre-set two-level multi-scale model consists of the image reconstruction model loss function and the image segmentation model loss function, the primary image reconstruction model obtained through iterative training focuses more on global image features, while the two-level multi-scale model iteratively trained based on the primary image reconstruction model focuses more on local image features. At the same time, since the target image interpretation model is trained through multiple rounds of iterative training of the pre-set two-level multi-scale model, it can solve the problem of image amplitude missing and classification ambiguity. That is, the target image interpretation model can perform global amplitude-preserving denoising and completion on the input image, so that the image output by the target image interpretation model can characterize the global and local features of the image. This overcomes the technical defect that deep learning models can only capture local relationships, which leads to the inability to obtain reasonable results for interpolation of low-amplitude missing components. Therefore, it improves the interpretability of the denoised and completed image. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the image global amplitude-preserving denoising and completion model construction method provided in Embodiment 1 of this application;

[0022] Figure 2 This diagram illustrates the connection relationship between the primary image reconstruction model and the secondary image reconstruction model in the image global amplitude-preserving denoising and completion model construction method provided in Embodiment 1 of this application.

[0023] Figure 3 This is a schematic diagram illustrating the processing of module input by the horizontal and vertical attention modules in the image global amplitude-preserving denoising and completion model construction method provided in Embodiment 1 of this application.

[0024] Figure 4 This is a flowchart illustrating the image global amplitude-preserving denoising and completion model construction method provided in Embodiment 2 of this application;

[0025] Figure 5 This is a schematic diagram of the structure of the image global amplitude-preserving denoising and completion model construction device provided in Embodiment 3 of this application;

[0026] Figure 6 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of this application.

[0027] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0028] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example 1

[0030] First, it should be understood that traditional methods for image denoising and incompleteness are typically based on various principles and assumptions, such as sparsity and low-rank assumptions. However, the complexity of real-world images makes it difficult to match these assumptions, leading to unreasonable practical results. In recent years, with the rapid development of artificial intelligence, deep learning-based image denoising and incompleteness methods have achieved good performance. However, existing research shows that current deep learning-based methods only focus on reducing the difference between the recovered data and the original data during training, ignoring the features of local structures. Furthermore, when images have consecutive large gaps, convolutional neural networks can only capture local relationships, and interpolation of low-amplitude missing components cannot yield reasonable results, posing a significant challenge to subsequent image processing and interpretation. Therefore, global amplitude-preserving denoising and incompleteness to characterize the global and local features of the image is essential and worthwhile. Otherwise, even after receiving the image output by the deep learning model, subsequent processing and interpretation will be difficult. In other words, there is an urgent need for a method that can perform global amplitude-preserving denoising and incompleteness to improve the interpretability of the denoised and incomplete image.

[0031] This application provides a method for constructing a global image amplitude-preserving denoising and completion model. In the first embodiment of this method, refer to... Figure 1 The method for constructing the global amplitude-preserving denoising and completion model for images includes:

[0032] Step S10: Obtain a two-dimensional image pair consisting of the original sample image and the corresponding noisy, missing image;

[0033] Step S20: Based on the two-dimensional image pair and the target loss function, iteratively train the primary image reconstruction model of the preset two-level multi-scale model to obtain the model reconstruction parameters and the model reconstruction image. The target loss function is composed of the image reconstruction model loss function and the image segmentation model loss function.

[0034] Step S30: Reconstruct the image according to the model, update the two-dimensional image pair, and obtain the target image pair;

[0035] Step S40: Based on the model reconstruction parameters, the target loss function, and the target image pair, iteratively train the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model to obtain the target image interpretation model.

[0036] In this embodiment, it should be noted that the target image interpretation model is used for global amplitude-preserving denoising and completion of the image. That is, the target image interpretation model is a global amplitude-preserving denoising and completion model. The original sample image is an unprocessed original image, which can be obtained by using conventional sampling methods in the field of machine learning. Specifically, it can be one or more images. The noisy missing image is an image with missing parts and noise pollution. The two-dimensional image pair is used to train the target image interpretation model. Specifically, it is one or more sets. In a set of two-dimensional image pairs, it specifically includes one original sample image and a noisy missing image corresponding to the original sample image. The image, wherein the original sample image is used as a label, and the noisy missing image corresponding to the original sample image is used as the input of a preset reconstruction model. For example, in one implementable method, the original sample image is an original image without image missing and without noise pollution, and the noisy missing image corresponding to the original sample image is an original image without image missing and without noise pollution. The image obtained after noise missing processing is an image in which the original sample image and the noisy missing image have the same image features except for image missing and noise pollution. The missing position of the image can be random (random size and position), or it can be continuous missing or block missing.

[0037] Additionally, it should be noted that the preset two-level multi-scale model is used to represent a two-level multi-scale neural network model awaiting model training. Specifically, it may include a primary image reconstruction network model and a secondary image segmentation network model. Both the primary multi-scale image reconstruction network model and the secondary multi-scale image pattern segmentation network model are horizontal and vertical attention encoder-decoder networks. The horizontal and vertical attention encoder-decoder network can be an improved U-shaped encoder-decoder network, where the encoder is used for feature extraction, the decoder is used for reconstructing image data, and the horizontal and vertical attention modules in the encoder-decoder network are used to capture long-distance relationships in the image.

[0038] Additionally, it should be noted that the primary image reconstruction model is an untrained image reconstruction model. The image reconstruction model is used to extract image information from the image to reconstruct the image. The model-reconstructed image is the output image of the trained primary image reconstruction model. The model reconstruction parameters are the model parameters of the trained primary image reconstruction model. The primary image reconstruction model obtained by iteratively training a preset two-level multi-scale model is used to denoise and complete missing and noisy images. The secondary image segmentation model obtained by iteratively training a preset two-level multi-scale model is used to ensure that the model-reconstructed image and the original sample image have the same texture, color, and contrast, focusing on the local features of the image.

[0039] As an example, steps S10 to S40 include: acquiring an original sample image and a noisy missing image corresponding to the original sample image; combining the original sample image and the noisy missing image one by one to obtain the two-dimensional image pair; inputting the noisy missing image into the primary image reconstruction model of the preset two-level multi-scale model; iteratively optimizing the primary image reconstruction model until the loss function of the image reconstruction model converges, and obtaining the model reconstruction parameters and model reconstruction image of the primary image reconstruction model; replacing the noisy missing image in the two-dimensional image pair with the model reconstruction image to obtain the target image pair; iteratively training the secondary image segmentation model of the preset two-level multi-scale model and the primary image reconstruction model according to the model reconstruction parameters, the target loss function, and the target image pair to obtain the target image interpretation model, wherein the target image interpretation model is used to interpret the image from the image reconstruction scale and the image segmentation scale. Because the trained target image interpretation model includes horizontal and vertical attention modules, which can capture long-distance correlations in the image and include information on direction and position, the model can more accurately locate and identify the target region of the image. As a result, when reconstructing and segmenting the image, it can pay more attention to the global and local features of the image, achieving the purpose of global amplitude-preserving denoising and completion of the image, thus improving the interpretability of the denoised and completed image.

[0040] In one feasible approach, the connection relationship between the primary multi-scale image reconstruction network model (primary image reconstruction model) and the secondary multi-scale image style segmentation network model (secondary image reconstruction model) of the pre-defined two-level multi-scale model is as follows: Figure 2 As shown, A1, A2, A3, and A4 are downsampling modules of the primary image reconstruction model at different sampling orders; B1, B2, B3, and B4 are upsampling modules of the secondary image segmentation model at different sampling orders; C1, C2, C3, and C4 are downsampling modules of the secondary image reconstruction model at different sampling orders; D1, D2, D3, and D4 are downsampling modules of the primary image reconstruction model at different sampling orders; the model output image is the output of the target image interpretation model; I1 represents the spatial dimension stitching operation; H represents the number of iterations; and the target image interpretation model includes K downsampling blocks and K upsampling blocks.

[0041] The target image pair includes the original sample image and the model-reconstructed image. The step of iteratively training the secondary image segmentation model of the preset two-level multi-scale model and the primary image reconstruction model based on the model reconstruction parameters, the target loss function, and the target image pair to obtain the target image interpretation model includes:

[0042] Step A10: Based on the image reconstruction model loss function and the reconstructed image, the reconstruction process parameters and the reconstructed image are obtained by training the primary image reconstruction model.

[0043] Step A20: Update the secondary image segmentation model using the parameters of the reconstruction process to obtain the image segmentation model to be trained;

[0044] Step A30: Based on the original sample image, the reconstructed process image, and the image segmentation model loss function, the reconstruction process parameters and the model segmentation parameters of the reconstructed process image and the secondary image segmentation model are updated by training the image segmentation model to be trained.

[0045] Step A40, return to the execution steps: reconstruct the image according to the model, update the two-dimensional image pair to obtain the target image pair, until the target loss function converges, and obtain the target image interpretation model.

[0046] As an example, steps A10 to A40 include: inputting the model-reconstructed image into the primary image reconstruction model; iteratively optimizing the preset image reconstruction model until the image reconstruction model loss function converges; obtaining the reconstruction process parameters and reconstruction process image of the primary image reconstruction model; wherein the reconstruction process parameters are used to characterize the process parameters output during iterative training of the primary image reconstruction model; the reconstruction process image is the process image output during iterative training of the primary image reconstruction model; the process image is used to characterize the reconstructed image output by the primary image reconstruction model between the noisy missing image and the model-reconstructed image; and the process parameters are used to characterize the reconstruction parameters output by the primary image reconstruction model between the preset model parameters and the model reconstruction parameters of the primary image reconstruction model; and inputting the model of the secondary image segmentation model into the primary image reconstruction model. The parameters are updated to the reconstruction process parameters to obtain the image segmentation model to be trained. The model parameters of the secondary image segmentation model can be preset according to actual needs. The original sample image and the reconstruction process image are input into the image segmentation model to be trained. The model is iteratively optimized until the image segmentation model loss function converges. The reconstruction process parameters, the reconstruction process image, and the model segmentation parameters of the secondary image segmentation model are updated. The process returns to the following steps: Based on the model-reconstructed image, the two-dimensional image pair is updated to obtain the target image pair until the target loss function converges. The target model reconstruction parameters, the target model segmentation parameters, and the target reconstructed image are obtained. The preset two-level multi-scale model is replaced with the target model reconstruction parameters and the target model segmentation parameters to obtain the target image interpretation model.

[0047] In one feasible approach, a pre-defined two-level multi-scale model can be trained using a multi-round training method to solve the problem of interpolating high-amplitude and low-amplitude missing components in continuously missing images over a wide amplitude range. Each training round generates a two-level multi-scale model and updates the training samples. The original sample image D of the two-dimensional image pair is denoted as Di. step1 The noisy missing image D corresponding to the original sample image obs Let it be D obs-step1 First, in the first round of training, the loss function of the image reconstruction model is denoted as... The loss function of the image segmentation model is denoted as: Start training, and add noisy, missing images D obs-step1 The input is fed into a primary image reconstruction model for primary network training. During training, the Adam optimizer is used to optimize the model. The exponential decay rate for the first-order moment estimation in the Adam optimizer can be set to 0.9, and the exponential decay rate for the second-order moment estimation can be set to 0.98. The learning rate can be initialized to 10. -4 ,when When the rate of decrease stabilizes, reduce the learning rate to 10%. -5 Stop training iterations for the primary image reconstruction model, and obtain the parameters for reconstruction process 1 as follows: and the reconstruction process image are Reconstruct process parameters Model parameters of the secondary image segmentation model, which serves as the basis for the pre-defined two-level multi-scale model. The secondary image segmentation model is obtained and awaits training. Then, the reconstructed images are... The corresponding original image D is input into the model parameters as follows: In the secondary image segmentation model awaiting training, the objective loss function of the preset two-level multi-scale model in the first round of training is denoted as... The iteration of the preset two-level multi-scale model is determined by the target loss function. Under control, during the training process, the model parameters of the primary image reconstruction model and the secondary image segmentation model of the two-level multi-scale model are preset as follows: and and model process images This will change; the Adam optimizer is used to optimize the preset two-level multi-scale model. The exponential decay rate of the first-order moment estimate in the Adam optimizer can be set to 0.91, and the exponential decay rate of the second-order moment estimate can be set to 0.99. The epsilon parameter is used to prevent division by zero in the implementation. The learning rate can be initialized to 10. -4 ,when When the rate of decrease stabilizes, reduce the learning rate to 10%. -5 Stop the first round of training for the preset two-level multi-scale model, and finally determine the model reconstruction parameters as follows: The model reconstructs the image as Before the second round of training, the noisy, missing images of the entire 2D image pair are replaced with the reconstructed images obtained from the model in the first round of training. In the second round of training, the loss function of the image reconstruction model is denoted as... The loss function of the image segmentation model is denoted as: Start training, and add noisy, missing images D obs-step2 The input is fed into a primary image reconstruction model for primary network training. During training, the Adam optimizer is used to optimize the model. The exponential decay rate for the first-order moment estimation in the Adam optimizer can be set to 0.9, and the exponential decay rate for the second-order moment estimation can be set to 0.98. The learning rate can be initialized to 10. -4 ,when When the rate of decrease stabilizes, reduce the learning rate to 10%. -5 Stop training iterations on the primary image reconstruction model, and obtain the reconstruction process parameters as follows: and the reconstruction process image are Reconstruct process parameters Model parameters of the secondary image segmentation model, which serves as the basis for the pre-defined two-level multi-scale model. The secondary image segmentation model is obtained and awaits training. Then, the reconstructed images are... The corresponding original image D is input into the model parameters as follows: In the secondary image segmentation model awaiting training, the objective loss function of the preset two-level multi-scale model in the first round of training is denoted as... The iteration of the preset two-level multi-scale model is determined by the target loss function. Under control, during the training process, the model parameters of the primary image reconstruction model and the secondary image segmentation model of the two-level multi-scale model are preset as follows: and and model process images This will change; the Adam optimizer is used to optimize the preset two-level multi-scale model. The exponential decay rate of the first-order moment estimate in the Adam optimizer can be set to 0.91, and the exponential decay rate of the second-order moment estimate can be set to 0.99. The epsilon parameter is used to prevent division by zero in the implementation. The learning rate can be initialized to 10. -4 ,when When the rate of decrease stabilizes, reduce the learning rate to 10%. -5 Stop the first round of training for the preset two-level multi-scale model, and finally determine the model reconstruction parameters as follows: The model reconstructs the image as Repeat the above steps to train the preset two-level multi-scale model for H rounds until the target loss function no longer decreases after H rounds of training. The training process ends, yielding the target image interpretation model. The model parameters of the primary image reconstruction model of the target image interpretation model are: The model parameters of the secondary image segmentation model of the target image interpretation model are: The output image of the target image interpretation model is

[0048] In one feasible approach, after the target image interpretation model is trained, a new noisy and missing image can be input into the target image interpretation model to obtain a globally amplitude-preserving denoised and completed image. Since the globally amplitude-preserving denoised and completed image can characterize the global and local features of the image, the efficiency and accuracy of subsequent image interpretation and image processing are improved.

[0049] The loss function of the image reconstruction model is as follows:

[0050]

[0051] in, The loss function for the image reconstruction model is... Assuming a pre-defined loss for the image reconstruction model, Let K be the image similarity index, K be the sampling order, and the loss of the preset image reconstruction model be as follows:

[0052]

[0053] Where ||·||1 is the 1-norm; θ F Indicates the preset image reconstruction process parameters; The model reconstructs the image; the image similarity metric is structural similarity loss, which is as follows:

[0054]

[0055] Wherein, D is the original sample image. The brightness measurement value between the original sample image and the model reconstructed image. The amplitude measurement value between the original sample image and the model reconstructed image. The contrast measurement between the original sample image and the model-reconstructed image.

[0056]

[0057]

[0058]

[0059] in, These are the average values ​​of the original sample image and the model-reconstructed image, respectively. These are the standard deviations of the original sample image and the model-reconstructed image, respectively. It is the covariance between the original sample image and the model reconstructed image, where c1, c2, and c3 are all constants, and the correlation between c1, c2, and c3 is as follows:

[0060] The target loss function is as follows:

[0061]

[0062] Where ξ is the weight coefficient of the model loss function. Let be the loss function for the image segmentation model, wherein the image segmentation model loss function is as follows:

[0063]

[0064] in, c is the output of the inner product of feature maps i and j on the l-th convolutional layer of the primary image reconstruction model. l This represents the number of feature channels in convolutional layer l.

[0065] Both the primary image reconstruction model and the secondary image segmentation model are U-shaped encoder-decoder neural network models. The encoder part of the primary image reconstruction model or the secondary image segmentation model consists of m downsampling modules, which are arranged from shallowest to deepest as follows:

[0066] Conv(a1, b1, c)+BN+ReLU+Conv(a2, b1, c)+BN+ReLU+HVAM+Max

[0067] Conv(a2, b2, c)+BN+ReLU+Conv(a3, b2, c)+BN+ReLU+HVAM+Max

[0068] Conv(a3, b3, c)+BN+ReLU+Conv(a4, b3, c)+BN+ReLU+HVAM+Max

[0069] Conv(a4, b4, c)+BN+ReLU+Conv(a5, b4, c)+BN+ReLU+HVAM+Max

[0070]

[0071] Conv(am b m ,C)+BN+ReLU+Conv(a m+1 b m ,C)+BN+ReLU+HVAM+Max

[0072] Where Conv() is the convolution operation, m is a constant, and a1, a2, a3, ... a m The number of input channels for different downsampling modules, b1, b2, b3, ... b m Here, c represents the number of output channels for different downsampling modules, c represents the side length of the square convolution kernel, Conv has a convolution stride of 1 and padding of 1 to ensure that the feature map size remains unchanged before and after convolution, BN represents batch normalization, ReLU is the activation function, HVAM is the horizontal and vertical attention module, after the feature map group passes through the horizontal and vertical attention module, the feature map size and the number of channels remain unchanged, Max represents max pooling, and the max pooling kernel size is 2x2 with a stride of 2;

[0073] The decoder part of the primary image reconstruction model or the secondary image segmentation model consists of one convolutional layer and m upsampling modules, wherein the m upsampling modules are arranged from shallowest to deepest as follows:

[0074] Up+Conv(a m+2 b m c)+BN+ReLU+Conv(a m+1 b m c)+BN+ReLU+HVAM

[0075]

[0076] Up+Conv(a5, b3, c)+BN+ReLU+Conv(a4, b3, c)+BN+ReLU+HVAM

[0077] Up+Conv(a4, b2, c)+BN+ReLU+Conv(a3, b2, c)+BN+ReLU+HVAM

[0078] Up+Conv(a3, b1, c)+BN+ReLU+Conv(a2, b1, c)+BN+ReLU+HVAM

[0079] In this process, after upsampling, the feature map size is doubled, and the number of channels is reduced to half. The convolutional layer is as follows:

[0080] Conv(a m+1 b m+1 c)+BN+ReLU

[0081] In this case, Conv has a convolution stride of 1 and padding of 1.

[0082] As an example, assuming a sampling order of 4, the decoder part of the primary image reconstruction model or the secondary image segmentation model consists of one convolutional layer and four upsampling modules, and the encoder part of the primary image reconstruction model or the secondary image segmentation model consists of four downsampling modules, wherein the four downsampling modules are arranged from shallowest to deep as follows:

[0083] Conv(1,64,3)+BN+ReLU+Conv(64,64,3)+BN+ReLU+HVAM+Max

[0084] Conv(64,128,3)+BN+ReLU+Conv(128,128,3)+BN+ReLU+HVAM+Max

[0085] Conv(128,256,3)+BN+ReLU+Conv(256,256,3)+BN+ReLU+HVAM+Max

[0086] Conv(256,512,3)+BN+ReLU+Conv(512,512,c)+BN+ReLU+HVAM+Max

[0087] The convolutional layer and four upsampling modules, from shallowest to deepest, are as follows:

[0088] Conv(512, 1024, 3) + BN + ReLU

[0089] Up+Conv(1024,512,3)+BN+ReLU+Conv(512,512,3)+BN+ReLU+HVAM

[0090] Up+Conv(512,256,3)+BN+ReLU+Conv(256,256,3)+BN+ReLU+HVAM

[0091] Up+Conv(256,128,3)+BN+ReLU+Conv(128,128,3)+BN+ReLU+HVAM

[0092] Up+Conv(128,64,3)+BN+ReLU+Conv(64,64,3)+BN+ReLU+HVAM

[0093] The convolution stride and padding can both be set to 1, the pooling kernel size can be set to 2x2, and the pooling stride can be set to 2.

[0094] The downsampling module has the following features:

[0095]

[0096] in, For the downsampling module, The downsampled feature map output by the downsampling module. Here, k represents the downsampling module parameter, and k is the sampling order, in the primary image reconstruction model. For the noisy, missing image, in the secondary image reconstruction model Reconstruct the image for the model;

[0097] The upsampling module has:

[0098]

[0099] in, For the upsampling module, This refers to the upsampled feature map output by the upsampling module. This represents the merged feature map of the encoder and decoder outputs at scale k, where Conc denotes merging. When k = K, in, The upsampling module parameters of the upsampling module

[0100] Wherein, the number of horizontal and vertical attention modules is the sum of the number of upsampling modules and the number of downsampling modules, and the horizontal and vertical attention modules have:

[0101]

[0102] in, For the horizontal and vertical attention modules, For k-th order scale or The feature map after convolutional layers, batch normalization, and ReLU is for The output AND For feature maps of the same size, the input X = [x1, x2, ..., x] of the horizontal and vertical attention modules C ]∈R C×H×W C represents the number of image channels, H represents the image height, and W represents the image width. The input to the horizontal and vertical attention modules is subjected to horizontal and vertical average pooling with sizes (H, p) and (p, W) to obtain horizontal and vertical pooling feature maps. The horizontal and vertical pooling feature maps have the following characteristics:

[0103]

[0104] in, The horizontal pooling feature map is the feature map of the c-th channel output with height h and size C×p×H. The vertical axis pooling feature map is a feature map of size C×W×p output by the c-th channel with width w, where p is the number of pooling layers;

[0105] By concatenating the horizontal-axis pooling feature map and the vertical-axis pooling feature map according to their spatial dimensions and then performing a convolutional transformation, a common intermediate pooling feature map is obtained, wherein the intermediate pooling feature map has:

[0106]

[0107] in, The concatenated feature map is the concatenated feature map of the horizontal axis pooling feature map and the vertical axis pooling feature map, where Γ is the convolution transformation operation and τ is the intermediate pooling feature map;

[0108] After batch normalization and nonlinear activation, the intermediate pooled feature map is divided according to the spatial dimension to obtain the spatial horizontal axis tensor and the spatial vertical axis tensor.

[0109] After performing convolution transformations on the spatial horizontal axis tensor and the spatial vertical axis tensor respectively, the horizontal axis attention weights and vertical axis attention weights are obtained using activation functions, wherein the horizontal axis attention weights and the vertical axis attention weights have:

[0110] ν h =sigmoid(Γ h (τ h )),ν w =si g moid(Γ w (τ w ))

[0111] Where, τ h Let τ be the horizontal tensor of the space. w Let v be the spatial longitudinal tensor. h For the horizontal axis attention weight, ν w The vertical axis attention weight;

[0112] Based on the horizontal axis attention weights and the vertical axis attention weights, a reconstructed feature map corresponding to the noisy, missing image is determined, wherein the reconstructed feature map has:

[0113]

[0114] in, Let the horizontal attention weight of the c-th channel at width i be , Let y be the vertical attention weight of the c-th channel with width w at height j. c (i, j) is the reconstructed feature map, x c (i, j) represents the input feature map of the c-th channel at width i and width j.

[0115] As an example, assuming the upsampling module and the downsampling module each have 4 modules, then the total number of horizontal and vertical attention modules is 8. Each module in both modules includes a convolutional layer, followed by batch normalization and ReLU correction, then a horizontal and vertical attention module, and finally a max-pooling layer. X can be the feature map from either the upsampling or downsampling module after the convolutional layer, batch normalization, and ReLU correction, awaiting input to the horizontal and vertical attention modules. p can be set to 1, the convolutional transformation operation can be set to 1x1, and linear activation can be implemented using a non-linear function, specifically the sigmoid function. Figure 3 , Figure 3 The diagram illustrates the processing of the module input by the horizontal and vertical attention modules. In this diagram, 11 is the feature map after convolution, batch normalization, and ReLU; 12 is the horizontal pooling feature map; 13 is the vertical pooling feature map; 14 is the intermediate pooling feature map; 15 is the spatial horizontal tensor; 16 is the spatial vertical tensor; and 17 is the reconstructed feature map.

[0116] This application provides a method, electronic device, and readable storage medium for constructing a global amplitude-preserving denoising and completion model for images. Specifically, it involves: acquiring a two-dimensional image pair consisting of an original sample image and its corresponding noisy, missing image; iteratively training a primary image reconstruction model with a preset two-level multi-scale model based on the two-dimensional image pair and a target loss function to obtain model reconstruction parameters and a model reconstruction image; wherein the target loss function consists of an image reconstruction model loss function and an image segmentation model loss function; updating the two-dimensional image pair based on the model reconstruction image to obtain a target image pair; and iteratively training a secondary image segmentation model with the preset two-level multi-scale model and the primary image reconstruction model based on the model reconstruction parameters, the target loss function, and the target image pair to obtain a target image interpretation model. Because the target loss function relied upon during iterative training of the pre-set two-level multi-scale model consists of the image reconstruction model loss function and the image segmentation model loss function, the primary image reconstruction model obtained through iterative training focuses more on global image features, while the two-level multi-scale model iteratively trained based on the primary image reconstruction model focuses more on local image features. At the same time, since the target image interpretation model is trained through multiple rounds of iterative training of the pre-set two-level multi-scale model, it can solve the problem of image amplitude missing and classification ambiguity. That is, the target image interpretation model can perform global amplitude-preserving denoising and completion on the input image, so that the image output by the target image interpretation model can characterize the global and local features of the image. This overcomes the technical defect that deep learning models can only capture local relationships, which leads to the inability to obtain reasonable results for interpolation of low-amplitude missing components. Therefore, it improves the interpretability of the denoised and completed image.

[0117] Example 2

[0118] Furthermore, referring to Figure 4 In another embodiment of this application, the content that is the same as or similar to that in Embodiment 1 described above can be referred to the above description and will not be repeated hereafter. Based on this, the step of obtaining a two-dimensional image pair composed of the original sample image and the corresponding noisy, missing image includes:

[0119] Step B10: Obtain the original sample image;

[0120] Step B20: Add independent and identically distributed random noise to at least one pixel of the original sample image to obtain the original noisy image;

[0121] Step B30: Based on the original noisy image, construct the noisy missing image, and combine the noisy missing image and the original sample image into the two-dimensional image pair, wherein the noisy missing image is as follows:

[0122]

[0123] Among them, D obs The noisy, missing image, The original noisy image, Let M represent the Hadamard product, and M represent the mask operator used to construct the missing image sample. The size of the matrix M is the same as that of the original sample image. The elements of M consist of 0 and 1, where 1 is used to indicate that the original sample image has a complete pixel at the corresponding position, and 0 is used to indicate that the original sample image has a missing pixel at the corresponding position.

[0124] In this embodiment, it should be noted that when only the original sample image is available, the corresponding noisy missing image can be constructed from the original sample image, and then the noisy missing image and the original sample image can be combined one by one into a two-dimensional image pair.

[0125] As an example, steps B10 to B30 include: acquiring the original sample image D, where D∈R n×m Where n and m are the number of pixels in the image horizontally and vertically, respectively, and R is the real number domain; independent and identically distributed random noise is added to at least one pixel of the original sample image to obtain the original noisy image; based on the original noisy image, the noisy missing image is constructed, and the noisy missing image and the original sample image are combined into the two-dimensional image pair, wherein the noisy missing image is as follows:

[0126]

[0127] Among them, D obs The noisy, missing image, The original noisy image, Let M represent the Hadamard product, and M represent the mask operator used to construct the missing image sample. The size of the matrix M is the same as that of the original sample image. The elements of M consist of 0 and 1, where 1 is used to indicate that the original sample image has a complete pixel at the corresponding position, and 0 is used to indicate that the original sample image has a missing pixel at the corresponding position.

[0128] In one feasible approach, after obtaining the two-dimensional image pairs, to prevent sample images from being used during model training and to improve the stability of the two-dimensional image pairs, min-max normalization can be used to normalize each pair of two-dimensional image pairs. The maximum pixel value in the normalization is the maximum value of all pixel values ​​of the input (noisy missing image) and label (original sample image) in the two-dimensional image pair, and the minimum pixel value in the normalization is the minimum value of all pixel values ​​of the input and label images in the image pair.

[0129] This application provides a method for obtaining two-dimensional image pairs. Specifically, it involves: acquiring the original sample image; adding independent and identically distributed random noise to at least one pixel of the original sample image to obtain an original noisy image; constructing the noisy missing image based on the original noisy image; and combining the noisy missing image and the original sample image into the two-dimensional image pair, wherein the noisy missing image is as follows: Among them, D obs The noisy, missing image, The original noisy image, Let M represent the Hadamard product, and M represent the mask operator used to construct the missing image sample. The matrix size of M is the same as that of the original sample image. The elements of M consist of 0 and 1, where 1 represents the integrity of the pixel at the corresponding position in the original sample image, and 0 represents the absence of the pixel at the corresponding position in the original sample image. This embodiment of the application performs noise processing on any pixel of the original sample image and autonomously constructs the image missing using the mask operator, thereby obtaining a two-dimensional image pair consisting of the original sample image and the corresponding noisy missing image. This achieves the goal of autonomously constructing the noisy missing image of the original sample image. Compared to directly acquiring two-dimensional image pairs, this improves the randomness of the two-dimensional image pairs. Since the two-dimensional image pairs are used to train a preset multi-scale model, this lays the foundation for improving the interpretability of the denoised and completed image.

[0130] Example 3

[0131] This application also provides an image global amplitude-preserving denoising and completion model construction apparatus, referring to... Figure 5 The image global amplitude-preserving denoising and completion model construction device includes:

[0132] The acquisition module 101 is used to acquire a two-dimensional image pair consisting of the original sample image and the corresponding noisy missing image;

[0133] The first training module 102 is used to iteratively train a primary image reconstruction model with a preset two-level multi-scale model based on the two-dimensional image pairs and the target loss function, so as to obtain model reconstruction parameters and model reconstruction images. The target loss function is composed of an image reconstruction model loss function and an image segmentation model loss function.

[0134] Update module 103 is used to reconstruct the image according to the model and update the two-dimensional image pair to obtain the target image pair;

[0135] The second training module 104 is used to iteratively train the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model based on the model reconstruction parameters, the target loss function and the target image pair, so as to obtain the target image interpretation model.

[0136] Optionally, the second training module 104 is further configured to:

[0137] Based on the image reconstruction model loss function and the reconstructed image, the reconstruction process parameters and the reconstructed image are obtained by training the primary image reconstruction model.

[0138] The secondary image segmentation model is updated by the parameters of the reconstruction process to obtain the image segmentation model to be trained;

[0139] Based on the original sample image, the reconstructed process image, and the image segmentation model loss function, the reconstruction process parameters and the model segmentation parameters of the reconstructed process image and the secondary image segmentation model are updated by training the image segmentation model to be trained.

[0140] Return to the execution steps: reconstruct the image according to the model, update the two-dimensional image pair to obtain the target image pair, until the target loss function converges, and obtain the target image interpretation model.

[0141] Optionally, the acquisition module is further configured to:

[0142] Obtain the original sample image;

[0143] Independent and identically distributed random noise is added to at least one pixel of the original sample image to obtain the original noisy image;

[0144] Based on the original noisy image, the noisy missing image is constructed, and the noisy missing image and the original sample image are combined into the two-dimensional image pair, wherein the noisy missing image is as follows:

[0145]

[0146] Among them, D obs The noisy, missing image, The original noisy image, Let M represent the Hadamard product, and M represent the mask operator used to construct the missing image sample. The size of the matrix M is the same as that of the original sample image. The elements of M consist of 0 and 1, where 1 is used to indicate that the original sample image has a complete pixel at the corresponding position, and 0 is used to indicate that the original sample image has a missing pixel at the corresponding position.

[0147] Optionally, the loss function of the image reconstruction model is as follows:

[0148]

[0149] in, The loss function for the image reconstruction model is... Assuming a pre-defined loss for the image reconstruction model, Let K be the image similarity index, K be the sampling order, and the loss of the preset image reconstruction model be as follows:

[0150]

[0151] Where ||·|‖1 is the 1-norm; θ F Indicates the preset image reconstruction process parameters; The model reconstructs the image; the image similarity metric is structural similarity loss, which is as follows:

[0152]

[0153] Wherein, D is the original sample image. The brightness measurement value between the original sample image and the model reconstructed image. The amplitude measurement value between the original sample image and the model reconstructed image. The contrast measurement between the original sample image and the model-reconstructed image.

[0154]

[0155]

[0156]

[0157] in, These are the average values ​​of the original sample image and the model-reconstructed image, respectively. These are the standard deviations of the original sample image and the model-reconstructed image, respectively. It is the covariance between the original sample image and the model reconstructed image, where c1, c2, and c3 are all constants, and the correlation between c1, c2, and c3 is as follows:

[0158] Optionally, the target loss function is as follows:

[0159]

[0160] Where ξ is the weight coefficient of the model loss function. Let be the loss function for the image segmentation model, wherein the image segmentation model loss function is as follows:

[0161]

[0162] in, c is the output of the inner product of feature maps i and j on the l-th convolutional layer of the primary image reconstruction model.l This represents the number of feature channels in convolutional layer l.

[0163] Optionally, both the primary image reconstruction model and the secondary image segmentation model are U-shaped encoder-decoder neural network models. The encoder portion of either the primary image reconstruction model or the secondary image segmentation model consists of m downsampling modules, which are arranged from shallowest to deepest as follows:

[0164] Conv(a1, b1, C)+BN+ReLU+Conv(a2, b1, c)+BN+ReLU+HVAM+Max

[0165] Conv(a2, b2, c)+BN+ReLU+Conv(a3, b2, c)+BN+ReLU+HVAM+Max

[0166] Conv(a3, b3, c)+BN+ReLU+Conv(a4, b3, c)+BN+ReLU+HVAM+Max

[0167] Conv(a4, b4, c)+BN+ReLU+Conv(a5, b4, c)+BN+ReLU+HVAM+Max

[0168]

[0169] Conv(a m b m c)+BN+ReLU+Conv(a m+1 b m c)+BN+ReLU+HVAM+Max

[0170] Where Conv() is the convolution operation, m is a constant, and a1, a2, a3, ... a m The number of input channels for different downsampling modules, b1, b2, b3, ... b m Here, c represents the number of output channels for different downsampling modules, c represents the side length of the square convolution kernel, Conv has a convolution stride of 1 and padding of 1 to ensure that the feature map size remains unchanged before and after convolution, BN represents batch normalization, ReLU is the activation function, HVAM is the horizontal and vertical attention module, after the feature map group passes through the horizontal and vertical attention module, the feature map size and the number of channels remain unchanged, Max represents max pooling, and the max pooling kernel size is 2x2 with a stride of 2;

[0171] The decoder part of the primary image reconstruction model or the secondary image segmentation model consists of one convolutional layer and m upsampling modules, wherein the m upsampling modules are arranged from shallowest to deepest as follows:

[0172] Up+Conv(a m+2 b m c)+BN+ReLU+Conv(a m+1 b m c)+BN+ReLU+HVAM

[0173]

[0174] Up+Conv(a5, b3, c)+BN+ReLU+Conv(a4, b3, c)+BN+ReLU+HVAM

[0175] Up+Conv(a4, b2, c)+BN+ReLU+Conv(a3, b2, c)+BN+ReLU+HVAM

[0176] Up+Conv(a3, b1, C)+BN+ReLU+Conv(a2, b1, c)+BN+ReLU+HVAM

[0177] In this process, after upsampling, the feature map size is doubled, and the number of channels is reduced to half. The convolutional layer is as follows:

[0178] Conv(a m+1 b m+1 c)+BN+ReLU

[0179] In this case, Conv has a convolution stride of 1 and padding of 1.

[0180] Optionally, the downsampling module has:

[0181]

[0182] in, For the downsampling module, The downsampled feature map output by the downsampling module. Here, k is a constant, representing the downsampling module parameter in the primary image reconstruction model. For the noisy, missing image, in the secondary image reconstruction model Reconstruct the image for the model;

[0183] The upsampling module has:

[0184]

[0185] in, For the upsampling module, This refers to the upsampled feature map output by the upsampling module. This represents the merged feature map of the encoder and decoder outputs at scale k, where Conc denotes merging. When k = K, in, These are the parameters of the upsampling module.

[0186] Optionally, the number of horizontal and vertical attention modules is the sum of the number of upsampling modules and the number of downsampling modules, and the horizontal and vertical attention modules have:

[0187]

[0188] in, For the horizontal and vertical attention modules, For k-th order scale or The feature map after convolutional layers, batch normalization, and ReLU is for The output AND For feature maps of the same size, the input X = [x1, x2, ..., x] of the horizontal and vertical attention modules C ]∈R C×H×W C represents the number of image channels, H represents the image height, and W represents the image width. The input to the horizontal and vertical attention modules is subjected to horizontal and vertical average pooling with sizes (H, p) and (p, W) to obtain horizontal and vertical pooling feature maps. The horizontal and vertical pooling feature maps have the following characteristics:

[0189]

[0190] in, The horizontal pooling feature map is the feature map of the c-th channel output with height h and size C×p×H. The vertical axis pooling feature map is a feature map of size C×W×p output by the c-th channel with width w, where p is the number of pooling layers;

[0191] By concatenating the horizontal-axis pooling feature map and the vertical-axis pooling feature map according to their spatial dimensions and then performing a convolutional transformation, a common intermediate pooling feature map is obtained, wherein the intermediate pooling feature map has:

[0192]

[0193] in, The concatenated feature map is the concatenated feature map of the horizontal axis pooling feature map and the vertical axis pooling feature map, where Γ is the convolution transformation operation and τ is the intermediate pooling feature map;

[0194] After batch normalization and nonlinear activation, the intermediate pooled feature map is divided according to the spatial dimension to obtain the spatial horizontal axis tensor and the spatial vertical axis tensor.

[0195] After performing convolution transformations on the spatial horizontal axis tensor and the spatial vertical axis tensor respectively, the horizontal axis attention weights and vertical axis attention weights are obtained using activation functions, wherein the horizontal axis attention weights and the vertical axis attention weights have:

[0196] ν h =sigmoid(Γ h (τ h )),ν w =sigmoid(Γ w (τ w ))

[0197] Where, τ h Let τ be the horizontal tensor of the space. w Let v be the spatial longitudinal tensor. h v represents the attention weight on the horizontal axis. w The vertical axis attention weight;

[0198] Based on the horizontal axis attention weights and the vertical axis attention weights, a reconstructed feature map corresponding to the noisy, missing image is determined, wherein the reconstructed feature map has:

[0199]

[0200] in, Let the horizontal attention weight of the c-th channel at width i be , Let y be the vertical attention weight of the c-th channel with width w at height j. c (i, j) is the reconstructed feature map, x c (i, j) represents the input feature map of the c-th channel at width i and width j.

[0201] The image global amplitude-preserving denoising and completion model construction apparatus provided by this invention employs the image global amplitude-preserving denoising and completion model construction method described in the above embodiments, solving the technical problem of poor interpretability of the denoised and completed image. Compared with the prior art, the beneficial effects of the image global amplitude-preserving denoising and completion model construction apparatus provided by this invention are the same as those of the image global amplitude-preserving denoising and completion model construction method described in the above embodiments, and other technical features in this image global amplitude-preserving denoising and completion model construction apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0202] Example 4

[0203] This invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image global amplitude preservation denoising and completion model construction method in Embodiment 1 above.

[0204] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0205] like Figure 6 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processor, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus.

[0206] Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0207] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0208] The electronic device provided by this invention employs the image global amplitude-preserving denoising and completion model construction method described in the above embodiments, thus solving the technical problem of poor interpretability of the denoised and completed image. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiments of this invention are the same as those of the image global amplitude-preserving denoising and completion model construction method described in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0209] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0210] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0211] Example 5

[0212] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the image global amplitude-preserving denoising and completion model construction method in the above embodiment.

[0213] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0214] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0215] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire a two-dimensional image pair consisting of an original sample image and a corresponding noisy, missing image; iteratively train a preset image reconstruction model based on the two-dimensional image pair and a target loss function to obtain a primary image reconstruction model, wherein the target loss function consists of an image reconstruction model loss function and an image segmentation model loss function; update the two-dimensional image pair based on the model reconstruction image output by the primary image reconstruction model to obtain a target image pair; iteratively train the primary image reconstruction model based on the target image pair and the target loss function to obtain a secondary image segmentation model; and combine the primary image reconstruction model and the secondary image segmentation model to obtain a target image interpretation model.

[0216] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0217] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0218] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0219] The computer-readable storage medium provided by this invention stores computer-readable program instructions for executing the above-described image global amplitude-preserving denoising and completion model construction method, thus solving the technical problem of poor interpretability of the denoised and completed image. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiments of this invention are the same as the beneficial effects of the image global amplitude-preserving denoising and completion model construction method provided by the above-described embodiments, and will not be repeated here.

[0220] Example 6

[0221] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image global amplitude preservation denoising and completion model construction method described above.

[0222] The computer program product provided in this application solves the technical problem of poor interpretability of denoised and incomplete images. Compared with the prior art, the beneficial effects of the computer program product provided in this embodiment are the same as those of the image global amplitude-preserving denoising and incomplete model construction method provided in the above embodiments, and will not be repeated here.

[0223] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for constructing a global amplitude-preserving denoising and completion model for images, characterized in that, The method for constructing the global amplitude-preserving denoising and completion model for images includes: The pre-defined two-level multi-scale model includes a secondary image segmentation model and a primary image reconstruction model; Obtain two-dimensional image pairs consisting of the original sample image and the corresponding noisy, missing image; Based on the two-dimensional image pairs and the target loss function, a primary image reconstruction model with a preset two-level multi-scale model is iteratively trained to obtain model reconstruction parameters and model reconstruction images. The target loss function consists of a primary image reconstruction model loss function and a secondary image segmentation model loss function. The image reconstruction model loss function is as follows: in, The loss function for the image reconstruction model is... Assuming a pre-defined loss for the image reconstruction model, , Let be the sampling order, where the loss of the preset image reconstruction model is as follows: in, It is a norm of 1; This indicates the preset image reconstruction process parameters; It is a model that reconstructs images. The noisy, missing image, This represents a mask operator used to construct missing image samples; the image similarity index is a structural similarity loss, which is as follows: Where D is the original sample image, The brightness measurement value between the original sample image and the model reconstructed image. The amplitude measurement value between the original sample image and the model reconstructed image. The contrast measurement between the original sample image and the model-reconstructed image. in, These are the average values ​​of the original sample image and the model-reconstructed image, respectively. , These are the standard deviations of the original sample image and the model-reconstructed image, respectively. It is the covariance between the original sample image and the model-reconstructed image. All are constants. The relationship between them is as follows: The target loss function is as follows: in, These are the weight coefficients of the model's loss function. Let be the loss function for the image segmentation model, wherein the image segmentation model loss function is as follows: in, For the primary image reconstruction model, the first Feature maps on convolutional layers and The output of the inner product, Represents a convolutional layer The number of feature channels; The image is reconstructed based on the model, and the two-dimensional image pair is updated to obtain the target image pair; Based on the model reconstruction parameters, the target loss function, and the target image pair, the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model are iteratively trained to obtain the target image interpretation model; The step of iteratively training the secondary image segmentation model and the primary image reconstruction model of the preset two-level multi-scale model based on the model reconstruction parameters, the target loss function, and the target image pair to obtain the target image interpretation model includes: Based on the image reconstruction model loss function and the reconstructed image, the reconstruction process parameters and the reconstructed image are obtained by training the primary image reconstruction model. The secondary image segmentation model is updated by the parameters of the reconstruction process to obtain the image segmentation model to be trained; Based on the original sample image, the reconstructed process image, and the image segmentation model loss function, the reconstruction process parameters and the model segmentation parameters of the reconstructed process image and the secondary image segmentation model are updated by training the image segmentation model to be trained. Return to the execution steps: reconstruct the image according to the model, update the two-dimensional image pair to obtain the target image pair, until the target loss function converges, and obtain the target image interpretation model; The target image interpretation model is trained by iteratively training a pre-set two-level multi-scale model through multiple rounds of training, which can perform global amplitude-preserving denoising and completion of the input image.

2. The image global amplitude-preserving denoising and completion model construction method as described in claim 1, characterized in that, The step of obtaining a two-dimensional image pair consisting of the original sample image and the corresponding noisy missing image includes: Obtain the original sample image; Independent and identically distributed random noise is added to at least one pixel of the original sample image to obtain the original noisy image; Based on the original noisy image, the noisy missing image is constructed, and the noisy missing image and the original sample image are combined into the two-dimensional image pair, wherein the noisy missing image is as follows: in, The original noisy image, It represents the Hadamah accumulation. The matrix size is the same as the original sample image. The elements are composed of 0 and 1, where 1 is used to indicate that the original sample image has a complete pixel at the corresponding position, and 0 is used to indicate that the original sample image has a missing pixel at the corresponding position.

3. The image global amplitude-preserving denoising and completion model construction method as described in claim 1, characterized in that, Both the primary image reconstruction model and the secondary image segmentation model are U-shaped encoder-decoder neural network models. The encoder portion of either the primary image reconstruction model or the secondary image segmentation model consists of m downsampling modules, which are arranged from shallowest to deepest as follows: Conv( , ,c) + BN + ReLU + Conv( , ,c) + BN + ReLU + HVAM + Max Conv( , ,c) + BN + ReLU + Conv( , ,c) + BN + ReLU +HVAM + Max Conv( , ,c) + BN + ReLU + Conv( , ,c) + BN + ReLU +HVAM + Max Conv( , ,c) + BN + ReLU + Conv( , ,c) + BN + ReLU +HVAM + Max Conv( , ,c) + BN + ReLU + Conv( , ,c) + BN + ReLU +HVAM + Max Where Conv() is the convolution operation, and m is a constant. The number of input channels for different downsampling modules. , , , Here, c represents the number of output channels for different downsampling modules, c represents the side length of the square convolution kernel, Conv has a convolution stride of 1 and padding of 1 to ensure that the feature map size remains unchanged before and after convolution, BN represents batch normalization, ReLU is the activation function, HVAM is the horizontal and vertical attention module, after the feature map group passes through the horizontal and vertical attention module, the feature map size and the number of channels remain unchanged, Max represents max pooling, and the max pooling kernel size is 2x2 with a stride of 2; The decoder part of the primary image reconstruction model or the secondary image segmentation model consists of one convolutional layer and m upsampling modules, wherein the m upsampling modules are arranged from shallowest to deepest as follows: Up + Conv( , ,c) + BN +ReLU + Conv( , ,c) + BN +ReLU + HVAM Up + Conv( , ,c) + BN +ReLU + Conv( , ,c) + BN +ReLU + HVAM Up + Conv( , ,c) + BN +ReLU + Conv( , ,c) + BN +ReLU + HVAM Up + Conv( , ,c) + BN +ReLU + Conv( , ,c) + BN +ReLU + HVAM In this process, after upsampling, the feature map size is doubled, and the number of channels is reduced to half. The convolutional layer is as follows: Conv( , ,c) + BN +ReLU In this case, Conv has a convolution stride of 1 and padding of 1.

4. The image global amplitude-preserving denoising and completion model construction method as described in claim 3, characterized in that, The downsampling module has: in, For the downsampling module, The downsampled feature map output by the downsampling module. Here, k is a constant, representing the downsampling module parameter in the primary image reconstruction model. For the noisy, missing image, in the secondary image segmentation model Reconstruct the image for the model; The upsampling module has: in, For the upsampling module, This refers to the upsampled feature map output by the upsampling module. This represents the merged feature map of the encoder and decoder outputs at a scale of k. Indicates merging, in k= hour, ,in, These are the parameters of the upsampling module.

5. The image global amplitude-preserving denoising and completion model construction method as described in claim 4, characterized in that, The number of horizontal and vertical attention modules is the sum of the number of upsampling modules and the number of downsampling modules, and the horizontal and vertical attention modules have: in, For the horizontal and vertical attention modules, For k-th order scale or The feature map after convolutional layers, batch normalization, and ReLU is for The output AND For feature maps of the same size, the inputs of the horizontal and vertical attention modules C represents the number of image channels, H represents the image height, and W represents the image width. The input to the horizontal and vertical attention modules is subjected to horizontal and vertical average pooling with sizes (H, p) and (p, W), respectively, to obtain horizontal and vertical pooling feature maps. The horizontal and vertical pooling feature maps have the following characteristics: in, The horizontal pooling feature map is the feature map of the c-th channel output with height h and size C×p×H. The vertical axis pooling feature map is a feature map of size C×W×p output by the c-th channel with width w, where p is the number of pooling layers; By concatenating the horizontal-axis pooling feature map and the vertical-axis pooling feature map according to their spatial dimensions and then performing a convolutional transformation, a common intermediate pooling feature map is obtained, wherein the intermediate pooling feature map has: in, This is a concatenated feature map of the horizontal-axis pooling feature map and the vertical-axis pooling feature map. This is a convolution transformation operation. The intermediate pooling feature map; After batch normalization and nonlinear activation, the intermediate pooled feature map is divided according to the spatial dimension to obtain the spatial horizontal axis tensor and the spatial vertical axis tensor. After performing convolution transformations on the spatial horizontal axis tensor and the spatial vertical axis tensor respectively, the horizontal axis attention weights and vertical axis attention weights are obtained using activation functions, wherein the horizontal axis attention weights and the vertical axis attention weights have: in, Let the space horizontal axis tensor be... Let be the tensor of the spatial vertical axis. The attention weights on the horizontal axis are... The vertical axis attention weight; Based on the horizontal axis attention weights and the vertical axis attention weights, a reconstructed feature map corresponding to the noisy, missing image is determined, wherein the reconstructed feature map has: in, For the c-th channel in width Attention weights on the horizontal axis at the location, For the c-th channel with width w, at height w Attention weights on the vertical axis at the location, For the reconstructed feature map, In width Place and The input feature map at that location.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the image global amplitude-preserving denoising and completion model construction method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that implements a method for constructing a global image amplitude-preserving denoising and completion model. The program that implements the method for constructing a global image amplitude-preserving denoising and completion model is executed by a processor to implement the steps of the method for constructing a global image amplitude-preserving denoising and completion model as described in any one of claims 1 to 5.

Images