A method for denoising the lunar permanent shadow region based on physical noise model

Abstract

The invention discloses a denoising method for a permanent shadow area of ​​the moon based on a physical noise model. A denoising dual network model is constructed, which comprises a noise recognition network model, a preliminary denoising module and a final denoising dual network model. The noise recognition network model takes a shadow area image to be denoised as input, and is used to identify mixed noises including dark noise, compression noise and reading noise; the preliminary denoising module is used to remove mixed noises from the shadow area image to be denoised, and then correct the DN value of the removed image to obtain a remaining shadow area image, and complete preliminary denoising; the final denoising dual network model takes the remaining shadow area image as input, and is used to identify a clean image, and complete final denoising; the noise recognition model and the final denoising model are trained; and the trained denoising dual network model is used to perform denoising on the shadow area image to be denoised, and obtain the shadow area image after final denoising.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to a method for denoising a lunar permanent shadow region based on a physical noise model. Background Art

[0002] Lunar exploration has been one of the research focuses of mankind since ancient times. Clear lunar surface images are an important part of human exploration of the moon. They can provide more accurate topographic, geological and geomorphic information, which is crucial for understanding the formation and evolution of the moon, exploring lunar resources and ensuring the safe landing of landers and probes in lunar landing missions. The permanent shadow area of ​​the moon is a special area usually located in the deep areas around the north and south poles of the moon. Since this area is an effective shielding area for external factors such as cosmic rays and solar wind, it may contain important information about the history, geology and solar system of the moon, and the water ice resources in the area may also provide important support for future space exploration. However, due to the poor lighting conditions in the permanent shadow area of ​​the moon, the images of the permanent shadow area of ​​the moon currently taken have too much noise, which obscures various terrain and geomorphic information. Therefore, it is necessary to carry out research on denoising of the permanent shadow area of ​​the moon.

[0003] At present, many research teams around the world have used different methods to reduce the noise of images. In the field of traditional methods, there are many denoising methods, such as filter-based denoising methods and model-based denoising methods. Among them, the filter-based denoising method mainly achieves the purpose of denoising by filtering the image in the space or frequency domain. With the development of deep learning technology, more and more applications have begun to use end-to-end deep learning methods to solve the denoising problem. These methods usually perform well under various noise conditions.

[0004] Currently commonly used deep learning denoising models include convolutional neural networks (CNN), generative adversarial networks (GANs), and methods that combine physical noise models with deep learning. In 2022, the U-Net denoising software HORUS developed by Ben Moseley's team, HORUS is based on a deep learning algorithm. Through physical noise models, real noise samples, and 3D ray tracing scene selection, the model is adjusted according to the camera's environmental metadata during image capture to generate effective training data for training. The authors used environmental information (such as charge-coupled device (CCD) temperature and track number) and the physical noise model of NAC to remove CCD-related noise and photon noise from low-light full-resolution LROC NAC regular pattern and summation pattern images in shadow areas. This work is the first to study the small-scale topography and trafficability of the lunar PSR, which has important scientific value in the lunar exploration program. The overall structure diagram of the network model used for this method is shown below. Figure 1However, this method also has the following defects:

[0005] 1. The DestripeNet network convolves a one-dimensional field into a corresponding noise map through multiple deconvolution operations, so it is difficult to recover the specific details of the noise and it is easy to only predict the dark noise of the vertical bars;

[0006] 2. The quality of DestripeNet prediction results will greatly affect the subsequent PhotonNet network denoising;

[0007] 3. The U-Net network used by PhotonNet is too simple and its feature extraction capability is relatively weak. Summary of the invention

[0008] The present invention provides a denoising method for the permanent shadow area of ​​the moon based on a physical noise model, which solves the technical problems of the existing denoising methods, such as the network model is too simple, the feature extraction capability is relatively weak, and the denoising effect is poor.

[0009] The present invention can be achieved through the following technical solutions:

[0010] A method for denoising the permanent shadow region of the moon based on a physical noise model comprises the following steps:

[0011] Step 1: Build a denoising dual network model

[0012] The denoising dual network model includes a noise recognition network model, a preliminary denoising module and a final denoising dual network model. The noise recognition network model takes the shadow area image to be denoised as input, and is used to identify mixed noise including dark noise, compression noise and read noise; the preliminary denoising module is used to remove mixed noise from the shadow area image to be denoised, and then correct the DN value of the removed image to obtain the remaining shadow area image, and complete the preliminary denoising; the final denoising dual network model takes the remaining shadow area image as input, and is used to identify the clean image and complete the final denoising;

[0013] Step 2: Train the noise recognition model and the final denoising model

[0014] S21, artificially constructing a simulated shadow area image I to be denoised containing various noises and a mixed noise image Nm including only dark noise, compression noise and read noise to form a first data set, taking the image I to be denoised as input and the mixed noise image Nm as output, and training the noise recognition network model;

[0015] S22, using the trained noise recognition model to identify the simulated denoising shadow area images I in the first data set one by one, obtain the corresponding mixed noise image Nm, and then use the preliminary denoising module to process it to obtain the corresponding residual shadow area image Nn, and artificially construct a clean image S, that is, an image that does not include any noise, to form a second data set, and use the residual shadow area image Nn as input and the clean image S as output to train the final denoising dual network model;

[0016] Step 3: Use the trained noise recognition network model and the final denoising dual network model together with the preliminary denoising module to denoise the denoised shadow area image to obtain the final denoised shadow area image.

[0017] Furthermore, the noise recognition network model and the final denoising dual network model have the same structure, both of which are based on the UNet network, and the jump connections in the UNet network are replaced by full-scale jump connections, and the convolution is replaced by the LeWInTransformer module.

[0018] Furthermore, the simulated shadow area image I to be denoised, the mixed noise image Nm, and the clean image S are all constructed based on a physical noise model constructed based on the LROCNAC imaging principle.

[0019] Furthermore, using the preliminary denoising module, the mixed noise image Nm is first subtracted from the simulated shadow area image I to be denoised, and then the image DN value is corrected to linearity using the following formula, and then divided by the flat field correction coefficient F to achieve uniformity of the image DN value;

[0020]

[0021] Where x represents the DN value of the recorded pixel, and a, b, c, and d are free parameters estimated through experimental calibration and are known parameters.

[0022] A computer storage medium includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the lunar permanent shadow region denoising method based on the physical noise model as described above.

[0023] An electronic device comprises one or more processors, one or more memories and one or more applications; wherein the one or more applications are stored in the memories, and when the one or more programs are executed by the processor, the electronic device performs the lunar permanent shadow area denoising method based on the physical noise model as described above.

[0024] The beneficial technical effects of the present invention are:

[0025] 1. Different from the prior art method of HORUS which predicts dark noise by using the vector of 8 selected environmental metadata fields available when the image is captured, the present invention uses the physical noise model based on the property assumption of NAC of the CCD sensor of LROC to realize the quantification and visualization of all noises including dark noise, and learns the mixture of the three kinds of noise, namely, read noise, compression noise and dark noise, as the true value of the preliminary denoising network model, so that the noise recognition network model can recognize not only dark noise but also compression noise and read noise, effectively remove most of the noise in the preliminary denoising module, and reduce the pressure of the final denoising network.

[0026] 2. The network model framework of D-UFormer Plus is cleverly designed, and the Unet network is improved using the ideas of full-scale skip connections and Transformer to enhance the ability to capture local and global low-frequency information of noise maps.

[0027] 3. The D-UFormer Plus neural network model with the same structure is used as the noise recognition network model and the final denoising dual network model to perform the tasks of identifying mixed noise and denoising the residual noise respectively. According to the recognition result of the noise recognition network model, the preliminary noise module is used to perform preliminary denoising on the denoised shadow area image. This dual network framework enables each network to better model and understand the noise in the input data, which helps to more accurately identify and process noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the network model structure in the prior art;

[0029] Figure 2 A schematic diagram of the process of synthesizing artificial noise of a shadow image to be denoised according to the present invention;

[0030] Figure 3 It is a structural schematic diagram of the denoising dual network model of the present invention;

[0031] Figure 4 It is a schematic diagram of a local full-scale skip connection of the present invention;

[0032] Figure 5 It is a schematic diagram of the structure of the LeWin Transformer module of the present invention;

[0033] Figure 6 It is a structural schematic diagram of the LeFF module of the present invention;

[0034] Figure 7(a)-Figure 7(c) They are schematic diagrams of the result of denoising a simulated shadow area image using the denoising method of the present invention;

[0035] Figure 8(a)-Figure 8(c) They are schematic diagrams of the result of denoising an actual shadow area image using the denoising method of the present invention. DETAILED DESCRIPTION

[0036] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings and preferred embodiments.

[0037] The present invention provides a method for denoising the permanent shadow region of the moon based on a physical noise model. First, the physical noise model proposed by the Ben Moseley team in the open document Extreme Low-Light Environment-Driven Image Denoising over Permanently Shadowed Lunar Regions with a Physical Noise Model is used as a theoretical basis to generate a data set, then a denoising dual network model is constructed and trained, and finally the trained denoising dual network model is used to perform mixed noise identification, preliminary denoising and final denoising in sequence, thereby completing the denoising process. The details are as follows:

[0038] Step 1: Build a dataset

[0039] The main reason that hinders the denoising of the lunar permanent shadow area images is that there are no corresponding noise-free images for network training. To address this problem, we use the physical noise model of the LROC NAC imaging principle to simulate the lunar permanent shadow area images. Since the imaging quality of the lunar permanent shadow area images captured by LROC is relatively poor, it is ultimately because the number of photons reflected back from the low-light areas of the moon is very limited, which leads to the dominance of photon noise and a series of noises generated by the LROC sensor. The present invention uses the physical noise model proposed by Ben Moseley's team in the public document Extreme Low-Light Environment-Driven Image Denoising over PermanentlyShadowed Lunar Regions with a Physical Noise Model as the theoretical basis to generate a data set. The physical noise model is formulated based on the NAC imaging principle of the LROC CCD sensor, and its noise synthesis process is as follows. Figure 2 As shown:

[0040] I=N*(F*(S+N p ))+(N d +T)+N b +N r +N c

[0041] Among them, the average photon signal S is the original image we need to synthesize the data set, that is, the clean image, I is the simulated low-light area image, that is, the simulated shadow area image to be denoised; S is the average photon signal; N p is photon noise; F is flat field response; N is nonlinear response; (N d +T) is dark current noise; N b is the dark bias noise; N r is the read noise; N c is the compression noise, and T is the temperature of the CCD element in the NAC camera.

[0042] Photon noise N p It is caused by the randomness of the photons reaching the CCD sensor and follows

[0043]

[0044] The above represents a Poisson distribution. The strength of this noise depends on the average rate at which photons hit the CCD sensor. Therefore, we use random Poisson noise to add to S to simulate the N generated by the camera when shooting in a low-light environment. p .

[0045] When LROC NAC records the DN value of a pixel, it generally has a linear response to the average photon signal S. However, for DN count images below 600DN, there is no linear relationship between the output DN value and the input average photon signal S. This nonlinearity is caused by the camera sensor and electronic components of LROC NAC, resulting in low brightness of the captured image. Therefore, a nonlinear response correction function can be used to correct this phenomenon. The formula is as follows:

[0046]

[0047] Where x is the DN value of the recorded pixel, N -1 (x) is the true DN value estimated from the recorded DN value. a, b, c, and d in the formula are free parameters estimated through experimental calibration and are known parameters. Since the above noise addition process needs to obtain the recorded DN value based on the known true DN value, it is necessary to add N -1 Taking the difference gives the inverse function.

[0048] The flat field response of LROC NAC refers to the phenomenon that the brightness of the captured image is uneven due to the uneven response of each pixel during the shooting process of the camera. The flat field effect will affect the quality of the image, especially in scientific research and precise measurement, and needs to be corrected. Flat field correction is used to correct the flat field response. This parameter is obtained by multiplying the positive response by the flat field response F pixel by pixel. Since we choose the noise-free illuminated area image as the noise addition object, this image only contains the positive response, so we need to multiply it by F. We use the pictures M106224610L and M106224610R taken specifically for calculating flat field correction to make a flat field table. After taking the average value of each column in the image, it is normalized by the average value of the 500 columns in the center of the image, so that the average value of a group of columns in the center of the image is 1.0, and a 5064*1 table is calculated to correspond to the normal mode of the NAC camera. For the summation mode, a 2532*1 flat field table corresponding to the summation mode is calculated by averaging the adjacent pixels in the 5064*1 flat field table in the normal mode.

[0049] Read Noise N r It is the random system noise introduced during the conversion of charge carriers into an analog voltage signal.

[0050] Dark noise is composed of dark current noise (N d +T) and dark bias noise N b The dark current noise is generated by thermally activated charge carriers in the CCD sensor and appears as horizontal and vertical stripes on the image. Dark bias noise is a deterministic noise source and appears as vertical stripes on the image. We randomly extract the dark noise frame D taken by NAC as the dark noise to add, which contains dark current noise, dark bias noise and read noise. Since the dark noise frame uses a one-to-one mapping method to compress the 12-bit image to 8-bit, no compression noise N is generated. c , so the specific relationship is as follows:

[0051]

[0052] Compression noise N c Before the satellite is downlinked to the earth, it uses a lossy compression scheme to compress the NAC image from 16 bits to 8 bits, which will cause noise. We chose NAC's compression scheme 3 because it is specifically designed for NAC images with DN < 500. We use compression scheme 3 to compress and decompress our synthetic noise image to generate this noise.

[0053] In summary, given a clean image representing the average photon signal And generate its corresponding noise image When, as shown below:

[0054]

[0055] Step 2: Build a denoising dual network model

[0056] After constructing the image of the permanent shadow area of ​​the moon, we have the image pairs of noisy and noise-free images of the permanent shadow area of ​​the moon for training. In order to better denoise the image, we constructed a denoising dual network model based on the ideas of UNet structure, full-scale skip connection and self-attention model (Transformer). The network model proposed in this paper is called D-UFormer Plus. Figure 3 shown.

[0057] The denoising dual network model includes a noise recognition network model, a preliminary denoising module and a final denoising dual network model. The noise recognition network model takes the shadow area image to be denoised as input, and is used to identify mixed noise including dark noise, compression noise and read noise; the preliminary denoising module is used to remove mixed noise from the shadow area image to be denoised, and then correct the image DN value of the removed image to obtain the remaining shadow area image to complete the preliminary denoising; the final denoising dual network model takes the remaining shadow area image as input, and is used to identify the clean image to complete the final denoising; wherein, the noise recognition network model and the final denoising dual network model have the same structure, both of which are based on the UNet network, and the jump connection in the UNet network is replaced by the full-scale jump connection, and the convolution is replaced by the LeWIn Transformer module.

[0058] The details are as follows:

[0059] Since the simple same-level skip connections in UNet are prone to lack the ability to learn enough information from the full scale, we use the full-scale skip connection idea to improve the network's ability to learn full-scale information. The full-scale skip connection changes the interconnection between the encoder and the decoder and the internal connection between the decoder subnets, so that each decoder layer integrates the small-scale and same-scale feature maps from the encoder and the large-scale feature maps from the decoder. The feature maps extracted by each layer of the decoder capture the fine-grained and coarse-grained semantics at the full scale, such as Figure 4 shown.

[0060] When i represents the i-th downsampling layer along the encoding direction and N represents the number of encoders, a network with full-scale skip connections is used to decode part of the feature map. The calculation formula is as follows;

[0061]

[0062] Among them, the function C(.) represents the convolution operation, which is used to perform downsampling and same-scale mapping respectively; the function TC(.) represents the deconvolution operation, which is used to perform upsampling; the function LWT(.) represents the use of the LeWin Transformer module for feature fusion; [] indicates that the feature maps are stacked in the channel dimension.

[0063] At the same time, we use the Transformer module to replace the two-dimensional convolution to extract features from the feature map. However, due to the high computational cost and the lack of ignoring local information of the standard Transformer [15,16,17,18], and in order for our network to capture more detailed texture details in the multi-scale feature map with a smaller amount of computation to restore the terrain information masked by noise, we use the LeWin Transformer for image restoration. The LeWin Transformer module uses a self-attention model to capture long-distance information, and also adds a convolutional layer to capture important local context information. The LeWin Transformer module is mainly built based on two core designs: Window-based Multi-head Self-Attention and Locally-enhanced Feed-Forward Network. If a feature map X at the l-1 position is given l-1 , the calculation of LeWinTransformer can be expressed as follows.

[0064] X′ l =W-MSA(LN(X l-1 ))+X l-1

[0065] X l =LeFF(LN(X′ l ))+X′ l

[0066] Among them, X l ′ and X l are the outputs of the W-MSA module and the LeFF module respectively. The structure of LeWin Transformer is as follows Figure 5 shown.

[0067] W-MSA is different from the global self-attention mechanism of the standard Transformer. It performs self-attention on non-overlapping local windows, which can effectively reduce the amount of computation. Given a 2D input feature X∈R C×H×W , W-MSA first splits X into M×M non-overlapping blocks and obtains the flattened and transposed features of each window After that, self-attention is performed on each window feature, and the k-th head calculation process can be described as follows;

[0068] X={X 1 ,X 2 ,…,X N}

[0069]

[0070] in, is the projection matrix of QKV;

[0071] Finally, the results of all heads are concatenated and linearly projected to obtain the final W-MSA output. According to existing research, relative position encoding is introduced in this module, so the calculation process of self-attention can be expressed as Formula 9.

[0072]

[0073] LeFF is a feedforward neural network that can learn local context information based on neighboring pixels in an image. Its structure is as follows: Figure 6 As shown in the figure, LeFF is improved by inserting a deep separable convolution in the middle of the feed-forward neural network (Feed-Forward Network) in the standard Transformer. It overcomes the shortcomings of FFN's insufficient ability to utilize local context information and can fully use the neighboring pixels in the image for image denoising. First, LeFF uses a linear projection layer to increase the dimension of each word, then reshapes the word into a 2D feature and uses a 3×3 deep separable convolution to capture local information, and finally flattens the above features and shrinks the dimension through another linear layer to match the input channel and size.

[0074] Step 3: Train the noise recognition model and the final denoising model

[0075] S21, artificially construct a simulated shadow area image I to be denoised containing various noises and a mixed noise image Nm including only dark noise, compression noise and read noise to form a first data set, take the image I to be denoised as input and the mixed noise image Nm as output, and train the noise recognition network model, as follows:

[0076] The input of the noise recognition network model is the shadow area image I to be denoised with a size of 256*256; the true value is the mixed noise image N with a size of 256*256 mixed by dark noise, compression noise and read noise. m , this mixed noise image is the image of the shadow area to be denoised I minus the image with uncorrected Poisson noise The shadow image I to be denoised is first obtained through the five-layer encoding part The subsequent decoding part is the result of the interconnection between the encoder and decoder and the inner connection between the decoder subnetworks. They are all convolved by stacking the results of all the previous encoders and decoders. If the size of the feature map does not match the number of channels, the maximum pooling or linear interpolation method is used for upsampling and then convolved with a convolution kernel of size 3. Finally, after the forward propagation is completed, the predicted result and the true value are handed over to the loss function, and then backward propagated. After N epochs of training, we can get the mixed noise image predicted and recognized by the network. It can be expressed as:

[0077]

[0078] S22, using the trained noise recognition model to identify the simulated denoising shadow area images I in the first data set one by one, obtain the corresponding mixed noise image Nm, and then use the preliminary denoising module to process it to obtain the corresponding residual shadow area image Nn, and artificially construct a clean image S, that is, an image that does not include any noise, to form a second data set, and use the residual shadow area image Nn as input and the clean image S as output to train the final denoising dual network model;

[0079] A. Preliminary denoising module

[0080] In this module, we get the mixed noise image of dark noise, compression noise and read noise identified by the noise recognition network model. According to the above-mentioned assumed LROC NAC physical noise model, we need to subtract this mixed noise image from the simulated shadow area image I to be denoised Then the obtained result is introduced into the nonlinear response correction function N -1 (x) to remove the nonlinear response to correct the nonlinearity of the image DN value, and finally divide it by the flat field value F to remove the flat field response and achieve the uniformity of the image DN value. After the calculation, the noise in the simulated shadow area image I to be denoised only contains the Poisson noise N p and a small amount of mixed noise N o Image It can be expressed as:

[0081]

[0082] B. Final denoising dual network model

[0083] The network input here is also a pair of images of size 256*256. One is the image to be denoised that only contains Poisson noise and a small amount of mixed noise. The other is the original image as the true value The network used here is also consistent with the noise recognition network model structure above, focusing on removing Poisson noise N p and a small amount of mixed noise N o After N epochs of training, we can get a clean image S of the average photon signal predicted by the network. It can be expressed as:

[0084]

[0085] Due to equipment limitations, the number of data sets we can generate is limited, so we choose the K-fold cross-validation method to train the model. This method can evaluate and verify the performance of the machine learning model on a limited data set, while helping to select the best model parameters and reduce the risk of overfitting, thereby improving the generalization ability and robustness of the model.

[0086] C. Loss Function

[0087] When training the network, we still use L MS-SSIM+L1 As the loss function, we ensure that the result is highly similar to the original image and the difference is small. In the training stage, the structural similarity index is the most important indicator for verifying the denoised image in this experiment. Using MS-SSIM can well preserve the edges and details of the image, but it will cause the brightness of the denoised image to change and the color to deviate. The L1 loss function can better ensure that the brightness and color of the denoised image do not change. Therefore, we choose to use the MS-SSIM+L1 loss function as the loss function for denoising network training. The specific formula is as follows:

[0088]

[0089] Among them, α is the ratio of the MS-SSIM loss function and the L1 loss function. We directly use the experimental value of 0.84 as the ratio of these two loss functions. G is the Gaussian distribution parameter. We selected 5 different values ​​to replace the complex pyramid structure to optimize the calculation efficiency. Each value is twice the previous one, so G = [0.5, 1, 2, 4, 8]. The L1 loss function will be multiplied by each value in G one by one.

[0090] LROC NAC images have a target name attribute, which has three values: Moon (MOON), Starry Sky (STAR), and Correction (CAL). Since the images with the target name CAL were taken by LROC NAC in a completely dark environment for dark current testing, they are used to help identify and correct the noise level of the sensor. Therefore, we collected all the data of this type for the whole month and cropped them to obtain 71,072 dark noise frames. These dark noise images were randomly selected as dark noise for data set synthesis. We also selected 1,191 images that met the screening criteria from the full-month data as noise-free images for data set synthesis. Finally, the data set has a total of 77,310 image pairs, which are divided into training set, validation set, and test set for training in a ratio of 72:18:10.

[0091] Step 4: Use the trained noise recognition network model and the final denoising dual network model together with the preliminary denoising module to denoise the denoised shadow area image to obtain the final denoised shadow area image.

[0092] The present invention also provides an electronic device for executing the above-mentioned lunar permanent shadow area denoising method based on the physical noise model, thereby achieving the same effect as the above-mentioned implementation method.

[0093] In the case of using integrated units, the electronic device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the above-mentioned display unit, acquisition unit and processing unit. The storage module can be used to support the electronic device to execute stored program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0094] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0095] The present invention also provides a computer storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the lunar permanent shadow area denoising method based on the physical noise model in the above-mentioned embodiment.

[0096] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0097] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0098] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0100] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0102] In order to verify the feasibility of the denoising method of the present invention, we conducted the following experiments:

[0103] Experimental results based on simulated images

[0104] Figure 7(a)-Figure 7(c) The results of different denoising methods are shown. As can be seen from the figure, BDNet without a skip connection structure can only remove some noise, and the denoising effect is poor; UNet with a skip connection can remove noise better than BDNet, and the terrain is clearer, but it introduces smaller point noise, which shows that the simple UNet has limitations in denoising such noisy images; UNet3Plus removes most of the noise, and the terrain can be restored to some extent, but Figure 7(a) and Figure 7(b) show that UNet3Plus will lose some information about smaller craters or hills, resulting in the loss of terrain information; although DRANet can remove noise, it is difficult to process the edges of the image, resulting in the introduction of black edges and stripes at the edges of the denoised image. At the same time, Figure 7(a) and Figure 7(b) show that DRANet will lose most of the terrain information, which also leads to the loss of terrain information; UFormer can remove noise better and will not introduce new noise, but compared with the results of UNet3Plus, there will still be a lot of unremoved noise. The method of the present invention combines the advantages of UNet3Plus and UFormer, and can remove most of the noise on the image. The denoised image has a good visual effect and does not lose too much landform information.

[0105] The quantitative results of denoising the simulated images using different denoising methods are shown in Table 2. The quantitative evaluation indicators are MAE, PSNR and SSIM. The results are the average values ​​of all test set images, where the best denoising results are in bold and the second best denoising results are in underline. It can be seen from the table that the method of the present invention shows stable performance on the simulated data set, while other denoising methods can denoise to a certain extent and restore the landform information covered by noise, but the denoising effect is not as good as the method of the present invention.

[0106] Table 2 Quantitative index results of simulated imaging experiment

[0107]

[0108] Note: 1) The best denoising results are in bold, and the second best denoising results are in underlined; 2) Except for the MAE indicator, the higher the other indicators, the better

[0109] Experimental results based on real images

[0110] Figure 8(a)-Figure 8(c) The result graphs of different denoising methods are shown. As can be seen from the figure, BDNet has a very poor denoising effect on real images, and it is even unable to denoise the noisy image in Figure 8(b); UNet can denoise better than BDNet, but it cannot effectively denoise some areas of the noisy image in Figure 8(b), resulting in holes. At the same time, Figures 8(a) and 8(c) also show that UNet still has a small amount of noise that has not been removed; UNet3Plus removes almost all noise, but does not properly smooth the denoised image, resulting in the denoised image being too bright at the top and too dark at the bottom, which also causes a lot of landform information to be covered up; DRANet also has a good denoising effect, but Figure 8(b) shows that this network is not sensitive to vertical stripe noise, resulting in more residual noise and introducing stripes at the edge of the image; UFormer restores more details while removing most of the noise, and properly smoothes the denoised image; The method of the present invention restores more details than UFormer while retaining the advantages of UFormer.

[0111] The quantitative results of denoising real images using different denoising methods are shown in Table 3. The quantitative evaluation indicators are IL-NIQE, NRQM and MUSIQ. The results are the average values ​​of all real images, where the bold marks are the best denoising results, and the underlined marks are the second best denoising results. It can be seen from Table 3 that the method of the present invention obtains higher MRQM and MUSIQ, while the biggest reason for the IL-NIQE of UNet3Plus is that the method does not perform moderate smoothing on the denoised image, which increases the variance of the grayscale value in the selected range, making the IL-NIQE value smaller, resulting in the phenomenon of missing landform information but higher indicators.

[0112] Table 3 Quantitative index results of real image experiments

[0113]

[0114] Note: 1) The best denoising results are in bold, and the second best denoising results are in underlined; 2) Except for the IL-NIQE index, the higher the other indexes, the better

[0115] Although specific embodiments of the present invention are described above, those skilled in the art should understand that these are merely examples and that various changes or modifications may be made to these embodiments without departing from the principles and essence of the present invention. Therefore, the scope of protection of the present invention is limited by the appended claims.

Claims

1. A method for denoising the permanent shadow area of ​​the moon based on a physical noise model, characterized in that The following steps are involved: Step 1: Build a denoising dual network model The denoising dual network model includes a noise recognition network model, a preliminary denoising module and a final denoising dual network model. The noise recognition network model takes the shadow area image to be denoised as input and is used to identify mixed noise including dark noise, compression noise and read noise; the preliminary denoising module is used to remove mixed noise from the shadow area image to be denoised, and then correct the DN value of the removed image to obtain the remaining shadow area image, thereby completing the preliminary denoising; The final denoising dual network model uses the remaining shadow area image as input to identify the clean image and complete the final denoising; Step 2: Train the noise recognition model and the final denoising model S21, artificially constructing a simulated shadow area image I to be denoised containing various noises and a mixed noise image Nm including only dark noise, compression noise and read noise to form a first data set, taking the image I to be denoised as input and the mixed noise image Nm as output, and training the noise recognition network model; S22, using the trained noise recognition model to identify the simulated denoising shadow area images I in the first data set one by one, obtain the corresponding mixed noise image Nm, and then use the preliminary denoising module to process it to obtain the corresponding residual shadow area image Nn, and artificially construct a clean image S, that is, an image that does not include any noise, to form a second data set, and use the residual shadow area image Nn as input and the clean image S as output to train the final denoising dual network model; Step 3: Use the trained noise recognition network model and the final denoising dual network model together with the preliminary denoising module to denoise the denoised shadow area image to obtain the final denoised shadow area image.

2. The method for denoising the lunar permanent shadow region based on a physical noise model according to claim 1, characterized in that: The noise recognition network model and the final denoising dual network model have the same structure, both of which are based on the UNet network, and the jump connections in the UNet network are replaced by full-scale jump connections, and the convolution is replaced by the LeWIn Transformer module.

3. The method for denoising the lunar permanent shadow region based on a physical noise model according to claim 2, characterized in that: The simulated shadow area image I to be denoised, the mixed noise image Nm, and the clean image S are all constructed based on a physical noise model constructed based on the LROC NAC imaging principle.

4. The method for denoising the lunar permanent shadow region based on a physical noise model according to claim 3, characterized in that: Using the preliminary denoising module, the mixed noise image Nm is first subtracted from the simulated shadow area image I to be denoised, and then the image DN value is corrected to linearity using the following formula, and then divided by the flat field correction coefficient F to achieve uniformity of the image DN value; Where x represents the DN value of the recorded pixel, and a, b, c, and d are free parameters estimated through experimental calibration and are known parameters.

5. A computer storage medium, characterized in that: It comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the lunar permanent shadow region denoising method based on a physical noise model as described in any one of claims 1 to 4.

6. An electronic device, characterized in that: It includes one or more processors, one or more memories and one or more applications; wherein the one or more applications are stored in the memory, and when the one or more programs are executed by the processor, the electronic device executes the lunar permanent shadow area denoising method based on the physical noise model as described in any one of claims 1 to 4.

Images