Neutron CT image denoising method and model training method, device and equipment

Abstract

Embodiments of the present application provide a neutron CT image denoising method and model training method, device and equipment, by acquiring the corresponding neutron CT image and X-ray CT image of the same lesion area; based on the image denoising model, the neutron CT image and the X-ray CT image are processed to obtain the denoised neutron CT image corresponding to the neutron CT image; wherein the image denoising model comprises an encoder, a feature fusion device for performing feature fusion on the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder for decoding the fused features to obtain the denoised neutron CT image, fully utilizing the complementary characteristics of neutron and X-ray CT images, effectively reducing the noise in the neutron CT image, improving the detail performance and overall clarity of the image, and improving the quality and reliability of the neutron CT image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a neutron CT image denoising method and model training method, apparatus and equipment. Background Technology

[0002] In the field of modern science and technology, computed tomography (CT) technology is widely used in research and diagnostic scenarios, such as neutron computed tomography (NCT). Unlike X-rays, neutrons possess unique penetrating properties; they can effectively penetrate certain metals and heavy elements while exhibiting high sensitivity to light elements (especially hydrogen), making NCT demonstrate great potential and value. However, noise is a recognized challenge in neutron CT image reconstruction. The presence of noise significantly reduces image clarity and accuracy, affecting diagnostic and analytical results.

[0003] Currently, commonly used neutron CT image denoising methods include filtering denoising methods and methods using iterative reconstruction algorithms, such as Algebra Reconstruction Technique (ART) and Statistical Iterative Reconstruction Technique (SIRT).

[0004] However, filtering and denoising methods sacrifice detail and resolution in neutron CT sample images, resulting in smooth transitions that blur image edges and affect image clarity. Iterative reconstruction algorithms, on the other hand, are typically computationally intensive, require long processing times, and may experience slow convergence during iteration. Summary of the Invention

[0005] This invention provides a neutron CT image denoising method, model training method, apparatus, and device to reduce noise in neutron CT images, improve the detail and overall clarity of neutron CT images, and enhance the quality and reliability of neutron CT images.

[0006] In a first aspect, embodiments of the present invention provide a neutron CT image denoising method, the method comprising: Acquire neutron CT images and X-ray CT images corresponding to the same lesion area; Neutron CT images and X-ray CT images are processed based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image.

[0007] Secondly, embodiments of the present invention also provide a training method for an image denoising model, the image denoising model including an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained, the method including: Multiple training samples are acquired; each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion area, as well as a denoised sample image corresponding to the neutron CT sample image. For each training sample, the neutron CT sample image and the X-ray CT sample image in the current training sample are input into the encoder to be trained to obtain the first sample feature vector corresponding to the neutron CT sample image and the second sample feature vector corresponding to the X-ray CT sample image. The first sample feature vector and the second sample feature vector are input into the feature fusion fusion unit to be trained to obtain the target sample feature vector. The target sample feature vector is input into the decoder to be trained to obtain the target sample output image; Based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, the target loss value is determined, and the parameters of each processor in the image denoising model are corrected based on the target loss value. The convergence of the loss function in the image denoising model is used as the training objective to obtain the trained image denoising model.

[0008] Thirdly, embodiments of the present invention also provide a neutron CT image denoising device, the device comprising: The dual-modal image acquisition module is used to acquire neutron CT images and X-ray CT images corresponding to the same lesion area; The denoising image output module is used to process neutron CT images and X-ray CT images based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features.

[0009] Fourthly, embodiments of the present invention also provide a training apparatus for an image denoising model, the image denoising model including an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained, the apparatus comprising: The sample acquisition module is used to acquire multiple training samples; each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion area, as well as a denoised sample image corresponding to the neutron CT sample image. The sample feature extraction module is used to input the neutron CT sample image and X-ray CT sample image in the current training sample into the encoder to be trained, and obtain the first sample feature vector corresponding to the neutron CT sample image and the second sample feature vector corresponding to the X-ray CT sample image. The sample feature fusion module is used to input the first sample feature vector and the second sample feature vector into the feature fusion fusion unit to be trained, so as to obtain the target sample feature vector. The image output module is used to input the feature vector of the target sample into the decoder to be trained, and obtain the output image of the target sample; The model correction module is used to determine the target loss value based on the target sample output image and the denoised sample image, so as to correct the parameters of each processor in the image denoising model based on the target loss value. The denoising model determination module is used to take the convergence of the loss function in the image denoising model as the training objective to obtain the trained image denoising model.

[0010] Fifthly, embodiments of the present invention also provide an electronic device, the electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the neutron CT image denoising method or the image denoising model training method as described in any embodiment of the present invention.

[0011] In a sixth aspect, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a neutron CT image denoising method or an image denoising model training method as described in any of the embodiments of the present invention.

[0012] The technical solution of this invention acquires neutron CT images and X-ray CT images corresponding to the same lesion area, and then processes the neutron CT images and X-ray CT images based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image. This fully utilizes the complementary characteristics of neutron and X-ray CT images, enhances the expressive power of the fused image, effectively reduces random noise and system noise in the neutron CT image, improves the detail and overall clarity of the image, and improves the quality and reliability of the neutron CT image. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0014] Figure 1 This is a flowchart illustrating a neutron CT image denoising method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the image denoising model involved in an embodiment of the present invention; Figure 3 This is a schematic diagram of an image denoising model with a Swin-Transformer network structure according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the feature fusion device structure according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating a training method for an image denoising model provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of the structure of a neutron CT image denoising device provided in an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a training device for an image denoising model provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0016] Example 1 Before introducing this technical solution, an exemplary application scenario can be provided. The neutron CT image denoising method provided in this embodiment can be applied to any scenario that requires denoising processing of neutron CT images.

[0017] In the field of modern science and technology, imaging technology has become an important tool for research and diagnosis. Traditional imaging techniques, such as X-ray computed tomography (X-ray CT), are widely known for their extensive applications in medical imaging and industrial non-destructive testing. X-ray CT technology utilizes X-rays to penetrate a sample, capturing the rays after they have passed through the sample using a detector, and then reconstructing an image of the sample's internal structure. However, X-ray CT technology has limitations in certain applications. X-rays have weak penetrating power for light elements, which limits its application in analyzing samples containing large amounts of light elements (such as biological tissues). Furthermore, X-rays also have limited penetrating power for metals and heavy elements, posing a challenge in materials science and archaeological research.

[0018] To overcome these limitations, scientists developed an imaging technique called neutron computed tomography (NCT). Unlike X-rays, neutrons possess unique penetrating properties; they can effectively penetrate certain metals and heavy elements while exhibiting high sensitivity to light elements, especially hydrogen. This characteristic makes NCT technology demonstrate enormous potential and value in numerous fields.

[0019] Noise is a recognized challenge in image reconstruction during computed tomography (CT). The presence of noise significantly reduces image sharpness and accuracy, affecting diagnostic and analytical results. Although existing technologies have implemented a series of measures to mitigate the impact of noise, some shortcomings remain, summarized below: 1. Limitations of Filtering Techniques: Traditional filtering techniques, such as Gaussian filtering, mean filtering, and median filtering, while reducing random noise in images to some extent, often sacrifice image detail and resolution. Gaussian filtering, in particular, can blur image edges due to its smoothing effect, affecting image clarity.

[0020] 2. Optimization space of reconstruction algorithms: Iterative reconstruction algorithms, including algebraic reconstruction techniques (ART) and statistical iterative reconstruction techniques (SIRT), can optimize image quality and reduce noise through multiple iterations. However, these algorithms are usually computationally intensive, require a long processing time, and may experience slow convergence during the iteration process.

[0021] 3. Challenges of Multimodal Fusion: While multimodal fusion technology can leverage the advantages of different imaging modalities to improve image quality, the accuracy of image registration is crucial for the fusion effect in practice. Differences in physical characteristics between images of different modalities can lead to registration difficulties, affecting the quality of the fused image.

[0022] 4. Shortcomings of raw projection data preprocessing: In the preprocessing stage before image reconstruction, although noise filtering and correction techniques are used, these methods may not be able to completely eliminate noise, especially for images with complex structures or high noise levels, the preprocessing effect may not be satisfactory.

[0023] In summary, although existing denoising techniques have addressed the noise problem in CT image reconstruction to some extent, many challenges remain to be overcome. Therefore, this application aims to provide a neutron CT denoising method under a neutron and X-ray dual-modal system to solve the problem of image quality being severely affected by noise in existing neutron CT techniques.

[0024] Figure 1 This is a flowchart illustrating a neutron CT image denoising method provided in an embodiment of the present invention. This embodiment is applicable to situations where neutron CT images need to be denoised. The method can be executed by a neutron CT image denoising device, which can be implemented in the form of software and / or hardware. The hardware can be electronic devices such as servers. The electronic devices can execute the neutron CT image denoising method provided in this technical solution to display target objects to users.

[0025] like Figure 1 As shown, the method includes: S110. Obtain neutron CT images and X-ray CT images corresponding to the same lesion area.

[0026] Neutron CT images are images of the lesion area acquired using neutron computed tomography (CT) technology. X-ray CT images are images of the lesion area acquired using X-ray computed tomography (CT).

[0027] In this embodiment, a dual-modal imaging system can be used to simultaneously acquire neutron CT images and X-ray CT images. The dual-modal imaging system combines the advantages of neutron CT and X-ray CT, enabling the simultaneous acquisition of neutron and X-ray images of the same sample, thus providing a more comprehensive and in-depth analysis. The key components of the dual-modal imaging system include: Neutron source and X-ray source: Two independent radiation sources, used to generate neutrons and X-rays respectively, to meet different imaging needs; Sample stage: A multifunctional device that can stably place and rotate samples in two imaging modes, ensuring consistency and accuracy in the imaging process; Dual-modal detector: An innovative detector design that can detect neutrons and X-rays simultaneously, or detect neutrons and X-rays separately using two independent detectors, to adapt to different imaging needs.

[0028] For example, a medical image can be captured on the abdominal region of a patient using a dual-modal imaging system to obtain a neutron CT image and an X-ray CT image corresponding to the abdominal region of the patient.

[0029] S120. Based on the image denoising model, process the neutron CT image and X-ray CT image to obtain the denoised neutron CT image corresponding to the neutron CT image.

[0030] The denoised neutron CT image is the image content obtained after denoising a neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image. In this embodiment, a schematic diagram of the image denoising model is shown below. Figure 2 ,like Figure 2 As shown, the image denoising model includes an encoder, a feature fusionist, and a decoder. For example, the encoder and decoder can be a Swin-Transformer network structure.

[0031] In practical applications, neutron CT images and X-ray CT images are input into an image denoising model, which can process the input images to obtain denoised neutron CT images.

[0032] More specifically, based on an image denoising model, neutron CT images and X-ray CT images are processed to obtain denoised neutron CT images corresponding to the neutron CT images. Specific implementation methods include: S1201. Input the neutron CT image and the X-ray CT image into the encoder to obtain the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image.

[0033] In this embodiment, neutron CT images and X-ray CT images are input into the encoder in parallel. The encoder encodes the neutron CT images to obtain a first feature vector corresponding to the neutron CT images; the encoder also encodes the X-ray CT images to obtain a second feature vector corresponding to the X-ray CT images.

[0034] For an example, see the schematic diagram of an image denoising model with a Swin-Transformer network structure. Figure 3 Neutron CT images (i.e. Figure 3 Noisy neutron CT images) and X-ray CT images (i.e. Figure 3 The raw data from both the neutron CT image and the X-ray CT image (which contain noise) are input in parallel into the Swin-Transformer network. The Swin-Transformer network consists of a Transformer encoder and a decoder. The encoder utilizes a multi-head self-attention mechanism to extract features from the input neutron CT image and X-ray CT image, capturing the long-range dependencies between features of different modalities. Specifically, the encoder performs multi-scale feature extraction on the input data through stacked multi-head self-attention and feedforward neural networks, ensuring that it can comprehensively capture the details and overall information of each modal data.

[0035] S1202. Input the first feature vector and the second feature vector into the feature fusion unit to obtain the target fused feature vector.

[0036] The target fusion feature vector is the feature vector obtained by fusing the first feature vector and the second feature vector.

[0037] In this embodiment, the first and second feature vectors can be concatenated to obtain the target fused feature vector; this feature fusion device is called a feature concatenation fusion device. Alternatively, bilinear pooling can be used to fuse the first and second feature vectors; this feature fusion device is called a bilinear pooling feature fusion device. The bilinear fusion method generates the fused feature representation by calculating bilinear pooling of the two modal features. Bilinear pooling is an effective multimodal feature fusion method that can capture the complex interactions between different modal features. This fusion method can fully utilize the complementarity of neutron and X-ray data, enhancing the expressive power of the features and making the fused features richer and more accurate.

[0038] Specifically, for the bilinear pooling fusion method, the feature fusion machine structure diagram is as follows: Figure 4As shown, the feature fusion unit may include: a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fused feature output unit. The specific implementation of inputting the first feature vector and the second feature vector into the feature fusion unit to obtain the target fused feature vector may include: S1. Based on the feature element standardization unit, the first feature vector and the second feature vector are standardized to obtain the first standard feature vector and the second standard feature vector.

[0039] Specifically, the first and second feature vectors can be represented as: the first feature vector X ∈ R^(b×d1); the second feature vector Y ∈ R^(b×d2), where b is the batch size, and d1 and d2 are their respective feature dimensions. Each feature element in the first and second feature vectors is standardized using a feature element standardization unit, i.e., L2 normalization is performed using the following formula: in, For any feature element in the first feature vector, is any feature element in the second feature vector.

[0040] S2. Based on the low-rank bilinear mapping unit, the first standard feature vector and the second standard feature vector are processed by low-rank bilinear mapping to obtain the fused feature vector to be processed.

[0041] Specifically, the specific implementation of performing low-rank bilinear mapping on the first and second standard feature vectors based on the low-rank bilinear mapping unit to obtain the fused feature vector to be processed can include: multiplying the first and second standard feature vectors element by element to obtain the fused feature vector to be transformed; and performing regularization and linear transformation on the fused feature vector to be transformed to obtain the fused feature vector to be processed.

[0042] In this embodiment, the specific processing steps in the low-rank bilinear mapping unit include: First, the low-rank bilinear mapping is implemented through matrix multiplication and element-wise multiplication: in, and The coefficient matrix, , , ⊙ represents the element-wise multiplication operator.

[0043] Subsequently, dropout and linear transformation are performed on Z using the following formula: in, The coefficient matrix, , This is the feature vector to be fused.

[0044] S3. Input the fusion feature vector to be processed into the self-attention unit to obtain the attention weight vector corresponding to the fusion feature vector to be processed.

[0045] In this embodiment, the specific processing steps in the self-attention unit include: After obtaining the fusion feature vector to be processed Based on this, firstly, the following formula is used to... Processing using a self-attention mechanism: in, This is the weight matrix for attention.

[0046] Furthermore, the attention weights are calculated as follows: Therefore, the attention weight vector corresponding to the fused feature vector to be processed can be expressed as: S4. Based on the fusion feature output unit, the attention weight vector is subjected to nonlinear activation processing and Euclidean normalization processing to obtain the target feature vector.

[0047] In this embodiment, the specific processing procedure of the fusion feature output unit includes: The attention weight vector is non-linearly activated and L2 normalized using the following formula: in, For the final target feature vector, ϵ is a small constant used to prevent division by zero.

[0048] Based on the above examples, such as Figure 3 As shown, this feature fusion tool ( Figure 3 The bilinear fusion module (shown) is embedded into the Swin-Transformer network to construct an end-to-end network structure, and then the weight matrices U, V, and W of the feature fusion unit are optimized through backpropagation. This bilinear pooling method can effectively capture the higher-order correlations between neutron and X-ray data, thereby generating richer and more discriminative fused features. Its advantage lies in its ability to learn fine-grained interactions between modes, rather than simply concatenating features.

[0049] It's important to note that bilinear pooling has high computational complexity, and in practical applications, computational efficiency and memory consumption may need to be considered. To address this, techniques such as the low-rank approximation or random projection mentioned above can be used for optimization.

[0050] S1203. Input the target fusion feature vector into the decoder to obtain the denoised neutron CT image corresponding to the neutron CT image.

[0051] In this embodiment, the fused target feature vector is input into the decoder to obtain the denoised neutron CT image corresponding to the neutron CT image.

[0052] Based on the above examples, such as Figure 3 As shown, the fused target feature vector is input into the Transformer decoder of the Swin-Transformer, and the resulting denoised CT image is the denoised neutron CT image corresponding to the neutron CT image. The decoder processes the fused features through a stacked multi-head self-attention and feedforward neural network to generate the reconstructed neutron image. In this process, the decoder module not only reconstructs the original neutron data but also applies noise reduction processing, effectively reducing the impact of noise and improving the quality of the neutron CT image. The decoder module generates a high-quality neutron CT image by decoding the fused features layer by layer, while preserving the complementary information of neutron and X-ray data.

[0053] The technical solution of this invention acquires neutron CT images and X-ray CT images corresponding to the same lesion area, and then processes the neutron CT images and X-ray CT images based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image. This fully utilizes the complementary characteristics of neutron and X-ray CT images, enhances the expressive power of the fused image, effectively reduces random noise and system noise in the neutron CT image, improves the detail and overall clarity of the image, and improves the quality and reliability of the neutron CT image.

[0054] Example 2 Figure 5This is a flowchart illustrating a training method for an image denoising model provided in an embodiment of the present invention. This embodiment is applicable to training an image denoising model for removing noise from neutron CT images. The method can be executed by an image denoising model training device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device such as a server. The electronic device can execute the image denoising model training method provided in this technical solution to obtain the image denoising model.

[0055] like Figure 5 As shown, the method includes: S210. Obtain multiple training samples.

[0056] Each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion region, as well as a denoised sample image corresponding to the neutron CT sample image. The denoised sample image is a high-quality image obtained by pre-processing the neutron CT sample image with conventional denoising techniques.

[0057] S220. For each training sample, input the neutron CT sample image and X-ray CT sample image in the current training sample into the encoder to be trained to obtain the first sample feature vector corresponding to the neutron CT sample image and the second sample feature vector corresponding to the X-ray CT sample image.

[0058] The image denoising model includes an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained.

[0059] In this embodiment, the training process is the same for any one of the multiple training samples. We will use any one of them as the current training sample as an example. Specifically, the neutron CT sample image and the X-ray CT sample image in the current training sample are input in parallel into the encoder to be trained. The encoder to be trained can output a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image.

[0060] S230. Input the first sample feature vector and the second sample feature vector into the feature fusion fusion unit to be trained to obtain the target sample feature vector.

[0061] In this embodiment, if the feature fusion processor to be trained is a feature splicing fusion processor, then the feature vectors of the first sample and the second sample are spliced ​​together based on the feature splicing fusion processor to obtain the target sample feature vector.

[0062] Optionally, if the feature fusion facilitator to be trained is a bilinear pooling feature fusion facilitator, the bilinear pooling feature fusion facilitator may include: a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fusion feature output unit. Then, the specific implementation method for determining the target sample feature vector may include: The first and second sample feature vectors are standardized by the feature element standardization unit to obtain the first and second standard sample feature vectors respectively. The first and second standard sample feature vectors are then subjected to low-rank bilinear mapping by the low-rank bilinear mapping unit to obtain the fused sample feature vector to be processed. The fused sample feature vector to be processed is then input into the self-attention unit to obtain the attention sample weight vector corresponding to the fused feature vector to be processed. The attention sample weight vector is then subjected to nonlinear activation and Euclidean normalization by the fused feature output unit to obtain the target sample feature vector.

[0063] S240. Input the feature vector of the target sample into the decoder to be trained to obtain the output image of the target sample.

[0064] In this embodiment, the feature vector of the target sample is input into the decoder to be trained, and the decoder to be trained outputs the target sample output image.

[0065] S250. Based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, determine the target loss value, and then perform parameter correction on each processor in the image denoising model based on the target loss value.

[0066] In this embodiment, the specific implementation method for determining the target loss value based on each first pixel value in the target sample output image and each second pixel value at the corresponding pixel position in the denoised sample image may include: Based on each first pixel value and the corresponding second pixel value, the mean squared error loss value and the structural similarity loss value are determined; based on the preset weight value, the mean squared error loss value, and the structural similarity loss value, the target loss value is determined.

[0067] In this embodiment, in order to optimize the model parameters, a loss function can be used to measure the difference between the neutron-denoised image output during the training of the image denoising model (i.e., the target sample output image) and the given high-quality neutron CT image (i.e., the denoised sample image).

[0068] In this embodiment, the target loss function is a composite loss function consisting of the Mean Squared Error (MSE) loss function and the SSIM (Structural Similarity Index) loss function. The MSE loss function measures the squared difference between the predicted value and the true value, and SSIM is the structural similarity index, ranging from [0,1], with a larger value indicating greater similarity between the two images.

[0069] The formula for the target loss function is expressed as follows: In the formula, It is the value of the first pixel of the i-th element in the output image of the target sample. It is the value of the second pixel of the i-th pixel in the denoised sample image; The structural similarity index between the target sample output image and the denoised sample image is calculated. and The preset weight values ​​can be adjusted based on actual results in practical applications. and The value is adjusted to balance the effects of MSE and SSIM.

[0070] The first pixel value and the corresponding second pixel value are substituted into the target loss function to obtain the target loss value. Then, the model parameters in the encoder, feature fusion unit, and decoder of the image denoising model are corrected based on the loss value.

[0071] S260. Take the convergence of the loss function in the image denoising model as the training objective to obtain the trained image denoising model.

[0072] In this embodiment, the image denoising model is trained based on each training sample, and the model parameters in the encoder, feature fusion unit and decoder of the image denoising model are corrected according to the loss value until the target loss function converges, thus obtaining the trained image denoising model.

[0073] The technical solution of this embodiment includes an image denoising model comprising an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained. The training method includes: acquiring multiple training samples, wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion region, and a denoised sample image corresponding to the neutron CT sample image; for each training sample, inputting the neutron CT sample image and the X-ray CT sample image from the current training sample into the encoder to be trained to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image; subsequently, the first sample feature vector and the second sample feature vector... The target sample feature vector is obtained by inputting it into the feature fusion unit to be trained. Then, the target sample feature vector is input into the decoder to be trained to obtain the target sample output image. Further, based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, the target loss value is determined. The parameters of each processor in the image denoising model are corrected based on the target loss value. Finally, the convergence of the loss function in the image denoising model is taken as the training objective to obtain the trained image denoising model. The image denoising model is trained by continuously reducing the difference between the neutron denoised image output by the image denoising model and the given high-quality denoised neutron CT image. In this training process, the decoder not only reconstructs the original neutron data, but also applies noise reduction processing to effectively reduce the impact of noise. The final image denoising model can improve the detail and overall clarity of the image, and improve the quality and reliability of the neutron CT image.

[0074] Example 3 Figure 6 This is a schematic diagram of a neutron CT image denoising device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes a dual-modal image acquisition module 310 and a denoised image output module 320.

[0075] Among them, the dual-modal image acquisition module 310 is used to acquire neutron CT images and X-ray CT images corresponding to the same lesion area; The denoised image output module 320 is used to process neutron CT images and X-ray CT images based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features.

[0076] Based on the above embodiments, the denoised image output module 320 includes: The feature vector determination module is used to input neutron CT images and X-ray CT images into the encoder to obtain a first feature vector corresponding to the neutron CT image and a second feature vector corresponding to the X-ray CT image; The feature fusion module is used to input the first feature vector and the second feature vector into the feature fusion unit to obtain the target fused feature vector; The denoised image output module is used to input the target fusion feature vector into the decoder to obtain a denoised neutron CT image corresponding to the neutron CT image.

[0077] Based on the above embodiments, the feature fusion unit includes: a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fused feature output unit; the feature fusion module includes: The feature vector standardization unit is used to standardize the first feature vector and the second feature vector based on the feature element standardization unit, respectively, to obtain the first standard feature vector and the second standard feature vector. The feature bilinear mapping unit is used to perform low-rank bilinear mapping processing on the first standard feature vector and the second standard feature vector based on the low-rank bilinear mapping unit to obtain the fused feature vector to be processed. The attention weight determination unit is used to input the fusion feature vector to be processed into the self-attention unit to obtain the attention weight vector corresponding to the fusion feature vector to be processed. The target feature vector determination unit is used to perform nonlinear activation and Euclidean normalization on the attention weight vector based on the fusion feature output unit to obtain the target feature vector.

[0078] Based on the above embodiments, the feature bilinear mapping unit is specifically used to perform element-wise multiplication of the first standard feature vector and the second standard feature vector to obtain the fused feature vector to be transformed; and to perform regularization and linear transformation on the fused feature vector to be transformed to obtain the fused feature vector to be processed.

[0079] The technical solution of this invention acquires neutron CT images and X-ray CT images corresponding to the same lesion area, and then processes the neutron CT images and X-ray CT images based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image. The image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image. This fully utilizes the complementary characteristics of neutron and X-ray CT images, enhances the expressive power of the fused image, effectively reduces random noise and system noise in the neutron CT image, improves the detail and overall clarity of the image, and improves the quality and reliability of the neutron CT image.

[0080] The neutron CT image denoising device provided in the embodiments of the present invention can execute the neutron CT image denoising method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0081] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0082] Example 4 Figure 7 This is a schematic diagram of the structure of a training device for an image denoising model provided in an embodiment of the present invention. The image denoising model includes an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained, such as... Figure 7 As shown, the device includes: a sample acquisition module 410, a sample feature extraction module 420, a sample feature fusion module 430, an image output module 440, a model correction module 450, and a denoising model determination module 460.

[0083] The sample acquisition module 410 is used to acquire multiple training samples; wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion area, as well as a denoised sample image corresponding to the neutron CT sample image; The sample feature extraction module 420 is used to input the neutron CT sample image and the X-ray CT sample image in the current training sample into the encoder to be trained for each training sample, so as to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image. The sample feature fusion module 430 is used to input the first sample feature vector and the second sample feature vector into the feature fusion fusion unit to be trained to obtain the target sample feature vector. Image output module 440 is used to input the feature vector of the target sample into the decoder to be trained to obtain the output image of the target sample; The model correction module 450 is used to determine the target loss value based on the target sample output image and the denoised sample image, so as to correct the parameters of each processor in the image denoising model based on the target loss value. The denoising model determination module 460 is used to take the convergence of the loss function in the image denoising model as the training target to obtain the trained image denoising model.

[0084] Based on the above embodiments, the model correction module 450 further includes: a loss value determination unit; The loss value determination unit is used to determine the mean squared error loss value and the structural similarity loss value based on each of the first pixel values ​​and the corresponding second pixel values; and to determine the target loss value based on the preset weight value, the mean squared error loss value and the structural similarity loss value.

[0085] The technical solution of this embodiment includes an image denoising model comprising an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained. The training method includes: acquiring multiple training samples, wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion region, and a denoised sample image corresponding to the neutron CT sample image; for each training sample, inputting the neutron CT sample image and the X-ray CT sample image from the current training sample into the encoder to be trained to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image; subsequently, the first sample feature vector and the second sample feature vector... The target sample feature vector is obtained by inputting it into the feature fusion unit to be trained. Then, the target sample feature vector is input into the decoder to be trained to obtain the target sample output image. Further, based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, the target loss value is determined. The parameters of each processor in the image denoising model are corrected based on the target loss value. Finally, the convergence of the loss function in the image denoising model is taken as the training objective to obtain the trained image denoising model. The image denoising model is trained by continuously reducing the difference between the neutron denoised image output by the image denoising model and the given high-quality denoised neutron CT image. In this training process, the decoder not only reconstructs the original neutron data, but also applies noise reduction processing to effectively reduce the impact of noise. The final image denoising model can improve the detail and overall clarity of the image, and improve the quality and reliability of the neutron CT image.

[0086] Example 5 Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 8 A block diagram is shown of an exemplary electronic device 50 suitable for implementing embodiments of the present invention. Figure 8 The electronic device 50 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0087] like Figure 8 As shown, the electronic device 50 is represented in the form of a general-purpose computing device. The components of the electronic device 50 may include, but are not limited to: one or more processors or processing units 501, system memory 502, and bus 503 connecting different system components (including system memory 502 and processing unit 501).

[0088] Bus 503 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0089] Electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0090] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 503 via one or more data media interfaces. Memory 502 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0091] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in memory 502. Such program modules 507 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 507 typically perform the functions and / or methods described in the embodiments of the present invention.

[0092] Electronic device 50 can also communicate with one or more external devices 509 (e.g., keyboard, pointing device, display 510, etc.), and with one or more devices that enable a user to interact with electronic device 50, and / or with any device that enables electronic device 50 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 511. Furthermore, electronic device 50 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 512. As shown, network adapter 512 communicates with other modules of electronic device 50 via bus 503. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0093] The processing unit 501 executes various functional applications and page processing by running programs stored in the system memory 502, such as implementing the neutron CT image denoising method or the training method of the image denoising model provided in the embodiments of the present invention.

[0094] Example 6 This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a neutron CT image denoising method, the method comprising: Acquire neutron CT images and X-ray CT images corresponding to the same lesion area; The neutron CT image and the X-ray CT image are processed based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image; wherein, the image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image.

[0095] Alternatively, the computer-executable instructions, when executed by a computer processor, are used to perform a training method for an image denoising model, the image denoising model including an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained, the method including: Multiple training samples are acquired; wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion region, as well as a denoised sample image corresponding to the neutron CT sample image; For each training sample, the neutron CT sample image and the X-ray CT sample image in the current training sample are input into the encoder to be trained to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image; The first sample feature vector and the second sample feature vector are input into the feature fusion fusion unit to be trained to obtain the target sample feature vector; The target sample feature vector is input into the decoder to be trained to obtain the target sample output image; Based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, a target loss value is determined, and the parameters of each processor in the image denoising model are corrected based on the target loss value. The convergence of the loss function in the image denoising model is used as the training objective to obtain the trained image denoising model.

[0096] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0097] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0098] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0099] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A neutron CT image denoising method, characterized in that, include: Acquire neutron CT images and X-ray CT images corresponding to the same lesion area; The neutron CT image and the X-ray CT image are processed based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image; wherein, the image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features to obtain the denoised neutron CT image; The feature fusion unit includes: a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fusion feature output unit. The process of processing the neutron CT image and the X-ray CT image based on the image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image includes: The neutron CT image and the X-ray CT image are input into the encoder to obtain a first feature vector corresponding to the neutron CT image and a second feature vector corresponding to the X-ray CT image; Based on the feature element standardization unit, the first feature vector and the second feature vector are standardized respectively to obtain the first standard feature vector and the second standard feature vector; Based on the low-rank bilinear mapping unit, the first standard feature vector and the second standard feature vector are subjected to low-rank bilinear mapping to obtain the fused feature vector to be processed. The fusion feature vector to be processed is input into the self-attention unit to obtain the attention weight vector corresponding to the fusion feature vector to be processed. Based on the fusion feature output unit, the attention weight vector is subjected to nonlinear activation processing and Euclidean normalization processing to obtain the target feature vector. The target fusion feature vector is input into the decoder to obtain a denoised neutron CT image corresponding to the neutron CT image.

2. The method according to claim 1, characterized in that, The step of performing low-rank bilinear mapping processing on the first standard feature vector and the second standard feature vector based on the low-rank bilinear mapping unit to obtain the fused feature vector to be processed includes: The first standard feature vector and the second standard feature vector are multiplied element by element to obtain the fused feature vector to be transformed; The fusion feature vector to be transformed is subjected to regularization and linear transformation to obtain the fusion feature vector to be processed.

3. A training method for an image denoising model, characterized in that, The image denoising model includes an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained. The method includes: Multiple training samples are acquired; wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion region, as well as a denoised sample image corresponding to the neutron CT sample image; For each training sample, the neutron CT sample image and the X-ray CT sample image in the current training sample are input into the encoder to be trained to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image; The first sample feature vector and the second sample feature vector are input into the feature fusion fusion unit to be trained to obtain the target sample feature vector; The target sample feature vector is input into the decoder to be trained to obtain the target sample output image; Based on the first pixel value in the target sample output image and the second pixel value at the corresponding pixel position in the denoised sample image, a target loss value is determined, and the parameters of each processor in the image denoising model are corrected based on the target loss value. The convergence of the loss function in the image denoising model is used as the training objective to obtain the trained image denoising model; The feature fusion unit to be trained includes: a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fused feature output unit; the step of inputting the first sample feature vector and the second sample feature vector into the feature fusion unit to be trained to obtain the target sample feature vector includes: Based on the feature element standardization unit, the first sample feature vector and the second sample feature vector are standardized respectively to obtain the first standard sample feature vector and the second standard sample feature vector. Based on the low-rank bilinear mapping unit, the first standard sample feature vector and the second standard sample feature vector are subjected to low-rank bilinear mapping to obtain the fused sample feature vector to be processed. The feature vector of the fusion sample to be processed is input into the self-attention unit to obtain the attention sample weight vector corresponding to the fusion feature vector to be processed. Based on the fusion feature output unit, the attention sample weight vector is subjected to nonlinear activation processing and Euclidean normalization processing to obtain the target sample feature vector.

4. The method according to claim 3, characterized in that, The step of determining the target loss value based on each first pixel value in the target sample output image and each second pixel value at the corresponding pixel position in the denoised sample image includes: Based on each of the first pixel values ​​and the corresponding second pixel values, the mean squared error loss value and the structural similarity loss value are determined; The target loss value is determined based on the preset weight value, the mean squared error loss value, and the structural similarity loss value.

5. A neutron CT image denoising device, characterized in that, include: The dual-modal image acquisition module is used to acquire neutron CT images and X-ray CT images corresponding to the same lesion area; A denoised image output module is used to process the neutron CT image and the X-ray CT image based on an image denoising model to obtain a denoised neutron CT image corresponding to the neutron CT image; wherein, the image denoising model includes an encoder, a feature fusion unit that fuses the first feature vector corresponding to the neutron CT image and the second feature vector corresponding to the X-ray CT image output by the encoder, and a decoder that decodes the fused features; The feature fusion unit includes a feature element normalization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fusion feature output unit. The denoised image output module is specifically used for: inputting the neutron CT image and the X-ray CT image into the encoder to obtain a first feature vector corresponding to the neutron CT image and a second feature vector corresponding to the X-ray CT image; normalizing the first feature vector and the second feature vector based on the feature element normalization unit to obtain a first standard feature vector and a second standard feature vector; performing low-rank bilinear mapping on the first standard feature vector and the second standard feature vector based on the low-rank bilinear mapping unit to obtain a fusion feature vector to be processed; inputting the fusion feature vector to be processed into the self-attention unit to obtain an attention weight vector corresponding to the fusion feature vector; performing nonlinear activation and Euclidean normalization on the attention weight vector based on the fusion feature output unit to obtain a target feature vector; and inputting the target fusion feature vector into the decoder to obtain a denoised neutron CT image corresponding to the neutron CT image.

6. A training device for an image denoising model, characterized in that, The image denoising model includes an encoder to be trained, a feature fusion unit to be trained, and a decoder to be trained. The device includes: The sample acquisition module is used to acquire multiple training samples; wherein each training sample includes a neutron CT sample image and an X-ray CT sample image corresponding to the same lesion area, as well as a denoised sample image corresponding to the neutron CT sample image; The sample feature extraction module is used to input the neutron CT sample image and the X-ray CT sample image in the current training sample into the encoder to be trained for each training sample, so as to obtain a first sample feature vector corresponding to the neutron CT sample image and a second sample feature vector corresponding to the X-ray CT sample image; The sample feature fusion module is used to input the first sample feature vector and the second sample feature vector into the feature fusion fusion unit to be trained, so as to obtain the target sample feature vector. The image output module is used to input the feature vector of the target sample into the decoder to be trained to obtain the output image of the target sample; The model correction module is used to determine the target loss value based on the target sample output image and the denoised sample image, so as to correct the parameters of each processor in the image denoising model based on the target loss value. The denoising model determination module is used to take the convergence of the loss function in the image denoising model as the training target to obtain the trained image denoising model. The feature fusion unit to be trained includes: a feature element standardization unit, a low-rank bilinear mapping unit, a self-attention unit, and a fusion feature output unit. Specifically, the sample feature fusion module is used to standardize the first sample feature vector and the second sample feature vector based on the feature element standardization unit to obtain a first standard sample feature vector and a second standard sample feature vector; to perform low-rank bilinear mapping on the first standard sample feature vector and the second standard sample feature vector based on the low-rank bilinear mapping unit to obtain a fusion sample feature vector to be processed; to input the fusion sample feature vector to be processed into the self-attention unit to obtain an attention sample weight vector corresponding to the fusion feature vector to be processed; and to perform nonlinear activation and Euclidean normalization on the attention sample weight vector based on the fusion feature output unit to obtain a target sample feature vector.

7. An electronic device, characterized in that, Electronic devices include: One or more processors; Storage device for storing one or more programs. When one or more programs are executed by one or more processors, the one or more processors implement the neutron CT image denoising method as described in any one of claims 1-2 or the training method of the image denoising model as described in any one of claims 3-4.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the neutron CT image denoising method as described in any one of claims 1-2 or the training method of the image denoising model as described in any one of claims 3-4.

Images