Gamma photon industrial reconstructed image quality enhancement method based on parallel deformable convolutional network

Abstract

The invention discloses a gamma photon industrial reconstruction image quality enhancement method based on a parallel deformable convolutional network, and the method comprises the following steps: inputting a gamma photon industrial detection image containing complex noise into a constructed denoising model which is composed of a convolution module and an adaptive feature fusion module; wherein the convolution modules are respectively an initial convolution block, an expansion convolution block and a deformable convolution block. A parallel network is established by an initial convolution block, an expansion convolution block and a deformable convolution block, global feature information of a gamma photon industrial image is fully captured by utilizing different receptive field sizes of the expansion convolution block and the deformable convolution block, and local feature information is captured by the initial convolution block. And local feature information and global feature information are combined through an adaptive feature fusion module, and the difference between noise and texture details in the gamma photon industrial image is effectively processed. And a high-quality reconstructed image is output, and the capability of gamma photon industrial detection imaging is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology and gamma photon industrial image reconstruction, and in particular to a gamma photon industrial reconstruction image quality enhancement method based on a parallel deformable convolutional network. Background Art

[0002] With the rapid development of computer technology and image processing technology, positron emission tomography (PET) has become an important non-destructive testing technology in the industrial field. PET technology captures high-energy gamma photons generated by positron annihilation to perform internal imaging of industrial objects and is widely used in the aviation industry, metal material testing and other fields. However, in this application scenario, the quality of the reconstructed image is easily affected by complex noise.

[0003] Although traditional image filters such as mean filtering and median filtering can remove noise to a certain extent, they do not perform well when processing complex boundary information. Mean filtering is suitable for removing additive Gaussian noise and reduces noise by averaging the grayscale values ​​of neighboring pixels. However, it blurs edge and detail information, causing signal distortion. Median filtering is good at removing salt and pepper noise, replacing pixel values ​​with the median of pixels in the neighborhood to eliminate extreme noise. However, it may still cause signal distortion when processing periodic signals and boundary information. The noise in gamma photon industrial images is caused by the scattering caused by gamma photons penetrating metal materials and the physical decay effect. Traditional denoising technology cannot effectively cope with these challenges and it is difficult to meet the needs of high-precision nondestructive testing.

[0004] Convolutional neural networks (CNNs) have been applied to image denoising tasks in recent years due to their powerful learning ability and flexibility. Through convolution operations, CNNs can extract complex noise features in images. However, in industrial large axial space PET image reconstruction, existing CNN models still have multiple problems: (1) In complex industrial noise environments, existing CNN models are difficult to effectively handle high-intensity noise, resulting in unsatisfactory denoising effects. (2) Deep networks are prone to gradient vanishing, which affects training results. (3) Although simply expanding the receptive field can capture more feature information, it increases the number of model parameters, resulting in slower training. Therefore, how to design a denoising network suitable for gamma photon industrial reconstruction images has become an urgent problem to be solved in current research. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for enhancing the quality of gamma photon industrial reconstruction images based on a deformable convolutional network.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for enhancing the quality of gamma photon industrial reconstruction images based on a parallel deformable convolutional network, comprising the following steps:

[0007] Step 1: Take the original gamma photon industrial reconstructed image obtained by the ordered subset maximum expectation method (OSEM) as input;

[0008] Step 2: Learning the global and local features of the industrial reconstructed image through the convolution module;

[0009] 1) The convolution module is mainly composed of three parallel network branches. The first layers of the three network branches are composed of the initial convolution kernel, the dilated convolution kernel, the deformable convolution kernel and the activation function ReLU to generate the initial global and local feature information maps;

[0010] 2) The second to fifth layers are composed of initial convolution kernel, dilated convolution kernel, deformable convolution kernel, activation function, and batch normalization, respectively, to improve the nonlinear expression ability of network branches and obtain more sophisticated and complex noise and texture detail feature maps;

[0011] 3) Each network branch finally outputs a feature information map through 32 convolution kernels of size 3×3;

[0012] Step 3: Input the global feature and local feature information into the adaptive feature fusion module. The adaptive feature fusion module contains a global feature fusion block and a local feature fusion block. It fuses the differences in noise and texture details in the reconstructed images of different regions to improve the model's ability to restore the details of the reconstructed image. The details are as follows:

[0013] 1) The local feature information F1 in the global and local feature information output by the convolution module is fused through the attention of the global feature fusion module to obtain the fused global feature information graph F′2. The operation is as follows:

[0014] After linear mapping, the global feature information F2 is multiplied by the local feature information F1; then the corresponding weight ω is obtained through activation function and normalization F , the corresponding weight ω F After weighted summation with the local feature information F1, a new feature information map F′1 is obtained. Then the new feature information map is linearly mapped and added to the global feature information map F2 to obtain the fused global feature information map F′2.

[0015] 2) The local feature information F2 in the global and local feature information output by the convolution module is fused through the attention of the local feature fusion module to obtain the fused local feature information graph F′1. The operation is as follows:

[0016] After linear mapping, the local feature information F1 is multiplied by the global feature information F2; then the corresponding weight ω is obtained through activation function and normalization F ′, the corresponding weight ω F′ and the global feature information F2 are weighted and summed to obtain a new feature information map F′2, and then the new feature information map is linearly mapped and added to the local feature information map F1 to obtain the fused local feature information map F′1;

[0017] Step 4: The output of the adaptive feature fusion module is then subjected to a 1×1 vector convolution operation Conv to obtain a high-quality gamma photon industrial reconstructed image;

[0018] As a preferred solution, the initial convolution kernel in the three parallel network branches is the first network branch to learn the local feature information of the input image; the deformable convolution kernel is the second network branch, and the deformable convolution kernel has directionality to learn the global feature information of the input image; the dilated convolution kernel is the third network branch, so that the convolution kernel samples on different receptive fields, thereby capturing multi-scale feature information.

[0019] As a preferred solution, the deformable convolution kernel is obtained by performing Fourier interpolation on the convolution kernel. The convolution kernel is first subjected to Fourier transform. Suppose the convolution kernel is K(m,n), and Fourier transform is performed:

[0020]

[0021] Where x and y are the sizes of the convolution kernel, and F(K)(μ, v) represents the convolution kernel in the frequency domain;

[0022] The convolution kernel is interpolated in the frequency domain. The interpolation process is as follows:

[0023]

[0024] Among them, δ is the deformation strength;

[0025] Finally, the inverse Fourier transform is performed to obtain the deformable convolution kernel K′(m, n). The inverse transform is shown as follows:

[0026]

[0027] As a preferred solution, the expansion rate of the dilated convolution kernel is 2 or 7.

[0028] The beneficial effects of the present invention are as follows: the method significantly improves the denoising performance by introducing a parallel three-branch network, and at the same time, while keeping the training object unchanged, indirectly reduces the depth of the network, effectively avoids the gradient vanishing or explosion problem, reduces the training cost, and enhances the generalization ability of the network; without increasing the amount of calculation, the receptive field is increased, information loss is avoided, and the deep mining ability of multi-scale information is enhanced; the complex geometric shapes in the industrial parts image are more flexibly fitted, and the generalization ability of the network is improved. By extracting features through deformable convolution, the contours and edges of the denoised image can better maintain the real effect.

[0029] Traditional convolution is difficult to effectively fit complex geometric shapes due to the fixed sampling method. The present invention performs Fourier interpolation on ordinary convolution kernels, allowing the convolution kernel to adaptively sample around the current position, thereby overcoming the limitations of regular sampling, enhancing the flexibility and accuracy of feature extraction, and significantly improving the network's adaptability to input features.

[0030] The adaptive feature fusion module effectively combines global and local feature information, which can better handle noise and texture differences in different areas and significantly improve the model's ability to restore spatial detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the method for enhancing the quality of industrial reconstruction images using gamma photons.

[0032] Figure 2 It is a schematic diagram of the parallel deformable convolutional network structure.

[0033] Figure 3(a) is a schematic diagram of the global feature fusion block structure.

[0034] Figure 3(b) is a schematic diagram of the local feature fusion block structure.

[0035] Figure 4 The figure is a schematic diagram of the denoising effect of an original industrial annular pipeline detection image in a specific embodiment. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, a method for enhancing the quality of gamma photon industrial reconstruction images based on a deformable convolutional network comprises the following steps:

[0038] Step 1: Take the original gamma photon industrial reconstructed image obtained by OSEM algorithm as input;

[0039] Step 2: The global and local features of the industrial reconstructed image are learned through the convolution module. Figure 2 As shown:

[0040] 1) The convolution module consists of three parallel network branches. The first layer of the three network branches consists of 16 initial convolution kernels of size 3×3, 16 dilated convolution kernels of size 2×2 with a dilation rate of 2, and 16 deformable convolution kernels of size 3×3 and activation function ReLU to generate the initial global and local feature information maps.

[0041] 2) The second to sixth layers are composed of 16 initial convolution kernels of size 3×3, 16 dilated convolution kernels of size 2×2, the expansion rate is set to 7, 16 deformable convolution kernels of size 3×3 and activation function, batch normalization, which improves the nonlinear expression ability of the network branches and obtains more sophisticated and complex noise and texture detail feature maps.

[0042] The deformable convolution kernel in the parallel network branch is interpolated by Fourier. The interpolation steps are as follows. First, the convolution kernel is Fourier transformed. Let the convolution kernel be K(m,n), and Fourier transform is performed:

[0043]

[0044] Among them, x and y are the sizes of the convolution kernel, and f(K)(μ, v) represents the convolution kernel in the frequency domain.

[0045] The convolution kernel is interpolated in the frequency domain. The interpolation process is as follows:

[0046]

[0047] Where δ is the deformation strength

[0048] Finally, the inverse Fourier transform is performed to obtain the deformable convolution kernel K′(m, n). The inverse transform is shown as follows:

[0049]

[0050] 3) Each network branch finally outputs a feature information map through 32 convolution kernels of size 3×3.

[0051] Step 3: Input the global and local feature information into the adaptive feature fusion module, which includes a global feature fusion block and a local feature fusion block. The differences in noise and texture details in the reconstructed images of different regions are fused to improve the model's ability to restore the details of the reconstructed image.

[0052] 1) As shown in Figure 3(a), the local feature information F1 in the global and local feature information output by the convolution module is fused through the attention of the global feature fusion module to obtain the fused global feature information graph F′2;

[0053] 2) As shown in Figure 3(b), the local feature information F2 in the global and local feature information output by the convolution module is fused through the attention of the local feature fusion module to obtain the fused local feature information graph F′1.

[0054] In the global feature fusion block, the local feature information F1 input into the adaptive feature fusion module is integrated into the global feature information F2. The integration process is as follows:

[0055] After linear mapping, the global feature information F2 is multiplied by the local feature information F1. Then the corresponding weight ω is obtained through activation function and normalization. F , the corresponding weight ω F After weighted summation with the local feature information F1, a new feature information map F′1 is obtained. Then the new feature information map is linearly mapped and added to the global feature information map F2 to obtain the fused global feature information map F′2.

[0056] In the local feature fusion block, the global feature information F2 input into the adaptive feature fusion module is integrated into the local feature information F1. The integration process is as follows:

[0057] After linear mapping of the local feature information F1, it is multiplied by the global feature information F2. Then the corresponding weight ω is obtained through activation function and normalization. F ′, the corresponding weight ω F ′ and the global feature information F2 are weightedly summed to obtain a new feature information graph F′2, and then the new feature information graph is linearly mapped and added to the local feature information graph F1 to obtain the fused local feature information graph F′1.

[0058] Step 4: The output of the adaptive feature fusion module is then processed through a 1×1 Conv to obtain a high-quality γ-photon industrial reconstructed image.

[0059] In a specific implementation example of annular pipeline detection imaging, Figure 4 As shown, Figure (a) is the original slice of the annular pipeline reconstructed image, Figure (b) is the denoised slice of the classic DnCNN method, and Figure (c) is the denoised slice of the method of the present invention. It can be seen that the method of the present invention can better enhance the detail information of the original reconstructed image while removing noise.

[0060] The above-mentioned embodiments are only illustrative of the principles and effects of the invention, as well as some embodiments of its application, and are not intended to limit the invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the invention, and all of these belong to the protection scope of the invention.

Claims

1. A method for enhancing the quality of gamma photon industrial reconstruction images based on a parallel deformable convolutional network, comprising the following steps: Step 1: Take the original gamma photon industrial reconstructed image obtained by the ordered subset maximum expectation method (OSEM) as input; Step 2: Learning the global and local features of the industrial reconstructed image through the convolution module; 1) The convolution module is mainly composed of three parallel network branches. The first layers of the three network branches are composed of the initial convolution kernel, the dilated convolution kernel, the deformable convolution kernel and the activation function ReLU to generate the initial global and local feature information maps; 2) The second to fifth layers are composed of initial convolution kernel, dilated convolution kernel, deformable convolution kernel, activation function, and batch normalization, respectively, to improve the nonlinear expression ability of network branches and obtain more sophisticated and complex noise and texture detail feature maps; 3) Each network branch finally outputs a feature information map through 32 convolution kernels of size 3×3; Step 3: Input the global feature and local feature information into the adaptive feature fusion module. The adaptive feature fusion module contains a global feature fusion block and a local feature fusion block. It fuses the differences in noise and texture details in the reconstructed images of different regions to improve the model's ability to restore the details of the reconstructed image. The details are as follows: 1) The local feature information F1 in the global and local feature information output by the convolution module is fused through the attention of the global feature fusion module to obtain the fused global feature information graph F′2. The operation is as follows: After linear mapping, the global feature information F2 is multiplied by the local feature information F1; then the corresponding weight ω is obtained through activation function and normalization F , the corresponding weight ω F After weighted summation with the local feature information F1, a new feature information map F′1 is obtained. Then the new feature information map is linearly mapped and added to the global feature information map F2 to obtain the fused global feature information map F′2. 2) The local feature information F2 in the global and local feature information output by the convolution module is fused through the attention of the local feature fusion module to obtain the fused local feature information graph F′1. The operation is as follows: After linear mapping, the local feature information F1 is multiplied by the global feature information F2; then the corresponding weight ω is obtained through activation function and normalization F ′, the corresponding weight ω F ′ and the global feature information F2 are weighted and summed to obtain a new feature information map F′2, and then the new feature information map is linearly mapped and added to the local feature information map F1 to obtain the fused local feature information map F′1; Step 4: The output of the adaptive feature fusion module is then subjected to a 1×1 vector convolution operation Conv to obtain a high-quality gamma photon industrial reconstructed image.

2. The method for enhancing the quality of gamma photon industrial reconstruction images based on a parallel deformable convolutional network according to claim 1, characterized in that: The initial convolution kernel in the three parallel network branches is the first network branch to learn the local feature information of the input image; the deformable convolution kernel is the second network branch, and the deformable convolution kernel has directionality to learn the global feature information of the input image; The dilated convolution kernel is the third network branch, which enables the convolution kernel to sample at different receptive fields, thereby capturing multi-scale feature information.

3. The method for enhancing the quality of gamma photon industrial reconstruction images based on a parallel deformable convolutional network according to claim 2, characterized in that: The deformable convolution kernel is obtained by performing Fourier interpolation on the convolution kernel. First, the convolution kernel is subjected to Fourier transform. Suppose the convolution kernel is K(m,n), and Fourier transform is performed: Where x and y are the sizes of the convolution kernel, and f(K)(μ, v) represents the convolution kernel in the frequency domain; The convolution kernel is interpolated in the frequency domain. The interpolation process is as follows: Among them, δ is the deformation strength; Finally, the inverse Fourier transform is performed to obtain the deformable convolution kernel K′(m, n). The inverse transform is shown as follows:

4. The method for enhancing the quality of gamma photon industrial reconstruction images based on a parallel deformable convolutional network according to claim 2, characterized in that: The dilation rate of the dilated convolution kernel is 2 or 7.

Images