Image enhancement method based on TransUNet beam

Abstract

The invention relates to the technical field of ultrasonic nondestructive testing and three-dimensional imaging, and provides an image enhancement method based on TransUNet beams. According to the method, training data is generated through acoustic simulation, a low-quality two-dimensional image obtained through single-angle plane wave excitation serves as network input, and a high-quality image obtained after multi-angle data coherent composite processing serves as a label. The constructed TransUNet deep neural network combines the multi-scale feature extraction capability of a convolutional network and the long-range dependence modeling advantage of a Transform module, and is used for learning the accurate mapping relation from sparse data to a high-resolution image. In practical application, the method can perform frame-by-frame enhancement on each cross-section image obtained by single-angle rapid scanning, and finally synthesize a high-precision three-dimensional image, so that efficient and accurate image enhancement of sparse data is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, more particularly, the present application relates to an image enhancement method based on a TransUNet beam. BACKGROUND

[0002] In the field of industrial non-destructive testing, three-dimensional ultrasonic imaging technology has become a core means for ensuring manufacturing quality and evaluating structural integrity in key fields such as aerospace, nuclear power equipment, and rail transportation, as it can provide three-dimensional information of the internal structure of the inspected material. Compared with traditional two-dimensional ultrasonic imaging, which can only obtain local cross-sectional information, three-dimensional imaging can clearly reveal the spatial distribution characteristics of complex defects such as cracks, holes, and delamination, providing irreplaceable data support for accurate assessment of remaining life and development of maintenance strategies.

[0003] However, traditional three-dimensional ultrasonic imaging is usually based on point-by-point focusing scanning technology, which has high spatial resolution but extremely low scanning efficiency. For example, in the detection of typical aircraft engine blades, it often takes several hours to complete a complete three-dimensional scanning of a moderately complex component, which severely restricts the practical application of online detection and rapid decision-making.

[0004] To improve the speed of ultrasonic imaging, plane wave imaging (PWI) technology was proposed, which covers the entire imaging area through a single excitation, greatly improving the imaging frame rate. However, this high frame rate comes at the cost of image quality. In practical applications, single-angle plane waves lack directional focusing ability, and energy diffusion can lead to a significant decrease in spatial resolution and signal-to-noise ratio, key image quality indicators. The obtained original images are often difficult to use for accurate identification of subtle defects such as delamination and crack tips, which has led to an urgent need for enhancement processing of such high-speed, low-quality images.

[0005] To compensate for the quality defects of single-angle plane wave imaging, coherent plane wave imaging (CPWI) technology was proposed. This technology acquires and fuses multiple plane wave data at different angles to restore the resolution and signal-to-noise ratio of the image to some extent. However, this method is essentially a "data for quality" physical level enhancement strategy. Each additional excitation angle doubles the amount of data, for example, a 40-angle CPWI scheme can generate up to 5GB / s of radio frequency data flow, which puts a huge burden on data transmission, storage, and real-time processing systems.

[0006] To this end, the image enhancement technology based on deep learning provides a new idea to solve the above-mentioned contradictions. Researchers began to try to use U-Net, ResNet and other convolutional neural network architectures to enhance the low-quality ultrasound images. However, due to the limitation of local receptive field of these traditional image enhancement networks, it is difficult to effectively capture the cross-scale and long-range phase and scattering information in the ultrasound signal, resulting in poor effect in reconstructing high-resolution details from the information sparse single-angle plane wave image. In addition, most of the existing methods still rely on multi-angle original data for training or processing, and have not fundamentally solved the contradiction between data redundancy and real-time performance in the image enhancement process. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art, the present application provides an image enhancement method based on TransUNet beam, which adopts a TransUNet deep neural network to construct an end-to-end mapping from a single-angle plane wave image to a full-angle coherent plane wave composite image. In view of the technical defects in the prior art that the traditional convolutional network is limited by the local receptive field and is difficult to effectively reconstruct global features and details from the information sparse low-quality image, the present method introduces a Transformer module to utilize its long-range dependency modeling capability to realize the enhancement from single-angle sparse data to high-resolution image.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solutions: An image enhancement method based on TransUNet beam, comprising the following steps: Step one, obtaining sample data for training a TransUNet deep neural network, the sample data including a two-dimensional single-angle plane wave image and a full-angle coherent plane wave composite image corresponding to the two-dimensional single-angle plane wave image, wherein the two-dimensional single-angle plane wave image is obtained by delay and superposition processing of radio frequency data acquired by a single preset excitation angle plane wave excitation method, and the full-angle coherent plane wave composite image is obtained by coherent plane wave composite imaging processing of all radio frequency data acquired by a plurality of preset excitation angle plane wave excitation methods; Step two, constructing and training the TransUNet deep neural network, taking the two-dimensional single-angle plane wave image as the network input and the full-angle coherent plane wave composite image as the network label, performing supervised learning training on the TransUNet deep neural network until the network model converges; Step three, inputting a two-dimensional single-angle plane wave image to be enhanced into the trained TransUNet deep neural network to obtain an enhanced two-dimensional image output by the TransUNet deep neural network.

[0009] As a further scheme of the present application, the sample data used for training the TransUNet deep neural network comprises the following steps: a two-dimensional simulation model containing a preset defect is established by using a k-Wave tool, and a time-domain finite difference method simulation is performed on the propagation process of an ultrasonic wave in the two-dimensional simulation model to generate the radio frequency data; the type of the preset defect is a crack or a hole.

[0010] As a further scheme of the present application, the structure of the TransUNet deep neural network is sequentially connected with a convolutional encoder module, a multi-head self-attention Transformer module and a convolutional decoder module; the convolutional encoder module is used for local feature extraction of the input two-dimensional single-angle plane wave image; the multi-head self-attention Transformer module is used for long-range dependency modeling of the local features extracted by the convolutional encoder module; and the convolutional decoder module is used for restoring the spatial resolution of the image and outputting the enhanced two-dimensional image.

[0011] As a further scheme of the present application, the convolutional encoder module comprises a four-level down-sampling structure, each level adopts a 3x3 convolution kernel and a ReLU activation function; the multi-head self-attention Transformer module comprises eight attention heads, and the embedding dimension of each attention head is sixty-four; and the convolutional decoder module adopts transposed convolution to realize up-sampling and is provided with a skip connection for feature fusion with the corresponding layer of the convolutional encoder module.

[0012] As a further scheme of the present application, the loss function used in the network training is a mean square error loss function.

[0013] As a further scheme of the present application, the sample data needs to be preprocessed before being input into the TransUNet deep neural network, and the preprocessing operation is: performing normalization processing on the two-dimensional single-angle plane wave image so that the pixel value is distributed in the interval of 0 to 1; and adding Gaussian noise to the image after normalization processing.

[0014] As a further scheme of the present application, the two-dimensional single-angle plane wave image to be enhanced is acquired by performing single-angle plane wave imaging scanning on the object to be tested along the thickness direction thereof at a fixed step of 1 millimeter for each cross section, and the two-dimensional single-angle plane wave images acquired by scanning are input into the trained and converged TransUNet deep neural network one by one to obtain an enhanced two-dimensional image corresponding to each cross section position.

[0015] As a further scheme of the present application, the enhanced two-dimensional images are sequentially stacked in the order of their spatial acquisition sequence in the thickness direction by means of voxel stacking to synthesize a three-dimensional ultrasonic imaging result of the object to be tested.

[0016] As a further scheme of the present application, the implementation of the plane wave excitation mode is that each array element of the ultrasonic array synchronously transmits, and a preset transmission delay is applied to each array element, so as to form a plane acoustic wave front with a constant tilt angle; the size of the transmission delay is in a linear proportional relationship with the position of the array element in the ultrasonic array and the sine value of the tilt angle of the plane wave.

[0017] As a further scheme of the present application, the range of the plurality of preset excitation angles is -15° to +15°, and the angle interval is 1°.

[0018] Compared with the prior art, the beneficial effects of the image enhancement method based on the TransUNet beam of the present application are that: The present application learns and establishes a direct mapping model from a single-angle plane wave image to a full-angle coherent plane wave composite image by constructing a deep neural network. Compared with the prior art in which a high-quality image can be obtained only by physically superimposing (CPWI) data of dozens of angles actually collected, the image enhancement method of the present application significantly improves the image quality by algorithm under the premise of only single-angle data input, maintains excellent imaging quality under the condition that the data acquisition amount is reduced by more than 80%, and greatly reduces the application cost.

[0019] The present application adopts the TransUNet network architecture, combines the Transformer module with the convolutional encoding and decoding structure to perform the image enhancement task. The conventional convolutional network (such as U-Net) used in the prior art is difficult to effectively capture long-range scattering information in the ultrasonic image due to its inherent local receptive field limitation, resulting in that the enhanced image is easy to lose details. The present method uses the Transformer module and the self-attention mechanism in the TransUNet to effectively model the global dependency relationship of the image, so as to more accurately recover fine structure features from the information sparse single-angle image, and improve the representation ability for complex defects. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The figure is a flowchart of the image enhancement method based on the TransUNet beam of the present application.

[0021] Figure 2 The figure is a technical scheme diagram of the image enhancement method based on the TransUNet beam of the present application.

[0022] Figure 3 The figure is a TransUNet model diagram of the image enhancement method based on the TransUNet beam of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0024] Embodiment 1

[0025] The embodiments of the present application disclose an image enhancement method based on TransUNet, and show its application in fast three-dimensional ultrasound imaging. The method first acquires a high-quality dataset for training an image enhancement network through simulation means, then constructs and trains a specific TransUNet deep neural network, and finally uses the trained network to enhance low-quality two-dimensional ultrasound images.

[0026] First, sample data for training the TransUNet deep neural network is acquired. This process is completely based on simulation generation to ensure the accuracy of data quality and labels. The embodiments of the present application use the acoustic simulation toolbox k-Wave to establish a two-dimensional simulation model, which is set to 50x50mm2 in size and 6300m / s in medium sound velocity. The control equation for solving the simulation is the linear acoustic wave equation. In order to make the trained network model have good generalization ability, the simulation model contains multiple material properties and multiple types of defects, of which the defect types explicitly cover the two common types of cracks and holes in industrial detection. In each simulation model, 1-10 damage units of varying quantities are randomly set, and the position of each damage unit is randomly distributed inside the model, thereby greatly enriching the diversity of the training samples.

[0027] In terms of simulation parameter setting, in order to balance the accuracy and efficiency of calculation, the unit size of the space grid is set to 1-4 microns, the time step is 5e-9, and the total simulation time is 30 microseconds. In terms of excitation source, the embodiment of the application uses a linear ultrasonic transducer array containing 64 elements, the excitation signal is a 5-period Hanning window modulated sinusoidal wave time domain pulse signal, the displacement signal amplitude is 1e-6m, and the center frequency is set to 5MHz. In order to obtain the multi-angle ultrasonic data required for subsequent generation of input samples and labels, the embodiment performs plane wave excitation at 31 different preset excitation angles in the range of-15° to +15° with an interval of 1°. The plane wave at each angle is realized by applying a precise preset excitation delay to each element of the ultrasonic array. By performing 31 independent simulation calculations, each simulation only changes the direction of the excitation signal (i.e. changes the applied excitation delay), without changing the actual position of the excitation point and the receiver. Finally, the complete wave field response information is collected by the fixedly arranged receiver array, and the radio frequency data set containing all angle excitations is recorded.

[0028] After obtaining the original multi-angle radio frequency data, it is processed into input samples and labels suitable for neural network image enhancement supervised learning. The generation method of the input sample is: for each preset excitation angle, the corresponding radio frequency data is extracted and delay sum beam forming processing is performed, and finally a two-dimensional single-angle plane wave image (PWI) is generated. The specific way of delay sum processing is: first, calculate the total delay, which is the sum of the transmission delay of the plane wave from the transducer to any point in the imaging plane and the reception delay of the echo from the point back to the transducer. Then, according to the total delay, the radio frequency data is transformed to obtain the signal strength of the pixel point. After traversing all pixel points, a single frame of plane wave imaging result is obtained.

[0029] All radio frequency data collected at all 31 preset excitation angles are subjected to coherent plane wave compound imaging processing, and a high-resolution, high-signal-to-noise ratio full-angle coherent plane wave compound image is generated by coherently superimposing the information of each angle. In this way, each low-quality single-angle PWI image has a high-quality full-angle compound image corresponding to it as its learning target label.

[0030] Before being fed into the network for training, preprocessing is required. This includes normalizing the original two-dimensional single-angle plane wave image to adjust the pixel value distribution range to between 0 and 1. Simultaneously, to improve the stability and generalization ability of the network model in real-world noisy environments, Gaussian noise is artificially added to the normalized image. Furthermore, the dataset is globally shuffled before training to disrupt the potential order associations of samples and enhance the model's generalization ability. Finally, the processed dataset is divided into training, validation, and test sets at a ratio of 70%, 20%, and 10%, respectively.

[0031] Subsequently, the TransUNet deep neural network was constructed and trained, and its detailed structure is as follows: Figure 3 As shown, the network receives a preprocessed single-frame PWI image of size 256x256 at the input. The network consists of three core parts: a convolutional encoder module, a multi-head self-attention Transformer module, and a convolutional decoder module. The convolutional encoder module is responsible for extracting local features from the input two-dimensional single-angle plane wave image. It contains a four-level downsampling structure, with each level using a 3x3 convolutional kernel for feature extraction and a ReLU activation function for nonlinear mapping. The multi-head self-attention Transformer module is the key difference between this network and traditional convolutional networks. Its function is to use a self-attention mechanism to model the long-range dependencies of image features, embedding and dividing the local features extracted by the CNN into patches before inputting them into the ViT module, effectively capturing scattering and diffraction information spanning a large spatial range in ultrasound images. In this embodiment, the Transformer module contains eight attention heads, each with an embedding dimension of 64. Finally, the convolutional decoder module is responsible for restoring the integrated features processed by the Transformer to the prediction result space. This module employs a progressive upsampling strategy, specifically by using transposed convolution to gradually increase spatial resolution. Furthermore, to prevent the loss of detail during downsampling, each upsampling layer of the decoder is fused with the output of the corresponding layer of the encoder through skip connections to preserve high-frequency details.

[0032] During the training phase, the processed 2D single-angle plane wave image is used as input, and its corresponding full-angle coherent plane wave composite image is used as the label for end-to-end supervised learning training. The mean squared error (MSE) loss function is used during training. The optimization of network parameters is guided by continuously calculating the pixel error between the reconstructed image and the label image. The Adam optimization algorithm is used during training, with a learning rate of 0.001, and the LeakyReLU activation function is used to achieve non-linear feature mapping. The total number of iterations is set to 500. The training process continues until the network model converges, ultimately resulting in a stable TransUNet network that can accurately map low-quality 2D single-angle plane wave images to reconstructed 2D images corresponding to the full-angle coherent plane wave composite image.

[0033] After training, the network can be applied to practical image enhancement tasks. First, data is acquired from the object under test by scanning each cross-section along its thickness direction (Z-axis) in fixed steps of 1 mm. For a structure with a thickness of N mm, a single-angle plane wave imaging scan is performed on each cross-section sequentially, acquiring one frame of the original PWI image for that cross-section. This results in a final sequence of N spatially ordered two-dimensional images.

[0034] Subsequently, the acquired image sequence is enhanced frame by frame. The N acquired two-dimensional single-angle plane wave images are sequentially input into the trained TransUNet network for inference processing of image quality enhancement and reconstruction. After processing by the network, each frame outputs a high-resolution reconstructed two-dimensional image corresponding one-to-one with the location of that cross-section.

[0035] Finally, after all N prediction results are generated, this series of reconstructed two-dimensional images are spatially superimposed and stitched together in strict accordance with their spatial acquisition order in the Z direction by voxel stacking. This completes the three-dimensional image reconstruction of the structure at the voxel level, synthesizing a complete three-dimensional ultrasound imaging result that can clearly reflect the internal structure and defect distribution of the measured object.

[0036] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0037] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image enhancement method based on TransUNet beamforming, characterized in that, It includes the following steps: Step 1: Obtain sample data for training the TransUNet deep neural network. The sample data includes a two-dimensional single-angle plane wave image and a full-angle coherent plane wave composite image corresponding to the two-dimensional single-angle plane wave image. The two-dimensional single-angle plane wave image is obtained by delaying and superimposing radio frequency data obtained by plane wave excitation with a single preset excitation angle. The full-angle coherent plane wave composite image is obtained by coherently plane wave composite imaging processing of all radio frequency data obtained by plane wave excitation with multiple preset excitation angles. Step 2: Construct and train the TransUNet deep neural network. Use the two-dimensional single-angle plane wave image as the network input and the full-angle coherent plane wave composite image as the network label. Perform supervised learning training on the TransUNet deep neural network until the network model converges. Step 3: Input the two-dimensional single-angle plane wave image to be enhanced into the TransUNet deep neural network that has been trained and converged, so as to obtain the enhanced two-dimensional image output by the TransUNet deep neural network.

2. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The process of obtaining sample data for training the TransUNet deep neural network includes the following steps: using the k-Wave tool to establish a two-dimensional simulation model containing preset defects, and performing a time-domain finite-difference simulation of the propagation process of ultrasonic waves in the two-dimensional simulation model to generate the radio frequency data; the type of preset defect is a crack or a hole.

3. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The TransUNet deep neural network is structured by sequentially connecting a convolutional encoder module, a multi-head self-attention Transformer module, and a convolutional decoder module. The convolutional encoder module is used to extract local features from the input two-dimensional single-angle plane wave image. The multi-head self-attention Transformer module is used to model long-range dependencies of the local features extracted by the convolutional encoder module. The convolutional decoder module is used to restore the spatial resolution of the image and output the enhanced two-dimensional image.

4. The image enhancement method based on TransUNet beams according to claim 3, characterized in that, The convolutional encoder module includes a four-level downsampling structure, with each level employing a 3×3 convolutional kernel and a ReLU activation function; the multi-head self-attention Transformer module includes eight attention heads, each with an embedding dimension of sixty-four; the convolutional decoder module uses transposed convolution to achieve upsampling and is equipped with skip connections for feature fusion with the corresponding layers of the convolutional encoder module.

5. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The loss function used for network training is the mean squared error loss function.

6. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, Before the sample data is input into the TransUNet deep neural network, it needs to be preprocessed. The preprocessing operation is as follows: normalize the two-dimensional single-angle plane wave image so that its pixel values ​​are distributed in the range of 0 to 1; and add Gaussian noise to the normalized image.

7. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The two-dimensional single-angle plane wave image to be enhanced is acquired by performing single-angle plane wave imaging scans on each cross section of the object under test along its thickness direction with a fixed step of 1 mm. The two-dimensional single-angle plane wave images acquired by scanning are then input one by one into the trained and converged TransUNet deep neural network to obtain an enhanced two-dimensional image corresponding to each cross section position.

8. The image enhancement method based on TransUNet beams according to claim 7, characterized in that, By stacking voxels, the enhanced two-dimensional images are sequentially superimposed according to their spatial acquisition order in the thickness direction to synthesize a three-dimensional ultrasound imaging result of the object under test.

9. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The plane wave excitation method is implemented by synchronously transmitting each element of the ultrasonic array and applying a preset transmission delay to each element, thereby forming a plane acoustic wavefront with a constant tilt angle; the magnitude of the transmission delay is linearly proportional to the position of the element in the ultrasonic array and the sine value of the plane wave tilt angle.

10. The image enhancement method based on TransUNet beams according to claim 1, characterized in that, The range of the multiple preset excitation angles is -15° to +15°, with an angle interval of 1°.

Images