An image enhancement method based on transunet beam

Abstract

The present application relates to the technical field of ultrasonic nondestructive testing and three-dimensional imaging, and provides an image enhancement method based on a TransUNet beam. The method generates training data through acoustic simulation, takes a low-quality two-dimensional image obtained by single-angle plane wave excitation as network input, and takes a high-quality image obtained by coherent compound processing of multi-angle data as a label. The constructed TransUNet deep neural network combines the multiscale feature extraction capability of a convolutional network and the long-range dependence modeling advantage of a Transformer module, and is used to learn an accurate mapping relationship from sparse data to a high-resolution image. In actual application, the method can frame by frame enhance each cross-sectional image obtained by single-angle rapid scanning, and finally synthesize a high-precision three-dimensional image, thereby realizing efficient and accurate image enhancement of sparse data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, more particularly, the present application relates to an image enhancement method based on a TransUNet beam. BACKGROUND

[0002] In the field of industrial non-destructive testing, three-dimensional ultrasonic imaging technology has become a core means for ensuring manufacturing quality and evaluating structural integrity in key fields such as aerospace, nuclear power equipment, and rail transportation, as it can provide three-dimensional information of the internal structure of the inspected material. Compared with traditional two-dimensional ultrasonic imaging, which can only obtain local cross-sectional information, three-dimensional imaging can clearly reveal the spatial distribution characteristics of complex defects such as cracks, holes, and delamination, providing irreplaceable data support for accurate assessment of remaining life and development of maintenance strategies.

[0003] However, traditional three-dimensional ultrasonic imaging is usually based on point-by-point focusing scanning technology, which has high spatial resolution but extremely low scanning efficiency. For example, in the detection of typical aircraft engine blades, it often takes several hours to complete a complete three-dimensional scanning of a moderately complex component, which severely restricts the practical application of online detection and rapid decision-making.

[0004] To improve the speed of ultrasonic imaging, plane wave imaging (PWI) technology was proposed, which covers the entire imaging area through a single excitation, greatly improving the imaging frame rate. However, this high frame rate comes at the cost of image quality. In practical applications, single-angle plane waves lack directional focusing ability, and energy diffusion can lead to a significant decrease in spatial resolution and signal-to-noise ratio, key image quality indicators. The obtained original images are often difficult to use for accurate identification of subtle defects such as delamination and crack tips, which has led to an urgent need for enhancement processing of such high-speed, low-quality images.

[0005] To compensate for the quality defects of single-angle plane wave imaging, coherent plane wave imaging (CPWI) technology was proposed. This technology acquires and fuses multiple plane wave data at different angles to restore the resolution and signal-to-noise ratio of the image to some extent. However, this method is essentially a "data for quality" physical level enhancement strategy. Each additional excitation angle doubles the amount of data, for example, a 40-angle CPWI scheme can have a radio frequency data flow of up to 5GB / s, which puts a huge burden on data transmission, storage, and real-time processing systems.

[0006] To this end, the image enhancement technology based on deep learning provides a new idea to solve the above-mentioned contradictions. Researchers began to try to use U-Net, ResNet and other convolutional neural network architectures to enhance the low-quality ultrasound images. However, due to the limitation of local receptive field of these traditional image enhancement networks, it is difficult to effectively capture the cross-scale and long-range phase and scattering information in the ultrasound signal, resulting in poor effect in reconstructing high-resolution details from the information sparse single-angle plane wave image. In addition, most of the existing methods still rely on multi-angle original data for training or processing, and have not fundamentally solved the contradiction between data redundancy and real-time performance in the image enhancement process. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art, the present application provides an image enhancement method based on TransUNet beam, which adopts a TransUNet deep neural network to construct an end-to-end mapping from a single-angle plane wave image to a full-angle coherent plane wave composite image. In view of the technical defects in the prior art that the traditional convolutional network is limited by the local receptive field and is difficult to effectively reconstruct global features and details from the information sparse low-quality image, the present method introduces a Transformer module to utilize its long-range dependency modeling capability to realize enhancement from single-angle sparse data to high-resolution images.

[0008] To achieve the above-mentioned purposes, the present application provides the following technical solutions:

[0009] An image enhancement method based on TransUNet beam, comprising the following steps:

[0010] Step one, obtaining sample data for training a TransUNet deep neural network, the sample data including a two-dimensional single-angle plane wave image and a full-angle coherent plane wave composite image corresponding to the two-dimensional single-angle plane wave image, wherein the two-dimensional single-angle plane wave image is obtained by delay and superposition processing of radio frequency data obtained by using a single preset excitation angle plane wave excitation method, and the full-angle coherent plane wave composite image is obtained by coherent plane wave composite imaging processing of all radio frequency data obtained by using a plurality of preset excitation angle plane wave excitation methods;

[0011] Step two, constructing and training the TransUNet deep neural network, taking the two-dimensional single-angle plane wave image as the network input and the full-angle coherent plane wave composite image as the network label, performing supervised learning training on the TransUNet deep neural network until the network model converges;

[0012] Step three, input the two-dimensional single-angle plane wave image to be enhanced into the TransUNet deep neural network trained to convergence to obtain an enhanced two-dimensional image output by the TransUNet deep neural network.

[0013] As a further scheme of the present application, the sample data used for training the TransUNet deep neural network comprises the following steps: a two-dimensional simulation model containing a preset defect is established by using the k-Wave tool, and a time-domain finite difference method simulation is performed on the propagation process of an ultrasonic wave in the two-dimensional simulation model to generate the radio frequency data; the type of the preset defect is a crack or a hole.

[0014] As a further scheme of the present application, the structure of the TransUNet deep neural network is sequentially connected with a convolutional encoder module, a multi-head self-attention Transformer module and a convolutional decoder module; the convolutional encoder module is used for local feature extraction of the input two-dimensional single-angle plane wave image; the multi-head self-attention Transformer module is used for long-range dependency modeling of the local features extracted by the convolutional encoder module; and the convolutional decoder module is used for restoring the spatial resolution of the image and outputting the enhanced two-dimensional image.

[0015] As a further scheme of the present application, the convolutional encoder module comprises a four-level down-sampling structure, each level adopts a 3x3 convolution kernel and a ReLU activation function; the multi-head self-attention Transformer module comprises eight attention heads, and the embedding dimension of each attention head is sixty-four; and the convolutional decoder module adopts a transposed convolution to realize up-sampling and is provided with a skip connection for feature fusion with the corresponding layer of the convolutional encoder module.

[0016] As a further scheme of the present application, the loss function used in the network training is a mean square error loss function.

[0017] As a further scheme of the present application, the sample data needs to be preprocessed before being input into the TransUNet deep neural network, and the preprocessing operation is: performing normalization processing on the two-dimensional single-angle plane wave image to make the pixel value thereof distributed in the interval of 0 to 1; and adding Gaussian noise to the image after the normalization processing.

[0018] As a further scheme of the present application, the two-dimensional single-angle plane wave image to be enhanced is acquired by performing single-angle plane wave imaging scanning on the object to be measured along its thickness direction at a fixed step of 1 millimeter for each cross section, and the two-dimensional single-angle plane wave image acquired by scanning is input into the trained and converged TransUNet deep neural network one by one to obtain an enhanced two-dimensional image corresponding to each cross section position.

[0019] As a further scheme of the present application, the three-dimensional ultrasonic imaging result of the object to be measured is synthesized by sequentially stacking the enhanced two-dimensional images in the order of spatial acquisition of the thickness direction.

[0020] As a further scheme of the present application, the implementation of the plane wave excitation mode is that each array element of the ultrasonic array synchronously transmits, and a preset transmission delay is applied to each array element to form a plane acoustic wave front with a constant tilt angle; the size of the transmission delay is in a linear proportional relationship with the position of the array element in the ultrasonic array and the sine value of the plane wave tilt angle.

[0021] As a further scheme of the present application, the range of the plurality of preset excitation angles is -15° to +15°, and the angle interval is 1°.

[0022] Compared with the prior art, the beneficial effects of the image enhancement method based on the TransUNet beam of the present application are that:

[0023] The present application learns and establishes a direct mapping model from a single-angle plane wave image to a full-angle coherent plane wave composite image by constructing a deep neural network. Compared with the prior art in which a high-quality image can be obtained only by physically superimposing (CPWI) data of dozens of angles actually collected, the image enhancement method of the present application significantly improves the image quality by algorithm under the premise of only single-angle data input, maintains excellent imaging quality under the condition that the data acquisition amount is reduced by more than 80%, and greatly reduces the application cost.

[0024] The present application adopts the TransUNet network architecture, combines the Transformer module with the convolutional encoding and decoding structure to perform the image enhancement task. The conventional convolutional network (such as U-Net) used in the prior art is difficult to effectively capture long-range scattering information in the ultrasonic image due to its inherent local receptive field limitation, resulting in that the enhanced image is easy to lose details. The Transformer module and the self-attention mechanism in the TransUNet can effectively model the global dependency relationship of the image, thereby more accurately recovering fine structure features from the information sparse single-angle image, and improving the representation ability for complex defects. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A flowchart of a TransUNet beam-based image enhancement method of the present application.

[0026] Figure 2 A technical scheme diagram of a TransUNet beam-based image enhancement method of the present application.

[0027] Figure 3 A TransUNet model diagram of a TransUNet beam-based image enhancement method of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0029] Embodiment 1

[0030] The embodiments of the present application disclose a TransUNet-based image enhancement method and show its application in fast three-dimensional ultrasound imaging. The method first acquires a high-quality dataset for training an image enhancement network through simulation means, then constructs and trains a specific TransUNet deep neural network, and finally uses the trained network to enhance low-quality two-dimensional ultrasound images.

[0031] First, sample data for training the TransUNet deep neural network is acquired. This process is completely based on simulation generation to ensure data quality and label accuracy. The embodiments of the present application use the acoustic simulation toolbox k-Wave to establish a two-dimensional simulation model, which is set to 50x50mm2 in size and has a medium sound speed of 6300m / s. The control equation for solving the simulation is the linear acoustic wave equation. In order to make the trained network model have good generalization ability, the simulation model contains multiple material properties and multiple types of defects, of which the defect types explicitly cover the two common types of cracks and holes in industrial detection. In each simulation model, 1-10 damage units of varying quantities are randomly set, and the position of each damage unit is randomly distributed within the model, thereby greatly enriching the diversity of training samples.

[0032] In terms of simulation parameter setting, in order to balance the accuracy and efficiency of calculation, the unit size of the space grid is set to 1-4 microns, the time step is 5e-9, and the total simulation time is 30 microseconds. In terms of excitation source, the embodiment of the application uses a linear ultrasonic transducer array containing 64 elements, the excitation signal is a 5-period Hanning window modulated sinusoidal wave time domain pulse signal, the displacement signal amplitude is 1e-6m, and the center frequency is set to 5MHz. In order to obtain the multi-angle ultrasonic data required for subsequent generation of input samples and labels, the embodiment performs plane wave excitation at 31 different preset excitation angles in the range of-15° to +15° with an interval of 1°. The plane wave at each angle is realized by applying a precise preset excitation delay to each element of the ultrasonic array. By performing 31 independent simulation calculations, each simulation only changes the direction of the excitation signal (i.e. changes the applied excitation delay), without changing the actual position of the excitation point and the receiver. Finally, the complete wave field response information is collected by the fixedly arranged receiver array, and the radio frequency data set containing all angle excitations is recorded.

[0033] After obtaining the original multi-angle radio frequency data, it is processed into input samples and labels suitable for neural network image enhancement supervised learning. The generation method of the input sample is: for each preset excitation angle, the corresponding radio frequency data is extracted and delay sum beam forming processing is performed, and finally a two-dimensional single-angle plane wave image (PWI) is generated. The specific way of delay sum processing is: first, calculate the total delay, which is the sum of the transmission delay of the plane wave from the transducer to any point in the imaging plane and the reception delay of the echo from the point back to a single element of the transducer; then, according to the total delay, the radio frequency data is transformed to obtain the signal strength of the pixel point, and after traversing all pixel points, a single frame of plane wave imaging result is obtained.

[0034] All radio frequency data collected at all 31 preset excitation angles are subjected to coherent plane wave compound imaging processing, and a high-resolution, high-signal-to-noise ratio full-angle coherent plane wave compound image is generated by coherently superimposing the information of each angle. In this way, each low-quality single-angle PWI image has a high-quality full-angle compound image corresponding to it as its learning target label.

[0035] Before being sent into the network for training, preprocessing is also needed. This includes normalizing the original two-dimensional single-angle plane wave image, so that the pixel value distribution range is adjusted to the interval of 0 to 1; at the same time, in order to improve the stability and generalization ability of the network model in the actual noise environment, Gaussian noise is also artificially added to the normalized image. In addition, the data set is globally randomized before training to destroy the potential sequential correlation of the samples and enhance the generalization ability of the model. Finally, the processed data set is divided into training set, validation set and test set according to the ratio of 70%, 20% and 10%.

[0036] Subsequently, the TransUNet deep neural network is constructed and trained, and the detailed structure thereof is as shown in Figure 3 The network receives the preprocessed single-frame PWI image with a size of 256x256 at the input end. The network is composed of three core parts: a convolutional encoder module, a multi-head self-attention Transformer module and a convolutional decoder module. The convolutional encoder module is responsible for local feature extraction of the input two-dimensional single-angle plane wave image, and its internal contains a four-level downsampling structure, each level uses a 3x3 convolution kernel for feature extraction, and uses a ReLU activation function for nonlinear mapping. The multi-head self-attention Transformer module is the key that distinguishes this network from traditional convolutional networks, which uses the self-attention mechanism to model the long-range dependence of image features, and inputs the local features extracted by CNN into the ViT module after embedding and dividing into patches, effectively capturing the scattering and diffraction information in the ultrasound image that spans a large spatial range. In the embodiment of the present application, the Transformer module contains eight attention heads, and the embedding dimension of each attention head is sixty-four. Finally, the convolutional decoder module is responsible for restoring the integrated features processed by the Transformer to the prediction result space. This module uses a progressive upsampling strategy, specifically through transposed convolution to realize the step-by-step improvement of spatial resolution, and in order to prevent the loss of detail information in the downsampling process, each upsampling layer of the decoder is connected with the output of the corresponding layer of the encoder through a jump connection to retain high-frequency details.

[0037] In the training stage, the processed two-dimensional single-angle plane wave image is taken as the input, and the corresponding full-angle coherent plane wave composite image is taken as the label for end-to-end supervised learning training. The loss function used in the training process is the mean square error (MSE) loss function, which continuously calculates the pixel error between the reconstructed image output by the network and the label image to guide the optimization of the network parameters. In the training process, the Adam optimization algorithm is selected, the learning rate is set to 0.001, and the LeakyReLU activation function is used to realize the nonlinear mapping of the features, and the total iteration number is set to 500 times. The training process is always carried out until the network model converges, and finally a trained TransUNet network is obtained, which can accurately map the low-quality two-dimensional single-angle plane wave image to the reconstructed two-dimensional image corresponding to the full-angle coherent plane wave composite image.

[0038] After the network training is completed, the network can be applied to the actual image enhancement task. First, data acquisition is performed on the object to be measured. The operation mode is to scan each section along the thickness direction, i.e., the Z direction, with a fixed step of 1 millimeter. For a structure with a thickness of N millimeters, a single-angle plane wave imaging scan is performed on each section in turn, and a frame of original PWI image corresponding to the section is obtained. Finally, a total of N two-dimensional image sequences arranged in spatial order can be obtained.

[0039] Subsequently, the acquired image sequence is enhanced frame by frame. The N two-dimensional single-angle plane wave images acquired are input into the trained TransUNet network one by one and in order for inference processing of image quality enhancement and reconstruction. After each frame of image is processed by the network, a high-resolution reconstructed two-dimensional image corresponding to the section position is output.

[0040] Finally, when all N prediction results are generated, a series of reconstructed two-dimensional images are spatially superimposed and spliced according to their spatial acquisition order in the Z direction by voxel stacking, so as to complete the three-dimensional image reconstruction of the structure at the voxel level, and synthesize a complete three-dimensional ultrasonic imaging result that can clearly reflect the internal structure and defect distribution of the measured object.

[0041] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0042] Finally: the above only for the preferred embodiments of the present application, and not for limiting the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application, should be included in the scope of protection of the present application.

Claims

1. A TransUNet beam-based image enhancement method, characterized in that, The method comprises the following steps: Step one, obtaining sample data for training a TransUNet deep neural network, the sample data comprising a two-dimensional single-angle plane wave image and a full-angle coherent plane wave complex image corresponding to the two-dimensional single-angle plane wave image, wherein the two-dimensional single-angle plane wave image is obtained by delay-and-sum processing of radio frequency data obtained by using a single preset excitation angle plane wave excitation method, and the full-angle coherent plane wave complex image is obtained by coherent plane wave complex imaging processing of all radio frequency data obtained by using a plurality of preset excitation angle plane wave excitation methods; Step two, constructing and training the TransUNet deep neural network, taking the two-dimensional single-angle plane wave image as network input and the full-angle coherent plane wave complex image as network label, and performing supervised learning training on the TransUNet deep neural network until the network model converges; Step three, inputting a two-dimensional single-angle plane wave image to be enhanced into the trained TransUNet deep neural network to obtain an enhanced two-dimensional image output by the TransUNet deep neural network; The structure of the TransUNet deep neural network is sequentially connected with a convolutional encoder module, a multi-head self-attention Transformer module and a convolutional decoder module; the convolutional encoder module is used for local feature extraction of the input two-dimensional single-angle plane wave image; the multi-head self-attention Transformer module is used for long-range dependency modeling of the local features extracted by the convolutional encoder module; and the convolutional decoder module is used for restoring the spatial resolution of the image and outputting the enhanced two-dimensional image; The convolutional encoder module comprises a four-level down-sampling structure, each level of which adopts a 3×3 convolution kernel and a ReLU activation function; the multi-head self-attention Transformer module comprises eight attention heads, each of which has an embedding dimension of sixty-four; and the convolutional decoder module adopts transposed convolution to realize up-sampling and is provided with a skip connection for feature fusion with the corresponding layer of the convolutional encoder module.

2. The TransUNet beam-based image enhancement method of claim 1, wherein, The sample data for training the TransUNet deep neural network comprises the following steps: using a k-Wave tool to establish a two-dimensional simulation model containing a preset defect, performing time-domain finite difference simulation on the propagation process of ultrasonic waves in the two-dimensional simulation model to generate the radio frequency data; and the type of the preset defect is a crack or a hole.

3. The TransUNet beam-based image enhancement method of claim 1, wherein, The loss function used in network training is a mean square error loss function.

4. The TransUNet beam-based image enhancement method of claim 1, wherein, The sample data needs to be preprocessed before being input into the TransUNet deep neural network, and the preprocessing operation is: performing normalization processing on the two-dimensional single-angle plane wave image to make the pixel value thereof distributed in the interval of 0 to 1; and adding Gaussian noise to the normalized image.

5. The TransUNet beam-based image enhancement method of claim 1, wherein, The two-dimensional single-angle plane wave image to be enhanced is acquired by performing single-angle plane wave imaging scanning on the object to be measured along its thickness direction at a fixed step of 1 millimeter, and the two-dimensional single-angle plane wave image acquired by scanning is input into the trained and converged TransUNet deep neural network one by one to obtain an enhanced two-dimensional image corresponding to each cross-sectional position.

6. The TransUNet beam-based image enhancement method of claim 5, wherein, The enhanced two-dimensional images are sequentially stacked in the order of spatial acquisition in the thickness direction to synthesize a three-dimensional ultrasound imaging result of the object to be measured.

7. The TransUNet beam-based image enhancement method of claim 1, wherein, The implementation of the plane wave excitation mode is that each array element of the ultrasound array is synchronously transmitted, and a preset transmission delay is applied to each array element to form a plane acoustic wave front with a constant tilt angle; the size of the transmission delay is in a linear proportional relationship with the position of the array element in the ultrasound array and the sine value of the plane wave tilt angle.

8. The TransUNet beam-based image enhancement method of claim 1, wherein, The range of the plurality of preset excitation angles is -15° to +15°, and the angle interval is 1°.

Images