A deep neural network based adaptive beamforming method and system
The adaptive beamforming method using deep neural networks solves the problem of balancing high image quality and low computational complexity in existing technologies. The generated ultrasound image quality is comparable to that of minimum variance beamforming technology, while significantly reducing the computational load.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PILOT ULTRASONIC (CHENGDU) TECHNOLOGY CO LTD
- Filing Date
- 2024-12-11
- Publication Date
- 2026-06-19
AI Technical Summary
Existing adaptive beamforming techniques struggle to achieve both high image quality and low computational complexity in ultrasound imaging, and existing neural network models cannot achieve imaging quality comparable to minimum variance beamforming techniques.
An adaptive beamforming method based on deep neural networks is adopted, including an input module, a beamforming module, and an output module. It uses convolutional layers, residual blocks, nonlinear activation layers, recurrent feature extraction layers, multi-layer recurrent coding layers, and high-resolution feature reconstruction layers, combined with LSTM layers for feature decoding, to directly output high-quality radio frequency data, skipping the complex inverse calculation of the covariance matrix.
It achieves high-quality ultrasound imaging, reduces computational complexity, improves processing speed, and generates image quality comparable to minimum variance beamforming techniques, while significantly reducing computational load.
Smart Images
Figure CN122238508A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of ultrasonic data processing technology, specifically relating to an adaptive beamforming method and system based on deep neural networks. Background Technology
[0002] Beamforming in ultrasound imaging refers to the processing of signals received from multiple probes to focus an ultrasonic beam at a specific direction and depth, thereby obtaining a clear image. Among existing beamforming techniques, traditional scan-line focusing imaging, while providing good image quality, has a limited frame rate. In recent years, with the increasing demand for ultra-high-speed imaging, plane-wave imaging technology has been widely studied due to its ability to improve the frame rate through parallel processing. However, compared to traditional scan-line focusing imaging, plane-wave imaging technology exhibits a significant decrease in image quality.
[0003] To address this issue, adaptive beamforming techniques have been proposed, which improve image quality by adjusting the receiving weights. Existing adaptive beamforming techniques, such as minimum variance beamforming (MV), while improving image quality, suffer from high computational complexity, making real-time processing difficult. Therefore, there is an urgent need in this field to develop a new beamforming method that combines high image quality with low computational complexity.
[0004] Artificial intelligence technology holds promise as a means to address the aforementioned problems, and some research has already made related attempts. For example, Chinese patent "CN201880015836.5 Ultrasonic Imaging System with Neural Network for Deriving Imaging Data and Tissue Information" discloses the use of a neural network to receive echo signals or beamforming signals and generate a first type of ultrasound imaging data. However, existing neural network models are not well-suited to ultrasound imaging tasks, and it is difficult to obtain high-quality images using existing neural network models. In particular, the ultrasound image quality obtained by current neural network models cannot reach the level comparable to minimum variance beamforming (MV) techniques. Therefore, how to further improve neural network models to achieve high-quality ultrasound imaging and low-consumption beamforming remains a technical challenge in this field. Summary of the Invention
[0005] To address the problems of existing technologies, this invention provides an adaptive beamforming method and system based on deep neural networks.
[0006] An adaptive beamforming system based on a deep neural network includes:
[0007] The input module is configured to input ultrasonic radio frequency signal data before beamforming;
[0008] The beamforming module is configured to perform beam synthesis via a deep neural network;
[0009] The output module is configured to output beam-synthesized ultrasonic radio frequency signal data;
[0010] The deep neural network comprises, in sequence: a convolutional layer, a residual block, a summation layer, a nonlinear activation layer, a recurrent feature extraction layer, a multi-layer recurrent coding layer, a high-resolution feature reconstruction layer, and a final feature reconstruction layer.
[0011] Preferably, the ultrasonic radio frequency signal data before beamforming is preprocessed data, and the preprocessing method is normalization processing.
[0012] Preferably, the number of convolutional layers is 3, the convolutional kernel is 3x1, and the stride is 1;
[0013] And / or, the nonlinear activation layer utilizes the Tanh function to process the bipolar data of the channel;
[0014] And / or, the recurrent feature extraction layer uses LSTM to capture long-term temporal dependencies;
[0015] And / or, the final feature reconstruction layer uses an LSTM layer and generates high-quality radio frequency data through the Tanh activation function, with an output data size of 1 scan line * 1024 scan points.
[0016] Preferably, the convolution kernel has batch normalization and ReLU activation.
[0017] Preferably, the multi-layer cyclic coding layer includes two stacked LSTM layers with 64 and 32 hidden units respectively, and a max pooling operation is performed after each LSTM layer.
[0018] Preferably, in the high-resolution feature reconstruction layer, two LSTM layers are used for feature decoding, and an upsampling operation is performed after each LSTM layer. The upsampling factor is 2, and the number of hidden units are 32 and 64, respectively.
[0019] Preferably, the loss function used to train the deep neural network is selected from: the MSE loss function for ultrasonic radio frequency signal data, or the MSE loss function for the envelope curve of the beamforming data.
[0020] Preferably, the training parameters of the deep neural network include:
[0021] An Adam optimizer with a learning rate of 0.001 was used, and a learning rate decay strategy was implemented. Each batch consisted of 32 samples, and a Dropout rate of 0.5 was applied for regularization. The training period was set to 100 epochs, and an early stopping strategy was adopted.
[0022] The present invention also provides an adaptive beamforming method based on a deep neural network, which applies the above-mentioned adaptive beamforming system based on a deep neural network to perform beamforming.
[0023] The present invention also provides a computer-readable storage medium storing: a computer program for implementing the above-described adaptive beamforming system based on a deep neural network, or a computer program for implementing the above-described adaptive beamforming method based on a deep neural network.
[0024] This invention addresses the beamforming task in ultrasound imaging by constructing a novel deep neural network. This deep neural network offers the following advantages in handling beamforming tasks in ultrasound imaging:
[0025] 1. The structure and parameter design of this deep neural network are adapted to the bipolar data characteristics of ultrasound radio frequency (RF) signal data, which can obtain the best image quality.
[0026] 2. This deep neural network uses a non-linear activation function design, which makes the beamforming process more efficient and preserves more image details.
[0027] 3. This deep neural network combines multi-layered recurrent coding layers with encoding and decoding functions, along with a high-resolution feature reconstruction layer, to better capture temporal dependencies and contextual information when processing sequential data, thereby improving the accuracy and stability of beamforming. Furthermore, the multi-layered recurrent coding structure can better handle sequential data of varying lengths, which is particularly important for dynamic scenes in ultrasound imaging.
[0028] 4. Compared with the MV method, this deep neural network skips the complex inverse calculation of the covariance matrix R, directly outputs the RF scan line data after beamforming, and directly learns the synthesis result of MV, avoiding the intermediate learning process, which greatly simplifies the calculation process, reduces the amount of computation, and improves the processing speed.
[0029] In summary, this invention combines the advantages of high ultrasound imaging quality and low power consumption, and has a promising application prospect in ultrasound imaging.
[0030] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions or alterations can be made without departing from the basic technical concept of the present invention.
[0031] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the deep neural network structure in Embodiment 1 of the present invention;
[0033] Figure 2 Comparison of images obtained by DAS, minimum variance beamforming, and the deep neural network method in this embodiment;
[0034] Figure 3 Comparison of images obtained by ResGAN, UnetGAN, WSRGAN and the deep neural network method of this embodiment. Detailed Implementation
[0035] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.
[0036] Example 1: Adaptive Beamforming Method and System Based on Deep Neural Networks
[0037] An adaptive beamforming system based on deep neural networks includes:
[0038] The input module is configured to input ultrasonic radio frequency signal data before beamforming;
[0039] The beamforming module is configured to perform beam synthesis via a deep neural network;
[0040] The output module is configured to output beam-synthesized ultrasonic radio frequency signal data;
[0041] The deep neural network comprises, in sequence: a convolutional layer, a residual block, a summation layer, a nonlinear activation layer, a recurrent feature extraction layer, a multi-layer recurrent coding layer, a high-resolution feature reconstruction layer, and a final feature reconstruction layer.
[0042] In this embodiment of the deep neural network, after the input layer receives the preprocessed RF data, it performs feature extraction via convolutional kernels with batch normalization and ReLU activation. This is followed by residual blocks, designed to allow information to bypass one or more layers and directly pass to the next, preserving information while introducing nonlinearity. Summation layers enhance the stability of deep network training. Nonlinear activation layers utilize the Tanh function to process the data, converting it into one-dimensional preliminary scanline RF data. Subsequently, recurrent feature extraction layers employ LSTM to capture long-term temporal dependencies, combining batch normalization and ReLU activation to enhance learning ability. Multi-layer recurrent encoding layers reduce spatial dimensionality through stacked LSTM layers and max-pooling operations, extracting higher-level feature representations. The high-resolution feature reconstruction layer employs the inverse operation of feature decoding and upsampling layers, using LSTM to decode features and upsampling to restore the data size, reconstructing high-resolution features. Finally, through a series of LSTM layers and subsequent Tanh activation, the final feature reconstruction layer generates high-quality RF data.
[0043] Specifically, in this embodiment, the method for acquiring and preprocessing ultrasonic radio frequency signal data is as follows:
[0044] A high-frequency linear array ultrasonic probe (8-20MHz) is used for voltage excitation. The excitation voltage forms a deflection sound field through delay array operation. The array delay is calculated based on the spatial relative position. The transmitted plane wave deflection angle ranges from -15° to +15°, with an angle interval of 0.5°, for a total of 61 angles. Both the transmit and receive channels have 128 channels. 61 consecutive plane waves are transmitted, and echo signals from the scattered points of each of the 128 channels are received simultaneously. Each channel is digitized at a sampling rate of 40MHz, receiving 4096 sampling points, with a quantization bit width of 16 bits per sampling point.
[0045] To train the deep neural network, this embodiment constructed a dataset containing 1000 different imaging scenes. A total of 128 (channels) x 4096 (sampling points) x 61 (angles) x 1000 (scenes) x 16 (bits) = 477Gb of data was collected.
[0046] The training data was labeled as follows: beamforming was performed on the plane wave data at each angle, with 128 scan lines and 4096 scan points for each scene. The required delay was calculated using the path difference from the current point to each array element. For each scan point, the data from all 128 channels were delayed and then superimposed. The superposition methods included the traditional DAS method and the adaptive MV algorithm. DAS weighted the 128 channels of data, with the weighting coefficients being fixed window functions. The MV algorithm constructed an autocorrelation matrix from the 128 channels of data and, based on statistical characteristics and constraints, performed the inverse matrix operation to calculate the weighting coefficients for each channel. A total of 61 (angles) x 1000 (scenes) = 61000 images were reconstructed, each containing 128 scan lines after beamforming, with each scan line having 4096 scan points.
[0047] During training, the input data consisted of 128 channels of delayed channel data, and the output was 128 channels of weighted and superimposed scan line data. The length of each scan line was chosen to be 1024, which ensured that the deep neural network could be trained according to the scan line context. We randomly selected 200,000 scan line segments from the 61,000 image data points. For each scan line segment, the 128 channels of pre-beamforming data at its corresponding position were used as input (128 channels * 1024 scan points), the scan line synthesized by MV (1 scan line * 1024 scan points) was used as output, and the scan line synthesized by DAS (1 scan line * 1024 scan points) was used for comparison.
[0048] After normalizing the data with dimensions of 128*1024*200000 to improve generalization, it is used as input to the deep neural network for learning. The learned label data is the scan line data with dimensions of 1024*200000 at the corresponding position.
[0049] In this embodiment, the structure of the deep neural network is as follows: Figure 1 As shown, it specifically includes:
[0050] 1. Convolutional layer
[0051] The input is data with 128 channels * 1024 scan points. The channel dimension is compressed by a convolutional layer with a kernel of 3x1 and a stride of 1. After passing through this layer, the output has 64 channels. Then it passes through two more convolutional layers with kernels of 3x1 and a stride of 1.
[0052] 2. Residual Block
[0053] The direct transfer of features is achieved through residual connections, which ensures that gradient vanishing of data features is avoided and enables more effective nonlinear fitting of data.
[0054] 3. Summation layer
[0055] The output of the residual block is added to the input data element by element.
[0056] 4. Nonlinear activation layer
[0057] The data undergoes nonlinear transformation using the Tanh activation function, which ensures the bipolarity of the ultrasound channel data and enables more efficient beamforming conversion.
[0058] The above structure enables the conversion of 128-dimensional beamforming data before beamforming to 1-dimensional beamforming data after beamforming.
[0059] 5. Recurrent Feature Extraction Layer
[0060] The input is data with 1 scan line * 1024 scan points, which is passed through an LSTM layer with a hidden unit of 128.
[0061] 6. Multi-layer cyclic coding layer
[0062] A multi-layer recurrent encoding layer is used by stacking two LSTM layers with 64 and 32 hidden units respectively, and a max pooling operation is performed after each layer.
[0063] 7. High-resolution feature reconstruction layer
[0064] The high-resolution feature reconstruction layer reconstructs high-resolution features step by step through two LSTM layers and upsampling operations, with 32 and 64 hidden units, respectively.
[0065] 8. Final Feature Reconstruction Layer
[0066] Finally, an LSTM layer is used, and high-quality RF data is generated through the Tanh activation function, with an output data size of 1 scan line * 1024 scan points.
[0067] The recurrent feature extraction layer, multi-layer recurrent coding layer, high-resolution feature reconstruction layer, and final feature reconstruction layer all employ ReLU or Tanh activation functions and batch normalization. This feature decoding selectively retains or forgets certain data from the time series to avoid the gradient explosion problem.
[0068] During model training, the input data before beamforming is transformed into beamformed data through nonlinear fitting and feature decoding. The beamformed data is then compared with the data obtained from the MV algorithm for parameter optimization. The selected loss function includes, but is not limited to, the MSE loss function for ultrasonic radio frequency signal data, or the MSE loss function for the envelope curve of the beamformed data. In this embodiment, the loss function used is the MSE loss function for radio frequency data.
[0069] Training was performed on an Ubuntu 20.04 LTS system and TensorFlow 2.3 environment, using an NVIDIA GeForce RTX3080 GPU. An Adam optimizer with a learning rate of 0.001 was used, with a learning rate decay strategy implemented to optimize the training process. Each batch consisted of 32 samples, and a Dropout rate of 0.5 was applied for regularization to prevent overfitting. The training epochs were set to 100 epochs, and an early stopping strategy was employed: training was stopped if there was no improvement in validation set performance within 10 consecutive epochs, ensuring the model achieved optimal performance on the validation set.
[0070] To verify the performance of the system and method in this embodiment, the ultrasonic radio frequency signal data output by the system in this embodiment is quantitatively analyzed with the DAS method and the minimum variance beamforming data. By calculating indicators such as resolution and contrast, the improvement effect of the network on the ultrasonic imaging quality is compared and analyzed.
[0071] The results are shown in Table 1 and Figure 2 As shown.
[0072] Table 1: Point Scattering Resolution and Contrast
[0073] parameter DAS MV This embodiment FWHMlat(mm) 0.91 0.74 0.72 FWHMax(mm) 0.46 0.44 0.44 CNR (dB) 6.25 8.86 8.91
[0074] As can be seen from Table 1, compared with the DAS method, the deep neural network in this embodiment can effectively improve the lateral resolution and contrast, and the results are comparable to the best MV method in the prior art.
[0075] Figure 2 Images obtained by three methods are shown. The image quality of MV and the deep neural network reconstruction in this embodiment is better than that of the DAS method, and the contrast of the image generated by the deep neural network beamforming in this embodiment is better than that of the image obtained by the MV method.
[0076] Furthermore, this embodiment further compares the image quality generated by beamforming using different neural network models. The neural network models compared include ResGAN, UnetGAN, and WSRGAN (the neural network structure can be found in Engineering Applications of Artificial Intelligence, 127(2024)107384), and other process steps and parameters are performed according to the description in this embodiment. The results are as follows... Figure 3As shown in the comparison, the deep neural network in this embodiment can clearly display the tissue structure in the longitudinal section of the human carotid artery compared to other methods, and also has sharper blood vessel edges on the vessel wall. The overall image contrast is greatly improved, and it also has a strong suppression effect on speckle noise present in the tissue.
[0077] Therefore, the method and system provided by this invention generate ultrasound images of comparable quality to the best existing MV method, while requiring significantly less computation. Thus, this invention balances the requirements of high ultrasound imaging quality and low power consumption, and has excellent application prospects.
Claims
1. An adaptive beamforming system based on a deep neural network, characterized in that, include: The input module is configured to input ultrasonic radio frequency signal data before beamforming; The beamforming module is configured to perform beam synthesis via a deep neural network; The output module is configured to output beam-synthesized ultrasonic radio frequency signal data; The deep neural network comprises, in sequence: a convolutional layer, a residual block, a summation layer, a nonlinear activation layer, a recurrent feature extraction layer, a multi-layer recurrent coding layer, a high-resolution feature reconstruction layer, and a final feature reconstruction layer.
2. The adaptive beamforming system based on a deep neural network according to claim 1, characterized in that: The ultrasonic radio frequency signal data before beamforming is preprocessed data, and the preprocessing method is normalization.
3. The adaptive beamforming system based on a deep neural network according to claim 1, characterized in that: The number of convolutional layers is 3, the convolutional kernel is 3x1, and the stride is 1. And / or, the nonlinear activation layer utilizes the Tanh function to process the bipolar data of the channel; And / or, the recurrent feature extraction layer uses LSTM to capture long-term temporal dependencies; And / or, the final feature reconstruction layer uses an LSTM layer and generates high-quality radio frequency data through the Tanh activation function, with an output data size of 1 scan line * 1024 scan points.
4. The adaptive beamforming system based on a deep neural network according to claim 3, characterized in that: The convolution kernel has batch normalization and ReLU activation.
5. The adaptive beamforming system based on a deep neural network according to claim 1, characterized in that: The multi-layer cyclic coding layer includes two stacked LSTM layers with 64 and 32 hidden units respectively, and a max pooling operation is performed after each LSTM layer.
6. The adaptive beamforming system based on a deep neural network according to claim 1, characterized in that: In the high-resolution feature reconstruction layer, two LSTM layers are used for feature decoding, and an upsampling operation is performed after each LSTM layer. The upsampling factor is 2, and the number of hidden units are 32 and 64, respectively.
7. The adaptive beamforming system based on a deep neural network according to any one of claims 1-6, characterized in that: The loss function used to train the deep neural network is selected from: the MSE loss function for ultrasonic radio frequency signal data, or the MSE loss function for the envelope curve of beamforming data.
8. The adaptive beamforming system based on a deep neural network according to any one of claims 1-6, characterized in that: The training parameters of the deep neural network include: An Adam optimizer with a learning rate of 0.001 was used, and a learning rate decay strategy was implemented. Each batch consisted of 32 samples, and a Dropout rate of 0.5 was applied for regularization. The training period was set to 100 epochs, and an early stopping strategy was adopted.
9. An adaptive beamforming method based on deep neural networks, characterized in that: Beamforming is performed using the adaptive beamforming system based on deep neural networks as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, It stores: a computer program for implementing the adaptive beamforming system based on a deep neural network as described in any one of claims 1-8, or a computer program for implementing the adaptive beamforming method based on a deep neural network as described in claim 9.
Citation Information
Patent Citations
An ultrasound imaging system with a neural network for deriving imaging data and tissue information.
CN110381845B