Method and system for adaptive beamforming of ultrasound signals

By using trained artificial neural networks to determine the content adaptive toe weight of ultrasound signals, the problem of time-consuming calculation of traditional adaptive beamforming algorithms is solved, achieving high-quality real-time ultrasound imaging and reducing computing costs.

CN112912758BActive Publication Date: 2024-11-01KONINKLIJKE PHILIPS NV

Patent Information

Application Number
CN201980070368.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-03-21
Filing Date
2019-10-22
Publication Date
2024-11-01
Estimated Expiration
2039-10-22

AI Technical Summary

Technical Problem

The existing adaptive beamforming algorithms take a long time to calculate, making it unsuitable for real-time ultrasound imaging, and it is difficult to realize real-time image reconstruction with high complexity matrix calculations.

Method used

Using trained artificial neural networks to determine content adaptive toe weights, instead of traditional computationally expensive methods such as minimum variance beamformers, operate in the time domain through the neural network and apply to ultrasonic signals for beamforming.

Benefits of technology

The calculation time is significantly reduced, achieving the same high image quality as traditional adaptive beamforming algorithms, while reducing the computational cost, making real-time ultrasound imaging possible.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112912758B_ABST
    Figure CN112912758B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for adaptive beamforming of ultrasound signals, the method comprising the steps of: (a) receiving time-aligned RF signals acquired by a plurality of ultrasound transducer elements in response to ultrasound transmissions; (b) determining content-adaptive apodization weights for beamforming the time-aligned RF signals by applying a trained artificial neural network (16) to the time-aligned RF signals; and (c) applying the content-adaptive apodization weights to the time-aligned RF signals to calculate a beamformed output signal. The present invention also relates to a method for training an artificial neural network (16) useful in adaptive beamforming of ultrasound signals, and related computer programs and systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for adaptive beamforming of ultrasound signals, a method for training an artificial neural network useful in adaptive beamforming of ultrasound signals, and a related computer program and system. Background Art

[0002] Conventional ultrasound imaging methods typically use delay-and-sum (DAS) beamforming, as described in KEThomenius, "Evolution of ultrasound beamformers," (1996 IEEE Ultrason. Symp. Proc., Vol. 2, pp. 1615-1622, 1996), due to its low complexity and fast reconstruction time. This method uses fixed, content-invariant, apodization weights for receive channels. Although its reconstruction speed allows real-time imaging, DAS beamforming does not provide optimal image contrast and resolution because it lacks content-adaptive array apodization. For example, backscatter from off-axis components is not adequately compensated. To improve this relatively poor quality, improved images are often obtained by compounding multiple acquisitions and / or using focused beams, both at the expense of increased acquisition time.

[0003] Adaptive beamforming algorithms improve on this by determining the optimal content adaptive apodization weights based on the acquired RF signal and applying them to the receive channel. However, these methods are computationally more demanding and therefore result in significantly longer reconstruction times. Therefore, currently known adaptive beamforming algorithms are not suitable for real-time ultrasound imaging.

[0004] A known adaptive beamforming algorithm is the minimum variance (MV) beamformer, in which the apodization weights are continuously optimized to minimize the variance of the received signal after apodization while maintaining unity gain in the desired direction. This process effectively suppresses the power of interfering signals from undesired directions, which typically results in a clutter image. Although MV beamforming has been shown to significantly improve resolution and contrast compared to DAS, it is also very slow, relying on having n 3 The computation of the nxn spatial covariance matrix is ​​a computationally demanding version of the complexity of , where n is the number of channels. Therefore, MV beamforming is not used in real-time imaging.

[0005] For example, in IK Holfort, Adaptive Beamforming for Medical Ultrasound Imaging (PhD Thesis, Technical University of Denmark, Lyngby, November 2009) and JF MV beamforming methods are described in A. Austeng, and S. Holm, "Benefits of minimum-variance beamforming inmedical ultrasound imaging," (IEEE Trans. Ultrason. Ferroelectr. Freq. Control, Vol. 56, No. 9, pp. 1868-1879, 2009). In the context of beamforming, "apodization" can be described as introducing a weighting function ("apodization weight") across the array, or to the signals acquired by the array. The general principles of beamforming are described, for example, in Chapter 2 of IK Holfort's doctoral thesis, Adaptive Beamforming for Medical Ultrasound Imaging, and are incorporated herein by reference.

[0006] In MV beamforming, the apodization weight w is determined by solving:

[0007] min w H Rw,

[0008] Undergo w H a=1,

[0009] where a is the steering vector and R is the covariance matrix of the input array. The solution to this minimization problem is

[0010]

[0011] Since the inverse of the covariance matrix R is computationally very expensive, especially for large matrices, it is not suitable for real-time implementation.

[0012] Artificial neural networks have been proposed in the context of ultrasound imaging for image formation and characterization, see for example WO2018 / 127498 A1. According to the document, an ultrasound imaging system may include: an ultrasound transducer configured to acquire echo signals; and a channel memory configured to store the acquired echo signals. The system also includes: a neural network coupled to the channel memory and configured to receive one or more samples of the acquired echo signals or beamforming signals or both; and to provide imaging data based on one or more samples of the acquired echo signals. The neural network is trained to generate imaging data or extract tissue information.

[0013] According to Adam Luchies and Brett Byram, "Suppressing off-axis scattering using deep neural networks" (Proceedings of SPIE 10580, Medical Imaging 2018: Ultrasonic Imaging and Tomography, 105800G, March 6, 2018), the RF signals (also referred to as channel data) acquired from the ultrasound transducer elements are short-time Fourier transformed and used as input to the deep neural network, which therefore operates in the frequency domain. The output has the same structure as the input and the inverse short-time Fourier transform is used to convert the processed data back to the time domain to generate processed channel data that is summed across channels.

[0014] In addition, in "Deep Neural Networks for Ultrasound Beamforming" (IEEE TRANSACTIONS ON MEDICAL IMAGING, Vol. 37, No. 9, September 2018), Adam Luchies and Brett Byram have disclosed the use of a deep neural network for suppressing off-axis scatter in ultrasound channel data. Thus, Fourier transformed channel data is processed by a deep neural network to produce channel data corresponding to improved ultrasound image quality.

[0015] Similarly, in "Robust Photoacoustic Beamforming Using Dense Convolutional Neural Networks" (Springer Nature Switzerland Ag 2018, LNCS 11042, pp. 3-11, 2018), Anas et al. proposed using a beamforming method based on a deep convolutional neural network to improve photoacoustic image reconstruction. Thus, the deep convolutional neural network maps the input channel data to the output photoacoustic image.

[0016] It is an object of the present invention to provide an adaptive beamforming method which is able to provide high image quality while having a reduced computational cost compared to known content adaptive beamforming methods. In particular, it is an object of the present invention to provide a solution which is able to provide the same high image quality as conventional adaptive beamforming algorithms (such as MV beamforming) while reducing the computational time to such an extent that it can be used for real-time ultrasound imaging. Summary of the invention

[0017] These objectives are met or exceeded by a method for adaptive beamforming of ultrasound signals according to claim 1, a method for providing a trained artificial neural network according to claim 13, a computer program according to claim 14 and a system according to claim 15. Any features, advantages or alternative embodiments described herein with respect to the claimed method also apply to other claim categories, in particular the claimed system, computer program and ultrasound imaging device. In particular, the trained neural network can be adjusted by the claimed training method. Furthermore, the input data of the artificial neural network can include advantageous features and embodiments of the input training data, and vice versa. Furthermore, the output of the trained neural network (content-adaptive apodization weights) can include advantageous features and embodiments of the output training data, and vice versa.

[0018] According to a first aspect, the present invention provides a method for adaptive beamforming of an ultrasound signal, the method comprising the steps of: (a) receiving an RF signal acquired by a plurality of ultrasound transducer elements in response to ultrasound transmission; and (b) determining content adaptive apodization weights for beamforming the RF signal by applying a trained artificial neural network to the RF signal.

[0019] Thus, the inventive method is able to obtain optimal or close to optimal apodization weights in a content-adaptive manner by using an artificial neural network (NN) as an alternative to current computationally expensive methods, such as MV beamforming. "Content-adaptive" in this context means that the apodization weights depend on the acquired RF signal, i.e. they are adapted to the content of the signal. In the case of ultrasound imaging, they are adapted to the imaging field of view. Contrary to the prior art, the output of the NN does not take the form of an RF signal (channel data) or a beamformed image, but the output is an apodization weight for each channel, which can then be applied to the RF signal to produce a beamformed output signal in the usual way, for example by multiplication and addition. Thus, the input to the NN is the RF signal, and these same signals can then be multiplied by the output of the NN, i.e. the apodization weights, and subsequently summed to produce a beamformed output signal.

[0020] The complete adaptive beamforming or image reconstruction process can be divided into 3 steps: 1) time-of-flight correction, 2) adjustment (and application) of the apodization weights and 3) envelope detection. The inventive method focuses on the calculation of the adaptive apodization weights, which is the most computationally expensive of the 3 steps mentioned above.

[0021] The input to the NN is a radio frequency (RF) signal acquired by a plurality of ultrasonic transducer elements in response to ultrasonic transmission into a medium, such as a part of a human or animal body. The RF signal as acquired has both amplitude and phase. Therefore, the RF signal used as input to the NN can take a real or complex representation. In ultrasonic imaging, the area imaged by the ultrasonic transmission is called the field of view. The plurality of ultrasonic transducer elements can be a plurality or all elements of an array transducer, for example, a linear transducer or a phased array. In some embodiments, the toe-off weights will be calculated for all RF signals acquired from the transducer elements in the array, but there are also imaging methods in which only a subset of the transducer elements in the array receive RF signals at any given time.

[0022] In a useful embodiment, the input of the NN is a time-aligned RF signal, i.e. a time-shifted RF signal or an RF signal after a time-of-flight correction, wherein the delay applied in the time-of-flight correction is relative to the transducer geometry. Such a time correction can be done in the same way as in DAS beamforming. Preferably, the method operates in the time domain, i.e. on a signal in the time domain. In order to generate a time-aligned RF signal that can form an input for the NN, according to a useful embodiment, a pixel-by-pixel time-of-flight correction is applied to the RF signal as received (raw RF signal). This results in a data array of size N, wherein N corresponds to the number of contributing transducer elements. Therefore, the method may additionally comprise the following step: performing a time-of-flight correction on the signal to generate a time-aligned signal.

[0023] The RF signal (also referred to as channel data) is fed into a trained artificial neural network, also referred to as a "neural network" or NN. An artificial neural network (NN) is based on a collection of connected artificial neurons, also referred to as nodes, where each connection (also referred to as an edge) can send a signal from one node to another. Each artificial neuron that receives a signal can process it and transmit it to another artificial neuron connected to it. In a useful embodiment, the artificial neurons of the inventive NN are arranged in layers. The input signal (e.g., the time-aligned RF signal) travels from the first layer (also referred to as the input layer) to the last layer (the output layer). In a useful embodiment, the NN is a feedforward network, although it may also be part of a recursive neural network or may be a recursive neural network so that it can remember its internal state and use it in the processing of a sequence of related inputs. In a useful embodiment, the NN is trained based on machine learning techniques (particularly deep learning), for example by back propagation. The training data used may be the output of a high-quality adaptive beamforming algorithm, such as an MV beamformer as a training target, and the related time-aligned RF signal as input training data. In a useful embodiment, the present invention teaches a model-based approach using a designed NN architecture that employs an adaptive beamformer structure while using a neural network to perform computationally demanding calculations.

[0024] The trained artificial neural network according to the present invention may be provided in the form of a software program, but may also be implemented as hardware. In addition, the trained NN may be provided in the form of a training function that does not have to be structured in exactly the same way as the trained neural network - for example, if a particular connection / edge has a weight of 0, then after training, such a connection may be omitted when providing a trained function based on the trained artificial NN.

[0025] After training, the use of the inventive NN was shown to result in higher image quality that approximates (if not improves beyond) the results of the MV beamformer but at a fraction of the computation time. According to current experiments, the time gain is approximately 500 times. Thus, the present invention provides an extremely high quality adaptive beamforming method that provides the same quality as known adaptive beamforming algorithms (such as MV beamformers), but at a fraction of the computation cost. Thus, the inventive method allows a much better quality of ultrasound images to be obtained in real time. Thus, the present invention provides image quality comparable to adaptive beamforming techniques but at high rates, while using conventional and / or GPU accelerated hardware.

[0026] According to a useful embodiment, the number of input nodes of the artificial neural network and the number of output nodes of the artificial neural network correspond to the number of contributing RF signals, where "contributing" means that the contributing RF signals are contributing to the corresponding beamforming output signal, such as a specific beamforming pixel. Therefore, the number of input nodes N corresponds to the number of receiving ultrasound transducer elements, which can be all transducer elements in the transducer array or a subset thereof. Preferably, each output node supplies content-adaptive apodization weights for one RF signal fed to one specific input node. Therefore, in most embodiments, the number of input nodes will be the same or approximately the same as the number of output nodes of the NN.

[0027] In an advantageous embodiment, in a next step, the content-adaptive apodization weights determined by the NN are applied to the RF signal - i.e., the same RF signal fed to the NN - to calculate the beamformed output signal. This can be done by adding a second connection from the input (e.g., the time-aligned RF signal) to a multiplication and addition element, wherein, signal by signal, the multiplication is performed between each of the time-aligned RF signals and each of the apodization weights appearing at the output of the NN. The beamformed output signal is preferably an RF signal corresponding to a pixel in the ultrasound image in the field of view. Thus, in a useful embodiment, the time-aligned RF signal corresponding to one pixel after another will be propagated through the neural network, and the same RF signal weighted with the content-adaptive apodization weights is summed to produce the beamformed output signal pixel by pixel. The beamformed output signal can be further processed by further method steps known in the art, in particular envelope detection, to produce the corresponding pixel value.

[0028] In a useful embodiment, the method comprises the steps of receiving raw RF signals acquired from a field of view by a plurality of ultrasound transducer elements in response to ultrasound transmission, followed by time-of-flight correction to generate the time-aligned RF signal. Such time-of-flight correction may be accomplished according to methods known in the art, for example, as described in the paper Adaptive Beamforming for Medical Ultrasound Imaging by IK Holfort (PhD Thesis, Technical University of Denmark, Lyngby, November 2009). The time-aligned RF signal may typically have a frequency in the range of 1-50 MHz. The sampling rate of the signal acquired by the transducer elements will typically be higher than 50 MHz, for example at 100-400 MHz, typically 200 MHz.

[0029] According to a useful embodiment, the inventive method is a computer-implemented method for adaptive beamforming of ultrasound signals, the method comprising the following steps:

[0030] a) receiving raw RF signals acquired from a field of view by an array of ultrasound transducers in response to ultrasound transmission;

[0031] b) performing a time-of-flight correction on the raw RF signal to generate a time-aligned RF signal;

[0032] c) determining content adaptive apodization weights for beamforming the time-aligned RF signals by applying a trained artificial neural network to the time-aligned RF signals;

[0033] d) applying the content adaptive apodization weights to the time aligned RF signal to compute a beamformed output signal.

[0034] The neural network preferably comprises several layers, including hidden layers, and is therefore preferably a deep network. The values ​​of the nodes of the (N+1)th layer are calculated based on the values ​​of the nodes of the Nth layer. This can be done by multiplying the value of each node in the Nth layer by the associated weight of the corresponding connection / edge (not to be confused with the toe-off weight) and adding it to one or several nodes in the (N+1)th layer. This operation is equivalent to matrix multiplication. A fully connected layer can be defined by the fact that most (in particular all) edges between the nodes of the fully connected layer and the nodes of the subsequent layer or the nodes of the previous layer exist, and wherein the weight of each of the edges can be adjusted individually. In addition, a bias can be added to the value of each node of the layer, and wherein the bias is also a trainable parameter.

[0035] In a preferred embodiment, the artificial neural network comprises at most four fully connected layers. In some embodiments, even two or three fully connected layers may be sufficient. The inventors have found that a relatively small neural network (thus requiring only little computational effort) is sufficient to achieve perfect results. For example, there may be a fully connected input layer and one or two fully connected hidden layers, and a fully connected output layer. Alternatively, there may be a fully connected input layer and two or three fully connected hidden layers, wherein the last fully connected hidden layer is directly connected to the output layer. In a useful embodiment, the NN comprises at most two or three fully connected hidden layers.

[0036] In a useful embodiment, the NN will also include an activation layer, which is preferably arranged between the fully connected layers. An activation layer is an activation layer that applies a non-linear function (called an "activation function" or "transfer function") to the value of each node and propagates the output of the activation function to the corresponding nodes in the following layers. In some nomenclatures, the activation function is also called a layer, but is partially considered to be another layer (i.e., a fully connected layer), but in this document, it should be called an "activation layer".

[0037] In a useful embodiment, the artificial neural network comprises at least one activation layer, comprising an activation function that propagates both positive and negative input values ​​with unbounded output values. By "unbounded", it is meant that the output value of the activation function is not limited to any particular value (such as +1 or -1). Preferably, any value can be obtained in principle, thereby preserving the dynamic range of the RF signal. An example of such a function may be an inverse rectifier function or a concatenated rectifier linear unit (CReLU), as described in the article "Understanding and Improving Convolutional Neural Networks via Concatenated Rectifier Linear Unit" by Shang et al. (Proceedings of the 33rd International Conference on Machine Learning, New York, USA, 2016).

[0038] Due to the large dynamic range of the RF signal fed to the NN (which, among other things, has positive and negative components), conventional activation functions, such as hyperbolic tangent and rectifier linear unit (ReLU), are very unsuitable for application of the present invention. ReLU avoids vanishing gradients due to its positive unbounded output. However, such nonlinearity may be inappropriate when processing RF input data because it inherently leads to many dying nodes, which impairs the training process. In contrast, the hyperbolic tangent activation is able to retain negative values. However, it is limited between -1 and 1 and therefore tends to saturate quickly for signals with a large dynamic range, which leads to vanishing gradients during backpropagation. Instead, in a useful embodiment, the present invention uses an activation function that introduces nonlinearity while retaining the negative signal component as well as the dynamic range of the signal. Therefore, the quality of the beamformed output image is improved using such an activation function. In addition, the neural network can be designed to accurately reproduce the adaptive beamforming target image while having a minimum network size.

[0039] In a further useful embodiment, the neural network comprises at least one activation layer comprising an activation function, the activation function concatenating the positive and negative parts of the input value, wherein "input value" means the value fed to the activation layer, i.e. the value of the nodes of the activation layer. Thus, negative and positive values ​​are propagated to different nodes in the next layer, wherein the value of at least one of the two nodes is 0. Thus, the number of nodes in the next layer is as high as twice the number of nodes in the activation layer. In other words, the output of the activation layer will have twice the size of the input. In a useful embodiment, the output will always be positive, which has the advantage that the activation preserves energy information, i.e. it keeps the module of the signal, but also preserves both positive and negative signal information.

[0040] In a useful embodiment, the activation function can be an antirectifier function that combines sample-by-sample L2 normalization with two ReLU activations to concatenate the positive and negative parts of the input, as described in F. Chollet, “Antirectifier,” GitHub, 2018. [Online]. Available: https: / / github.com / keras-team / keras / blob / 4f2e65c385d60fa87bb143c6c506cbe428895f44 / examples / antirectifier.py.

[0041] The reverse rectifier operation can be described as:

[0042]

[0043] Among them, μ xis the mean of the vector or batch of input values ​​propagating through the activation layer at any one time, and ||x-μ x || is the L2 norm or variance. Therefore, the activation function preferably applies L2 normalization before activation. This normalization allows the activation to operate more symmetrically on the signal. In addition, L2 normalization has the advantage that the training converges more quickly.

[0044] According to a useful embodiment, the artificial neural network comprises at most three activation layers. In a useful embodiment, the activation layer is placed between two fully connected layers, preferably the activation layer is placed between each pair of fully connected layers, i.e. after each of the fully connected layers except the last one (which is preferably directly connected to the output layer).

[0045] In a useful embodiment, the beamformed output signals are used to reconstruct an ultrasound image of a field of view, wherein the signals, preferably time-aligned RF signals, are rearranged before applying the trained artificial neural network so that the RF signals (RF data) associated with one or at most a few pixels in the ultrasound image are processed by the NN in one or more batches. At the same time, if a vector is a set of values ​​fed into the input layer of the NN, a batch will be a sequence of such vectors that together include all the data required to reconstruct one (or several) pixels in the beamformed ultrasound image. In other words, the NN can be arranged to output the apodization weights pixel by pixel. To do this, in some embodiments, it is useful to rearrange the time-aligned RF signals so that the RF signals contributing to each pixel are placed behind each other so that each pixel can be processed independently. Each batch of data fed into the NN relates to one (or at most a few) specific pixels, wherein some parts of the RF signal can be duplicated in the rearrangement because it contributes to several pixels.

[0046] In a preferred embodiment, the RF signals associated with one single pixel of the ultrasound image are processed in one batch by the NN, wherein the number of input nodes corresponds to the number of RF signals contributing to the one pixel.

[0047] In another embodiment, the NN is extended to increase its field of view by also incorporating adjacent pixels into the beamforming process. Thus, RF data related to several adjacent pixels (e.g., up to 9) are processed in one batch by the neural network. Thus, the number of input nodes of the NN is increased. This embodiment improves clutter suppression, thereby further increasing image quality, however, at the expense of higher hardware requirements due to the increased network size. Several adjacent pixels can be adjacent to each other in any direction, i.e., within the plane of the ultrasound image or across the image depth.

[0048] In another embodiment, the artificial NN comprises at least one convolutional layer and / or one or several fully connected layers. Such convolutional layers may be applied across the time aligned RF signal, or may be applied across the input space. A convolutional neural network typically comprises at least a convolutional layer and a pooling layer, wherein a kernel is applied to the convolutional layer. The kernel operation is performed on a subset of the nodes of the convolutional layer, and the output is propagated to a node in the pooling layer. Typically, within a convolutional neural network, the nodes of the convolutional layer may be considered to be arranged as a d-dimensional matrix, such as a d-dimensional image. The advantage of using convolutional layers is that, in particular, local correlations of the input values ​​can be exploited by local connectivity patterns between nodes or layers, in particular by each node being connected only to a small region of nodes of the previous layer.

[0049] Thus, the use of convolutional layers reduces the amount of trainable parameters and speeds up both training and prediction. In a useful embodiment, one of the fully connected layers is replaced by a convolutional layer (and possibly a subsequent pooling layer). In another useful embodiment, all hidden layers are convolutional layers (possibly followed by a pooling layer).

[0050] In another useful embodiment, the NN is a recurrent neural network (RNN) or is part of a recurrent neural network (RNN). In a recurrent network, the feedback creates a memory function, giving the network memory properties. Such a memory function can be useful in processing sequences of related data, such as real-time ultrasound images, where the transducer moves slowly over the imaged body part so that the field of view of the ultrasound image changes incompletely from one image to the next. In a useful embodiment, the RNN has several layers, where at least one of the hidden layers is connected to a set of background nodes using a fixed weight of 1. At each time step, the input is fed forward through the layers. Fixed back connections store copies of previous values ​​of hidden nodes in background nodes. Thus, the network can maintain a state that allows it to perform tasks such as sequence prediction that exceed the capabilities of standard feedforward NNs.

[0051] According to a useful embodiment, some or all of the weights of the artificial neural network are quantized, in particular to 1 to 4 bits. Thus, the weights are rounded to one of a limited number of possible values, for example, one of 2 values ​​(0 and 1), or one of 4 or 8 values. Thereby, the memory and processing requirements of the NN are reduced. In a useful example, the weights are quantized to 1 bit so that they can only take binary values. This embodiment is very advantageous for memory-constrained implementations such as on a field programmable gate array (FPGA). Surprisingly, it has been shown that good results can be obtained even if the weights are therefore limited in terms of memory size.

[0052] In useful embodiments, the trained neural network of the present invention may be part of a larger neural network that completely processes the RF signals acquired from the ultrasound transducer, for example to ultimately generate a reconstructed image or provide analytical data on the image. Preferably, the trained neural network of the present invention is part of a larger neural network that provides apodization weights useful in content adaptive beamforming.

[0053] According to a useful embodiment, the trained neural network of the present invention comprises at least one hidden layer having fewer nodes than the input layer and / or output layer of the artificial neural network, for example 2-32 times, preferably 22 to 24 times fewer nodes. In a useful embodiment, all hidden fully connected layers have fewer nodes than the input and output layers. This dimensionality reduction forces the network to find a more compact representation of the data which helps with noise suppression. For example, the first hidden fully connected layer may have 2-16 times, preferably 4-8 times fewer nodes than the input layer.

[0054] According to another aspect, the present invention provides a method for providing a trained artificial neural network useful in content adaptive beamforming of ultrasound signals, the method comprising: (a) receiving input training data, i.e., RF signals acquired by a plurality of ultrasound transducer elements in response to ultrasound transmission, e.g., time-aligned RF signals; (b) receiving output training data, wherein the output training data are content adaptive apodization weights, wherein such content adaptive apodization weights have been calculated from the RF signals by a content adaptive beamforming algorithm, in particular a minimum square error algorithm, or wherein the output training data are beamformed output signals calculated from the RF signals by a content adaptive beamforming algorithm; (c) training an artificial neural network by using the input training data and the output training data; and (d) providing the trained artificial neural network.

[0055] Therefore, in a first embodiment, the NN is trained using beamformed RF signals as training targets. These target signals should represent the desired quality and characteristics of the signals produced when the trained NN is used (e.g., having high spatial resolution, minimal clutter, or maximum contrast). These target signals may originate from numerical simulations that provide input signals and associated beamformed output signals. Alternatively or additionally, these input signals may originate from captured RF signals that have been processed using a beamforming algorithm that is known to provide beamformed output signals with the desired quality and known desired characteristics. The beamforming algorithm may be a content adaptive beamforming algorithm, in particular a minimum variance beamforming algorithm. The NN is then arranged to calculate the apodization weights in a content adaptive manner. In this embodiment, a constraint that the apodization weights add up to 1 is preferably used during training, so that any deviation from 1 is penalized by adding to the cost function.

[0056] In a second embodiment, the NN is trained by providing content-adaptive apodization weights as training targets, which have been calculated using known content-adaptive beamforming algorithms, such as a minimum variance algorithm. These known techniques are typically data-dependent beamforming methods that attempt to adaptively process the available channel data in order to suppress off-axis clutter and / or produce a lateral resolution beyond the diffraction limit. Other known adaptive beamforming techniques include nonlinear inversion schemes that implement sparsity in a specified domain (such as used in compressed sensing), amplitude and phase estimation (APES), and aperture domain model image reconstruction (ADMIRE). However, these algorithms are computationally very expensive and can therefore be used only on training data and not on actual real-time image data. For example, for minimum variance beamforming, matrix inversion must be performed for every pixel in the image.

[0057] As described above, the artificial neural network can be arranged, i.e. it can be a deep network with several layers, preferably up to four fully connected layers and up to three activation layers. The training can be done by back propagation. In this method, the time aligned RF signal is propagated through the NN using predetermined weights for the edges. The output is compared with the output training data using an error or cost function, the output of which is propagated back through the NN, thereby calculating the gradient to find the network weights (and possibly other parameters, such as bias) that produce the minimum error. This can be done by adjusting the weights of the edges and following negative gradients in the cost function. It has been found that according to the present invention, the weights of the neural network converge to a minimum very quickly, so that the NN can be trained on a very limited number of data related to only one or a few ultrasound images.

[0058] In a useful embodiment, the neural network uses a dropout layer during training. Thus, specific nodes / connections in the dropout layer are randomly selected and the values / weights are set to 0. For example, the dropout layer may have a predetermined percentage of dropped nodes, such as 30-80%, for example 50% of all nodes are dropped and their values / weights are set to 0. For the next back propagation of the training data, different sets of nodes in the dropout layer are set to 0. This produces noise during training, but has the advantage that the training converges to a useful minimum. Therefore, the NN is much better trainable when using dropout layers. In a useful embodiment, one, two or three fully connected layers are followed by dropout layers during training. By using dropout layers, the number of connections is reduced, and during each iteration of back propagation, only a sub-network is trained. However, the actual network in use is a full NN (without dropout layers).

[0059] It is further useful during training if the artificial neural network comprises at least one activation layer comprising an activation function using L2 normalization. This will cause the training to converge more quickly.

[0060] According to another embodiment, the apodization weights generated by the NN are constrained by implementing a cost function (also referred to as a loss function) that penalizes deviations from unity gain. Thus, the neural network generates apodization weights whose sum is close to 1. In a useful embodiment, this is not strictly enforced during training of the neural network, but any deviation from the sum of 1 is penalized by adding to the cost function.

[0061] Thus, the present invention advantageously provides a robust beamformer that produces an output signal that remains proportional to the input.

[0062] Preferably, the trained artificial neural network used in determining the content adaptive apodization weights is trained using the above method.More preferably, the trained artificial neural network of the inventive method is provided by the above training method.

[0063] The invention also relates to a computer program comprising instructions which, when the program is executed by a computing unit, cause the computing unit to perform the inventive method. This is true for both the adaptive beamforming method and the training method. Alternatively, the neural network may be implemented as hardware, for example with fixed connections on a chip or other processing unit. The computing unit that may perform the inventive method may be any processing unit, such as a CPU (central processing unit) or a GPU (graphics processing unit). The computing unit may be part of a computer, a cloud, a server, a mobile device (such as a laptop, a mobile phone, a smart phone), etc. In particular, the computing unit may be part of an ultrasound imaging system.

[0064] The invention also relates to a computer readable medium comprising instructions which, when executed by a computing unit, cause the computing unit to perform a method according to the invention. Such a computer readable medium may be any digital storage medium, such as a hard disk, a server, a cloud server, an optical or magnetic digital storage medium, a CD-ROM, an SSD card, an SD card, a DVD or a USB or other memory stick.

[0065] According to another aspect, the present invention also relates to a system for adaptive beamforming of an ultrasonic signal, the system comprising: a) a first interface configured to receive an RF signal collected by a plurality of ultrasonic transducer elements in response to ultrasonic transmission; b) a computing unit configured to apply a trained artificial neural network to the RF signal, wherein content adaptive apodization weights for beamforming the RF signal are generated, and the computing unit is used to apply the content adaptive apodization weights to the RF signal to calculate a beamforming output signal; c) a second interface configured to output the beamforming output signal.

[0066] The system is preferably configured to perform the inventive method for adaptive beamforming of ultrasound signals. Such a system may be implemented on an ultrasound imaging system, for example, it is implemented on one of its processing units, such as a GPU. However, it is also conceivable that the RF signal acquired by the ultrasound transducer of the ultrasound imaging system is transmitted locally or remotely, for example via the Internet, to another computing unit, and the adaptive apodization weights are transmitted back from there and used in computing the beamforming output signal.

[0067] Furthermore, the present invention also relates to a system for training an artificial neural network by means of the inventive training method.

[0068] According to another aspect, the present invention relates to an ultrasound imaging system, comprising: an ultrasound transducer array, configured to transmit and receive ultrasound signals; a plurality of channels, operatively coupled to the ultrasound transducer array, configured to transmit channel data, also referred to as RF signals, based at least in part on received ultrasound signals; a beamformer, operatively coupled to the plurality of channels, wherein the beamformer is configured to time-align the channel data; and a signal processor, operatively coupled to the beamformer, wherein the signal processor is configured to perform the method according to any one of claims 1 to 12. Due to the low computational cost of the method, such a signal processor can be integrated into existing ultrasound systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The useful embodiments of the present invention should now be described with reference to the accompanying drawings. Similar elements or features are designated by the same reference numerals in the drawings. In the drawings:

[0070] Figure 1 is a schematic illustration of a conventional DAS beamforming technique.

[0071] Figure 2 A schematic overview of an adaptive beamformer is depicted.

[0072] Figure 3 is a simplified illustration of a neural network according to an embodiment of the present invention.

[0073] Figure 4 is a schematic overview of an implementation of an embodiment of the inventive method.

[0074] Figure 5 A schematic representation of an artificial neural network according to an embodiment of the invention is shown.

[0075] Figure 6 Ultrasound images obtained using (A) conventional DAS beamforming, (B) minimum variance beamforming, and (C) deep learning-based beamformer according to an embodiment of the present invention are shown.

[0076] Figure 7 An ultrasound imaging system according to an embodiment of the present invention is shown.

[0077] Figure 8 An overview of an alternative neural network-based beamforming approach is shown.

[0078] Fig. 9 A training data set from a simulated phantom used in an alternative method is shown, with original images on the left, images obtained by the alternative neural network method in the middle, and minimum variance beamformed images on the right.

[0079] Fig.10 A test data set from a simulated phantom belonging to an alternative method is shown, where the original image is shown on the left, the image obtained by the alternative neural network method is shown in the middle, and the minimum variance beamformed image is shown on the right. DETAILED DESCRIPTION

[0080] Figure 1Conventional beamforming using a delay-and-add (DAS) method is illustrated. In response to an ultrasound pulse, for example, transmitted by an array 4 of transducer elements, echoes 3 are reflected from a point structure (focus) 2 in the field of view. The echoes 3 are recorded by the array 4 of ultrasound transducers. Therefore, the acquired raw RF signals 5 are also referred to as channel data, each raw RF signal 5 having been acquired by one transducer element and therefore relating to one channel. Figure 1 The example in shows 8 channels. The channel data 5 are the flight times corrected in step 6 for beamforming, i.e., the different time shifts t1, t2, ... t2 at which the echo 3 is acquired by the array 4 are corrected according to the geometry of the transducer array 4 and the focus 2. n In step 7, these time-aligned RF signals S1 ... S n Then multiply by the apodization weights w1…w n In conventional DAS beamforming, these weights are preset and are not adapted to the content of the ultrasound image, i.e. they are not adapted to the RF signal. The weighted signals 8 are summed in step 9 to produce a beamformed output signal 10. The beamformed output signal 10 can be further processed to produce image data for one pixel.

[0081] Figure 2 An adaptive beamforming method is shown. In this aspect, the RF signals 18 S1, S2 ... S are time aligned. n The content adaptive apodization weights 12 are used by the beam imaging algorithm 14 to calculate the content adaptive apodization weights 12, which are therefore not predetermined for the DAS beamformer. Instead, the signal is processed by an adaptive beamformer 14 (e.g., a minimum variance beamformer) which calculates the optimal weights in order to maximize the image quality. The weighted RF signals are summed in step 9, which results in a beamformed output signal.

[0082] According to the present invention, the conventional adaptive beamforming algorithm / processor 14 is replaced by a neural network. Figure 3 An example of such a neural network 16 is shown. The example network is arranged into layers 20, 24, 26, 28, 32, 36, each layer comprising a plurality of nodes, wherein nodes between adjacent layers are connected by edges. Each edge / connection corresponds to a simple operation, which is performed on the value of the first node, and the value of the operation is added to the value of the connected node. In particular, a real or complex number may be assigned as a value to each node of the neural network.

[0083] The neural network 16 receives time-aligned RF signals 18 S1, S2, ..., S2 acquired from a plurality of ultrasound transducers. nAs input, and it is used to calculate a pixel. The number of nodes 21 in the input layer 20 corresponds to the number n of contributing RF signals. In this embodiment, the number n of nodes 34 of the output layer 36 corresponds to the number n of nodes 21 of the input layer 20. In order to calculate the content-adaptive apodization weights w1, ..., w n , input value signals S1, S2..., S n Propagation through neural networks.

[0084] In this embodiment, the input layer 20 is a fully connected layer, i.e., each node 21 in the input layer is connected to each node 23 in the next layer 24 by an edge 22. This operation corresponds to a matrix multiplication, in which each value of the input layer 20 is multiplied by the weight of the edge connecting it to the node 23 in the next layer 24.

[0085] The next layer is an activation layer 24, in this example, an anti-rectifier layer. The anti-rectifier effectively introduces nonlinearity while preserving negative signal components as well as the dynamic range of the input. Since it connects the positive and negative parts of the input, the number of nodes 25 in the following layer 26 is effectively doubled, because each node 23 has a different output depending on whether it has a positive or negative input value, as illustrated by the two edges 24a and 24b. Otherwise, the structure of the nodes 25 contained in the following layer 26 is equivalent to the structure of the nodes 23 in the activation layer 24, that is, there is no interconnection between adjacent nodes 23 in the layer 24.

[0086] The layer 26 following the activation layer 24 is again a fully connected layer, i.e., each node 25 in this layer is connected to each node 27 in the following layer 28. The following layer 28 has significantly fewer nodes 27 than the previous layer 26. By reducing the number of nodes, the number of parameters / weights that need to be trained is reduced. This leads to a reduction in the amount of computation in the network and control of overfitting. For example, there can be a dimensionality reduction of 3-6 times, in the example shown, the factor is 3, i.e., layer 28 has one third the size of the previous layer 26. In a useful embodiment, the factor will be 5. Layer 28 is again an activation layer, i.e., an inverse rectifier layer, which combines a sample-by-sample L2 normalization with two ReLU activations, thereby concatenating the positive and negative parts of the input. This results in a doubling of the number of nodes 29 in the next layer 32. This layer 32 is again a fully connected layer, since each node in layer 32 is connected to each node 34 in the output layer 36. The values ​​output at the output layer 36 are the content-adaptive apodization weights w1, ..., w n .

[0087] exist Figure 3In the embodiment of , the neural network has three fully connected layers (output layer 36 is not counted as one because it does not propagate any values ​​to the following layers), where the fully connected layers are named 20, 26, and 32. In addition, the network has two activation layers 23 and 28, each between pairs of fully connected layers. In other useful embodiments, there can be another fully connected layer followed by an activation layer, i.e., a total of 4 fully connected layers and three activation layers.

[0088] exist Figure 4 , a schematic overview of a possible implementation of the inventive method is given. At 40, a (raw) RF signal is illustrated as input data, wherein the data comprises a plurality of channels, each channel having a plurality of axial samples. In step 42, the raw RF data is time aligned using conventional methods, wherein different planes represent data of different channels. In this embodiment, the time-of-flight correction of the RF signal is calculated in advance and stored in a buffer. Alternatively, the time-of-flight correction can also be calculated in the GPU on the fly, thereby reducing communication and memory overhead. In addition, in this embodiment, in step 44, all data 43 from various channels related to one pixel are rearranged into a new format 45, so that the data 43 for each pixel can be processed as a single batch in the NN. The next step of applying the NN 16 to the time-aligned and rearranged RF signal 45 is shown at 46. At 50, a skip connection 48 is added from the input (time-aligned RF signal) to the output, wherein the time-aligned RF signal is multiplied by the apodization weights generated by the NN in step 52. The result is beamformed RF data 55 related to one pixel 54, which is used to reconstruct an ultrasound image 51. After beamforming by the NN, the beamformed pixels are rearranged according to their spatial locations.

[0089] Figure 5 The neural network 16 of the preferred embodiment is shown in more detail. Above each layer, its output size (for 128 contributing RF signals) is indicated. Fully connected layers are illustrated by dark shading, derectifier layers are illustrated in white, and dropout layers (which are present only during training of the network) are illustrated by light shading. The NN 16 includes four fully connected layers, including 128 nodes for the input and output layers, and 32 nodes for the inner layers. This dimensionality reduction 58 of 8 (23) times or 4 times relative to the input layer following the first derectifier layer forces the network to find a more compact representation of the data. Each of the fully connected layers (except the last layer) is followed by a derectifier layer. The last fully connected layer 60 is either the output layer or is directly connected to the output layer (not shown).

[0090] During training, dropout is applied between each pair of fully connected layers, for example, with a probability of 0.2. In other words, a fixed percentage of the nodes in the dropout layer are dropped during training. Thus, the dropout layer exists only during training the network. Dropout helps reduce overfitting of the neural network to the training data.

[0091] The NN can be implemented in Python using the Keras API with a TensorFlow (Google, CA, USA) backend. For training, the Adam optimizer is used with a learning rate of 0.001 that is speculatively optimized across a batch of pixels belonging to a single image. Figure 5 The neural network shown is implemented and trained on in vivo ultrasound image data.

[0092] When training Figure 5 When the neural network 16 is shown, the apodization weights calculated by the known adaptive beamforming technique using conventional algorithms can be used as output training data. The input training data is the corresponding time-aligned RF signal. During training, the NN 16 is applied to the training input data to generate calculated output data. The comparison between the calculated output data and the output training data is used to recursively adjust the weights within the neural network 16, in this case at a learning rate of, for example, 0.0005 to 0.01. In order to prevent overfitting, artificially calculated data or a regularization method based on normalized weight decay, i.e., the dropping of nodes, can be used.

[0093] Figure 5 The NN is tested on images acquired using a single plane wave transmission, and Figure 6 The results are shown. The image designated A shows an image reconstructed using a DAS beamformer. Image B uses a minimum variance beamformer, and in Figure C, a deep learning based beamformer according to an embodiment of the present invention is applied. It can be observed that the NN beamformer is able to generate high contrast images comparable to MV targets with significantly less clutter. In addition, both adaptive techniques show an increase in CNR (contrast to noise ratio) and resolution compared to DAS, wherein the NN even outperforms the MV target on the latter, possibly due to its ability to incorporate generative priors into the beamforming process by averaging the statistics of the training data. Training on higher quality images allows to improve the performance of the NN even further.

[0094] The method according to this embodiment of the invention was also tested on simulated images to compare resolution and contrast. Resolution was assessed by evaluating the average full width at half maximum (FWHM) of all point scatterers. Contrast was estimated using the average CNR of anechoic cysts. The results are shown in Table 1. Therefore, the NN beamformer is able to generate high contrast images with significantly less clutter than MV targets.

[0095] parameter DAS NN MV <![CDATA[FWHM lat (mm)]]> 0.846 0.704 0.778 <![CDATA[FWHM ax (mm)]]> 0.431 0.342 0.434 CNR(dB) 10.96 11.48 12.45

[0096] Table 1: Resolution and contrast metrics

[0097] Figure 7 1 is a schematic representation of an ultrasound system 100 according to an embodiment of the present invention and configured to perform the inventive method. The ultrasound system 100 includes a general ultrasound hardware unit 102, including a CPU 104, a GPU 106, and a digital storage medium 108, such as a hard disk or a solid state disk. The computer program can be loaded into the hardware unit from a CD-ROM 110 or via the Internet 112. The hardware unit 102 is connected to a user interface 114, which includes a keyboard 116 and optionally a touch pad 118. The touch pad 118 can also serve as a display device for displaying imaging parameters. The hardware unit 102 is connected to an ultrasound probe 120, which includes an array of ultrasound transducers 122, which allows live ultrasound images to be acquired from an object or patient (not shown). A live image 124 acquired by ultrasound probe 120 and beamformed according to the inventive method executed by CPU 104 and / or GPU is displayed on screen 126, which may be any commercially available display unit, such as a screen, television, flat screen, projector, etc.

[0098] Additionally, there may be a connection to a remote computer or server 128, for example via the Internet 112. The method according to the invention may be performed by the CPU 104 or GPU 106 of the hardware unit 102, but may also be performed by a processor of a remote server 128.

[0099] Figures 8 to 10 Concerning an alternative aspect of the invention, in which a NN is not used to compute apodization weights, but outputs adaptive beamforming RF data. The goal of this alternative aspect can be described as more accurately learning the behavior of a given adaptive beamforming (BF) algorithm by learning the actual mathematical operations of the adaptive BF algorithm involving per-channel data (i.e., RF signals) and beamformed RF data rather than just learning the effects of adaptive beamforming purely in the image domain. This alternative aspect provides a machine learning framework involving a multi-layer perception (MLP) to learn computationally expensive adaptive beamforming algorithms, such as MV or other techniques described above. However, as opposed to a mapping between pixel values ​​from the original and adaptive beamformed images, the machine learning framework is intended to learn a mapping between quasi-complex per-channel data and adaptive beamformed RF data.

[0100] MLP is a feed-forward artificial neural network that takes an input data set and maps it to an appropriate output set. MLP consists of multiple layers of neurons with nonlinear activation functions, where each layer is fully connected to the next. It has been shown previously that for any continuous mapping y=f(x1,x2,..,x n ) The maximum number of layers required is 3, which has an input layer, a hidden layer, and an output layer. A 3-layer MLP (or equivalently a 1-hidden-layer MLP) is a function f:R n →R l , where n is the size of the input vector x and l is the size of the output vector f(x), such that in matrix notation:

[0101]

[0102] Among them, b (1) and b (2) is the bias vector, W (1) and W (2) is the weight matrix, and G and s are activation functions. Common activation functions take the form of sigmoid functions:

[0103]

[0104] where λ determines the slope of the transition from 0 to 1. The weight matrix W (1) and W (2) Computed using a training algorithm, such as the Levenberg-Marquardt method or the back-propagation algorithm.

[0105] The neural network used in this alternative aspect is first trained to learn an adaptive beamforming algorithm based on a training data set that has been generated by Field II simulations, and then the trained neural network is applied to two different test data sets to prove the concept. The alternative aspect is a framework that can be generalized to many other computationally expensive adaptive beamforming techniques as long as a sufficient amount of input-output data pairs are available.

[0106] An alternative aspect provides a machine learning framework that can learn a computationally expensive adaptive beamforming algorithm from a limited amount of training data sets and apply the learned algorithm on a new data set via inference at significantly lower computational cost. An alternative aspect of the invention can be an enabler of real-time processing of computationally expensive adaptive beamforming algorithms that are otherwise very difficult to run in real-time, sometimes even utilizing GPUs at above average price points.

[0107] The main element of the alternative aspect of the present invention is a neural network that maps time-aligned complex per-channel RF data to complex beamformed RF data. Although several types of neural networks (such as multi-layer perceptrons (MLPs), convolutional neural networks (CNNs), and more advanced networks such as regression and / or generative networks) can be used to perform similar tasks, the alternative aspect uses an MLP model to demonstrate the feasibility of using a machine learning / deep learning framework to learn and apply advanced adaptive beamforming techniques. The MV beamformer is used as a test algorithm, but the core concepts presented here can also be extended to other adaptive beamforming algorithms.

[0108] The input-output pairs used in training the neural network in an alternative aspect of the invention are not pixels from the original and MV beamformer images, but rather the input data comprises time-aligned complex channel RF signals at a given depth, and the output data is the corresponding complex beamformer output for the MV beamformer.

[0109] Figure 8 The main steps are illustrated in . In step 1, a training data set is prepared for neural network training. It includes time-aligned complex per-channel data as input and complex MV beamformed RF data as target. For an N-channel system, M input-output pairs are obtained from 2N input per-channel data (real and imaginary) and corresponding MV beamformer outputs (real and imaginary) at a given pixel position. Both input and output data have real and imaginary parts. Therefore, the input data matrix is ​​M×2N, and the output data matrix is ​​M×2. MV beamforming should be performed offline to obtain such input-output pairs. Since MV beamforming is performed only once at the data preparation stage, the computational burden associated with it is not a limitation of the method.

[0110] In step 2, the training data set is used to train the learning algorithm. This step is performed iteratively until the mapping error converges to a certain pre-specified level. The MLP model is used to prove the concept. However, a more advanced network architecture involving a convolutional neural network can be used. This will be described in more detail as an embodiment later.

[0111] In step 3, a test data set in the form of time-aligned complex per-channel data that has not been observed before by the learning algorithm is introduced. The trained algorithm operates on the input data to predict (or infer) its complex MV beamformer output. Since its approximation is computationally intensive in the MVBF using only additions and multiplications, the inference step is expected to be significantly faster than direct calculation of the MVBF. For example, the computational complexity associated with the standard DAS is linear in the number of elements, O(N). However, the computational complexity for the MVBF scales with the subarray size, L, and becomes O(L) due to the matrix operations required to calculate the optimal aperture weights. 3). However, using MLP, the added computational burden can be significantly reduced, potentially making it more feasible for real-time processing.

[0112] Fig. 9 and Fig.10 Some preliminary simulation results are provided in: Fig. 9 A training data set from a simulated phantom containing a single large anechoic cyst is shown, showing the original image (left), the neural network image (center), and the real MVBF image (right). A 64-element P4-2 phased array was simulated. All images are pre-scan converted images, and all images are displayed over a 60dB dynamic range. Note that the MVBF image exhibits a finer spot size and a reduced number of sidelobes within the cyst. The neural network image also shows reduced sidelobes in the cyst and a slightly smaller spot size.

[0113] Fig.10 A test data set from a simulated phantom containing 3 small anechoic cysts is shown. The original image (left), the neural network image (middle), and the true MVBF image (right) are shown. A 64-element P4-2 phased array was simulated. All images are pre-scan converted images, and all images are displayed over a 60dB dynamic range. Note that the MVBF image exhibits finer spot sizes and reduced sidelobes within the anechoic cysts. The neural network image also shows similar improvements.

[0114] Other network architectures can be used to learn adaptive beamforming algorithms. The key component is that the network maps from per-channel inputs to beamformed outputs. For example, convolutional neural networks are expected to give good results. The input data is aligned real (or complex) per-channel data. Processing can be local (learning one pixel value from the relevant per-channel data) or global (learning all beamformed RF frames from the entire aligned data stack). Local processing seems suitable for emulating algorithms whose input data is local anyway (such as minimum variance beamformers). Global algorithms also have the potential to learn and use anatomical information if enough data is provided.

[0115] Consistent with the basic principles of this alternative aspect of the invention, a local approach to convolutional neural networks is described below. The aligned per-channel data for each pixel is quickly cropped around the sample depth of interest, producing a (numTime*numElements) data matrix. The time dimension is typically a few wavelengths that are sensitive to steering effects. The training data set size is determined by the number of such data windows in the number of available images. A single image can typically produce thousands of independent training input-output pairs. A fully convolutional neural network with a receptive field spanning all input data and outputting a single scalar can be trained to learn adaptive beamforming values ​​at the depth of interest.

[0116] The above discussion is intended to be merely illustrative of the present system and should not be construed as limiting the claims to any particular embodiment or group of embodiments. Thus, although the present system has been described in particular detail with reference to exemplary embodiments, it should also be appreciated that many modifications and alternative embodiments may be envisioned by one of ordinary skill in the art without departing from the broader and intended spirit and scope of the present system as set forth in the claims below. Accordingly, the specification and drawings are to be viewed in an illustrative manner and are not intended to limit the scope of the claims.

Claims

1. A method for adaptively beamforming an ultrasonic signal, the method comprising the following steps: a) receiving RF signals (18) acquired by a plurality of ultrasonic transducer elements (4) in response to ultrasonic transmission; b) determining content adaptive apodization weights (12) for beamforming the RF signal (18) by applying a trained artificial neural network (16) to the RF signal.

2. The method according to claim 1, wherein: The number of input nodes (21) and the number of output nodes (34) of the trained artificial neural network (16) correspond to the number of contributing RF signals.

3. The method according to claim 1 or 2, comprising the following additional steps: c) applying the content adaptive apodization weights (12) to the RF signal (18) to calculate a beamformed output signal.

4. The method according to claim 1 or 2, wherein: The trained artificial neural network (16) includes at least one activation layer (24, 28) including an activation function that propagates both positive and negative input values ​​with unbounded output values.

5. The method according to claim 1 or 2, wherein: The trained artificial neural network (16) includes at least one activation layer (24, 28) including an activation function that concatenates positive and negative parts of an input value.

6. The method according to claim 1 or 2, wherein: The trained artificial neural network (16) includes at most four fully connected layers (20, 26, 32).

7. The method according to claim 1 or 2, wherein: The trained artificial neural network (16) comprises at most three activation layers (24, 28).

8. The method according to claim 3, wherein: The beamformed output signals are used to reconstruct an ultrasound image of a field of view, and wherein the RF signals (18) are rearranged (44) prior to applying the trained artificial neural network (16) so that RF data relating to one or at most several pixels (54) of the ultrasound image (51) are processed in one or more batches by the trained artificial neural network (16).

9. The method according to any one of claims 1, 2 and 8, wherein: The trained artificial neural network (16) comprises at least one convolutional layer and / or one or several fully connected layers.

10. The method according to any one of claims 1, 2 and 8, wherein: The trained artificial neural network (16) is part of a recurrent neural network.

11. The method according to any one of claims 1, 2 and 8, wherein: Some or all of the weights of the trained artificial neural network (16) are quantized, in particular to 1 to 4 bits.

12. The method according to any one of claims 1, 2 and 8, wherein: The trained artificial neural network (16) comprises at least one hidden layer (28) having fewer nodes (27) than an input layer (20) and / or an output layer (36) of the trained artificial neural network (16).

13. A method for providing a trained artificial neural network (16) useful in content adaptive beamforming of ultrasound signals, the method comprising: (a) receiving input training data, the input training data being RF signals (18) acquired by a plurality of ultrasound transducer elements (4) in response to ultrasound transmission, (b) receiving output training data, wherein the output training data are content adaptive apodization weights (12), wherein such content adaptive apodization weights have been calculated from the RF signal (18) by a content adaptive beamforming algorithm (14), in particular a minimum square error algorithm; (c) training an artificial neural network (16) by using the input training data and the output training data; (d) providing the trained artificial neural network (16).

14. A computer program (110) comprising instructions which, when executed by a computing unit (104, 106), cause the computing unit to carry out the method according to one of claims 1 to 13.

15. A system (102) for adaptive beamforming of an ultrasound signal, the system comprising: a) a first interface configured to receive RF signals (18) acquired by a plurality of ultrasound transducer elements (4) in response to ultrasound transmission; b) a computing unit (104, 106) configured to: - applying a trained artificial neural network (16) to the RF signal, thereby generating content-adaptive apodization weights (12) for beamforming the RF signal (18), and - applying the content adaptive apodization weights (12) to the RF signal to calculate a beamformed output signal; c) A second interface configured to output the beamformed output signal.

Citation Information

Patent Citations

  • Ultrasound imaging system with a neural network for image formation and tissue characterization

    WO2018127498A1

  • Neural network method and apparatus

    CN107665364A

  • A neural network and signal processing units

    EP0575716A1

  • Method of training a neural network and a neural network trained according to the method

    US20050149463A1

  • Method and apparatus for filling doppler signal gaps in ultrasound diagnostic imaging

    US20090012398A1

Cited By

  • Secondary ultrasonic imaging system and method capable of automatically highlighting interest domain

    CN120114091A