Ultrasonic image reconstruction method, system, device and medium

Abstract

The present invention relates to an ultrasonic image reconstruction method, system, device and medium, including establishing a measurement model and / or measurement matrix describing the relationship between RF channel data received by an ultrasonic system and data after beam synthesis to be obtained; obtaining RF channel data reflecting the acoustic characteristics of the target by transmitting ultrasonic waves to an imaging target and receiving echo signals; obtaining high-quality beam synthesis data based on the RF channel data and the measurement matrix and a pre-trained neural network for solving an inverse beam synthesis problem, and completing ultrasonic image reconstruction. Therefore, the present invention can significantly shorten the reconstruction time of a single image, so that multiple images can be reconstructed in a short time and coherently compounded, further improving image quality.

Description

Technical Field

[0001] The present invention relates to an ultrasonic image reconstruction method, system, equipment and medium, and relates to the technical field of image processing. Background Art

[0002] In PWI (Plane wave imaging), all array elements of the linear array probe are excited simultaneously, thereby emitting ultrasonic signals with plane wavefronts to explore the entire imaging area. Since the acoustic beam is not focused during the PWI transmission process, the signal-to-noise ratio of the echo signal is usually low. In addition, the DAS method commonly used in ultrasonic beam synthesis is designed for the situation where the echo signals received by each channel in focused wave imaging all come from a certain focal line. When the DAS method is applied to PWI, a large amount of off-axis scattered signals caused by the non-focused transmission beam will not be effectively suppressed. These two points together lead to the fact that the image quality of PWI is usually poor. Plane wave coherent compounding technology can significantly improve the image quality by superimposing the low-quality images reconstructed after PW transmission at different angles. The study of Montaldo et al. showed that after completing 71 angles of plane wave transmission, the image quality obtained by coherent compounding can even exceed that of multi-focus focused transmission imaging. However, dozens of transmissions will sacrifice the advantage of PWI's high frame rate and further affect its capture of instantaneous / fast events.

[0003] In order to improve the image quality of PWI without sacrificing the frame rate of PWI, various advanced beamforming methods have been introduced into PWI in the existing technology. The SR (Sparse regularization) method can improve the spatial resolution and contrast of PWI imaging and effectively suppress the sidelobe and grating lobe artifacts in the image, thereby reconstructing higher quality B-mode images. However, there are two major problems with the current SR-based beamforming method: (1) it takes up a lot of memory space to store the huge measurement model; (2) it requires an iterative convex optimization algorithm for time-consuming solution. For a real-time imaging technology such as ultrasound imaging, the reconstruction time of the traditional SR method is unacceptable. As ultrasound imaging moves further towards the era of ultra-fast imaging based on PWI, the massive amount of data collected in a short period of time puts higher requirements on the computational efficiency of the beamforming method.

[0004] Therefore, under the premise of retaining (or even improving) the quality of the reconstructed image by the SR method, significantly improving the reconstruction speed is the key to helping it move towards practical application. Although introducing more efficient convex optimization algorithms or using hardware platforms with higher computing power can have a certain acceleration effect, they cannot fundamentally solve the problem of high computational complexity of the SR method. Introducing deep learning into the solution of the inverse problem of beamforming and using its efficient and high-quality nonlinear mapping to replace the time-consuming iterative solution process in the traditional SR method is a very potential way to improve the speed of solving the inverse problem of beamforming.

[0005] When using traditional supervised learning strategies to train deep neural networks, it is necessary to provide the true solution of the inverse problem of beamforming (i.e., the true tissue scattering intensity distribution) as a training label. However, this true value cannot be obtained under in vivo conditions. If a large number of images are first reconstructed using traditional SR methods and then used as network training labels, the following problems will also be encountered: (1) Traditional SR methods take several minutes to reconstruct a frame of image, so generating a training set with a large number of samples will take a huge amount of time and computational cost; (2) When reconstructing images, traditional SR methods need to adjust parameters such as the weight of the regularization constraint, the allowable error, and the maximum number of iterations based on the characteristics of the target to be reconstructed. When the above parameters are not adjusted to the optimal level, the poor quality training data generated will limit the performance of the trained network. Summary of the invention

[0006] In view of the above problems, the object of the present invention is to provide an ultrasonic image reconstruction method, system, device and medium that can significantly shorten the reconstruction time of a single-angle image, reconstruct multi-angle plane wave images in a short time and perform coherent compounding, thereby further improving the image quality.

[0007] In a first aspect, the present invention provides an ultrasound image reconstruction method, comprising:

[0008] Establishing a measurement model and / or measurement matrix describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beam synthesis;

[0009] By transmitting ultrasonic waves to an imaging target and receiving echo signals, RF channel data reflecting the acoustic characteristics of the target are obtained;

[0010] Based on the RF channel data and the measurement model and / or the measurement matrix, high-quality beamforming data is obtained based on a pre-trained neural network that solves the inverse beamforming problem to complete ultrasound image reconstruction.

[0011] Furthermore, the measurement model describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beam synthesis includes a linear measurement model and a nonlinear measurement model.

[0012] Furthermore, the linear measurement model describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beamformation is:

[0013] y=Hx+n

[0014] in, is the measurement matrix, is additive Gaussian noise, r = D × K is the total number of sampling points of RF channel data, there are K channels in total, each channel contains D sampling points, l = N z ×N x is the total number of grid points of the data after beam synthesis, a total of N x imaging lines, each imaging line contains N z Pixels.

[0015] Furthermore, the step of establishing a measurement model and / or a measurement matrix includes:

[0016] 1) Place a point scattering sub-target in the imaging area;

[0017] 2) Obtaining ultrasound channel data;

[0018] 3) Calculate the intensity histogram and cumulative distribution function of the ultrasound channel data;

[0019] 4) Generate a binary mask based on the cumulative distribution function;

[0020] 5) Pair the mask with the point target and apply the mask to the channel data to extract the echo signal corresponding to the point scattering sub-target;

[0021] 6) Repeat steps 1)-5) until the echo signals corresponding to all grid points in the imaging area are completed;

[0022] 7) The echo signal of each grid point in the imaging area is sequentially rearranged into each column of the measurement matrix to obtain the measurement matrix.

[0023] Furthermore, the training of the neural network for solving the inverse beamforming problem includes:

[0024] Set the relevant parameters of the neural network;

[0025] Set the mean square error loss function;

[0026] According to the prior knowledge of ultrasonic signal characteristics, relevant constraint loss functions are set;

[0027] Based on the set neural network parameters, mean square error loss function and related constraint loss function, a self-supervised learning strategy is used to complete the training of the neural network for solving the inverse beamforming problem.

[0028] Furthermore, a related constraint loss function is set based on the prior knowledge of ultrasonic signal characteristics, including:

[0029] Sparsity constraint function in the wavelet domain;

[0030] Smoothness constraint function in the spatial domain;

[0031] Sparse constraint function for the signal envelope.

[0032] Furthermore, a self-supervised learning strategy is used to complete the training of the neural network for solving the inverse beamforming problem, including:

[0033] Training phase 1: initial solution is used as both input and label for the neural network and is solved by minimizing the initial solution and recovery solution The signal recovery network, also known as the autoencoder, is obtained by training the mean square error between the two networks.

[0034] Training stage 2: The high-quality solution x′ obtained by network reconstruction is used to recover the corresponding channel data y′=Hx′, and the network for solving the inverse beamforming problem is trained by minimizing the mean square error loss function between the measured channel data y and the restored channel data y′, and introducing the relevant constraint loss function. In each stage, the ultrasound channel data is first back-projected to generate the initial solution of the inverse beamforming problem. As the input of the network, it is then mapped to the target solution through the neural network.

[0035] In a second aspect, the present invention further provides an ultrasound image reconstruction system, the system comprising:

[0036] A first unit is configured to establish a measurement model and / or a measurement matrix describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beam synthesis;

[0037] A second unit is configured to obtain RF channel data reflecting acoustic characteristics of the target by transmitting ultrasonic waves to the imaging target and receiving echo signals;

[0038] The third unit is configured to obtain high-quality beamforming data based on the RF channel data and the measurement matrix and / or the measurement matrix and based on a pre-trained neural network for solving the inverse beamforming problem to complete ultrasound image reconstruction.

[0039] In a third aspect, the present invention further provides an electronic device, comprising computer program instructions, wherein the program instructions are used to implement the ultrasonic image reconstruction method when executed by a processor.

[0040] In a fourth aspect, the present invention further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the program instructions are used to implement the ultrasound image reconstruction method when executed by a processor.

[0041] The present invention adopts the above technical solution, and has the following characteristics:

[0042] 1. The self-supervised learning training strategy proposed in the present invention uses the RF (Radio Frequency) channel data collected by the ultrasound system as both the input and label of the network training process. This strategy bypasses the need for a true solution to the inverse problem in network training, so that the performance of the network will not be limited by the traditional algorithm used to generate training labels. Specifically, when using the self-supervised learning strategy, the output of the network is still the solution to the inverse problem of beamforming (tissue scattering intensity distribution, i.e., high-quality ultrasound images). However, unlike the supervised learning strategy that directly applies labels to the network output, the self-supervised strategy first restores the network output to RF channel data through a pre-established measurement model, and then calculates the error between the restored RF data and the actual RF data collected by the ultrasound system, and uses it to update the network.

[0043] 2. The self-supervised neural network of the present invention does not need to use a perfect "ultrasound image" (ie, tissue scattering intensity distribution map) as a gold standard or data label.

[0044] 3. The present invention proposes a general framework based on a self-supervised learning (SSL) scheme to train DNN to solve the inverse problem. In this way, during the training process of DNN, the measurement values ​​are used as both input and label. Compared with the traditional method, using the framework proposed by the present invention, the trained DNN can improve the reconstruction accuracy while reducing the calculation time.

[0045] In summary, the present invention can be widely used in image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings:

[0047] Figure 1 A method flow for establishing a measurement matrix based on the PSF of an ultrasound system according to an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of a two-stage self-supervised learning training strategy according to an embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of the structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0050] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0051] For ease of description, spatially relative terms may be used herein to describe the relationship of one element or feature relative to another element or feature as shown in the figures, such as "inside", "outside", "inner side", "outer side", "below", "above", etc. Such spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the figures.

[0052] The ultrasonic image reconstruction method, system, device and medium provided by the present invention include establishing a measurement model and / or measurement matrix describing the relationship between the RF channel data received by the ultrasonic system and the data after beam synthesis to be obtained; by transmitting ultrasonic waves to the imaging target and receiving echo signals, RF channel data reflecting the acoustic characteristics of the target is obtained; based on the RF channel data and the measurement model and / or measurement matrix, a neural network that is pre-trained to solve the inverse problem of beam synthesis is used to obtain high-quality beam synthesis data to complete ultrasonic image reconstruction. Therefore, the present invention can significantly shorten the reconstruction time of a single image, so that multiple images can be reconstructed in a short time and coherently compounded, further improving the image quality.

[0053] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0054] Embodiment 1: The ultrasonic image reconstruction method provided in this embodiment includes:

[0055] S1. Establish a measurement model. The measurement model includes a linear measurement model and a nonlinear measurement model. This embodiment is described using a linear measurement model. The implementation process of a nonlinear measurement model is basically similar and will not be described in detail.

[0056] Specifically, beamforming is a method for reconstructing ultrasound images. This embodiment reconstructs high-quality ultrasound images by solving the inverse problem of beamforming. The premise of solving the inverse problem is to establish a linear measurement model that describes the relationship between known measurement quantities and unknown quantities to be solved. When it comes to ultrasound beamforming, it is necessary to establish a linear measurement model that describes the RF channel data received by the ultrasound system. and the data after beam synthesis Linear measurement model between:

[0057] y=Hx+n (1)

[0058] in, is the measurement matrix, which describes the system response at each imaging grid point in space (i.e., the echo signal received by the ultrasound system after placing a scatterer at each grid point). is additive Gaussian noise, r = D × K is the total number of sampling points of RF channel data (there are K channels in total, each channel contains D sampling points), l = N z ×N x is the total number of grid points of the beamformed data (a total of N x imaging lines, each imaging line contains N z pixels).

[0059] S2. Obtaining a measurement model and / or a measurement matrix. It should be noted that when the measurement model is a linear measurement model, it can be equivalent to obtaining a measurement matrix. The obtaining process includes:

[0060] S21. Place point scattering sub-targets in the imaging area

[0061] Specifically, three point scattering sub-targets A, B, and C are placed on the grid divided in the imaging area, such as Figure 1 (a) shown.

[0062] Furthermore, the method of dividing the network in the imaging area is as follows: the width of the ultrasound probe is usually used as the lateral width of the imaging area, and within this lateral range, a lateral grid is divided with a spacing of half the width of the array element, that is, the number of lateral grid points will be twice the number of probe array elements; the division of the axial grid can be combined with the imaging depth and the center frequency of the ultrasound signal at the same time, and within the imaging depth range, the axial grid is divided with a spacing of one-quarter of the wavelength; it should be noted that the grid division method of this embodiment is only a preferred method given after comprehensive calculation complexity and image quality, but other grid division methods can be selected according to actual needs (for example, when convex array or phased array imaging, grid division can be performed in a fan-shaped area), where the lateral refers to the direction perpendicular to the central axis of the probe, and the same lateral position refers to the position with the same horizontal coordinate in the lateral direction.

[0063] Furthermore, it is first necessary to explain that this embodiment is provided with three point scattering sub-targets, but the actual number of point targets can be greater than or equal to one. The number of point target settings is taken as an example, but is not limited to this. A single or multiple points can be set according to actual needs. The method of placing three point scattering sub-targets A, B, and C is: if it is set in a simulation, it is only necessary to set the scattering intensity of a certain position on a given grid point to not be 0 according to the requirements of the simulation software; if it is set in an experiment, it is necessary to place nylon threads or hair threads in a water tank and let the probe scan the cross section of the thread. The thread type here is the preferred method, and it can also be implemented in other ways, which is not limited here.

[0064] S22, obtaining ultrasound channel data, such as Figure 1 (b) as shown.

[0065] Specifically, the ultrasonic channel data can be obtained by using K-wave simulation software, Field II simulation software or using a real system for actual measurement. For the real system, it is preferred to place nylon thread or the like in a water tank for the ultrasonic system to measure. It should be noted that the ultrasonic channel data of this embodiment is transmitted in parallel with the array direction of the ultrasonic probe as a zero-angle plane wave signal, but the actual measurement may not be a zero-angle plane wave signal, which is determined according to the specific situation.

[0066] S23, calculating the intensity histogram and cumulative distribution function of the ultrasound channel data, such as Figure 1 (c) and (d), where the cumulative distribution function is obtained by integrating the intensity histogram of the channel data.

[0067] S24. Generate a binary mask (Mask) based on the cumulative distribution function to retain only the channel data sampling points with the top 6% absolute intensity. The retention size range here is a better way and can be adjusted according to the compromise between accuracy requirements and memory usage size, such as Figure 1 (e) and the mask is smoothed by closing operation to obtain the smoothed mask as shown in Figure 1 (f) as shown.

[0068] S25, such as Figure 1 As shown in (g), each mask is paired with a point target and the echo signal corresponding to the point target is extracted by applying the mask to the channel data, as shown in Figure 1 (h) shown.

[0069] Specifically, the method for pairing each mask with a point target is: matching according to the spatial position, because the mask extracts the position of the ultrasonic echo signal, and the ultrasonic echo signal must be next to the corresponding point target.

[0070] Furthermore, the method of extracting the echo signal corresponding to the point target by applying the mask to the channel data is: performing point multiplication on the mask and the channel data to extract the corresponding echo signal.

[0071] S26, repeating steps S21-S25 until the echo signals corresponding to all grid points in the image area are measured, so that point targets can be placed on the grid points in sequence, and eventually every grid point on the imaging area grid is covered.

[0072] S27. Rearrange the echo signal of each grid point into each column of the measurement matrix H in sequence to obtain the measurement matrix.

[0073] S3. By transmitting ultrasonic waves to the imaging target and receiving echo signals, information reflecting the acoustic characteristics of the target can be obtained.

[0074] S4. Beamforming based on self-supervised deep learning

[0075] Specifically, after the linear measurement model y=Hx+n is established, the inverse problem solving and self-supervised deep learning methods can be introduced to solve the high-quality beamformed data x under the condition of known RF channel data and measurement matrix H (usually non-square and non-invertible). The process includes:

[0076] S41. Set the relevant parameters of the neural network

[0077] After determining the network training framework for solving the inverse problem, a (convolutional) neural network with appropriate parameters such as the number of layers and convolution kernel size can be designed for use according to specific problems (problem complexity, computing power, etc.).

[0078] This embodiment uses a 12-layer encoding-decoding network, in which the encoding part is used to extract the high-order features of the signal, and the decoding part is used to restore the high-order features to outputs with target characteristics and sizes. The detailed information of the network is shown in Tables 1 and 2.

[0079] Table 1 Neural network parameters

[0080]

[0081] Table 2 Neural network structure

[0082]

[0083] Furthermore, the activation function is not limited to Tanh, and ReLU or LeakyReLU may be used. In addition, maxpooling may not be used, but the stride of the convolution kernel may be set to 2 to achieve image downsampling. This is taken as an example and is not limited to this.

[0084] Furthermore, the neural network of this embodiment can also introduce residual blocks into the network based on the U-net framework to further improve the nonlinear mapping capability of the network.

[0085] S42. Set mean squared error (MSE)

[0086] In order to improve the training convergence speed of the neural network, the training of the neural network is divided into two stages. (obtained using the delay-and-add beamforming method or the backprojection algorithm) is used as both input and label of the network and is solved by minimizing the initial solution and the recovery solution of the network output The mean square error between the measured channel data y and the restored channel data y′ is minimized as shown in formula (3), and a network for solving the inverse problem of beamforming is trained.

[0087] The setting of the mean square error loss function is to evaluate the gap between the network output solution and the desired solution, and to guide the network parameter update. The setting of the mean square error here is the preferred method, and other loss functions for evaluating the error size can be selected according to the actual situation.

[0088]

[0089]

[0090] S43. Introduce a loss function based on signal prior related constraints to improve the quality of the reconstructed image.

[0091] The inverse problem of ultrasonic beam synthesis is ill-conditioned and underdetermined. Therefore, it is necessary to design a suitable regularization constraint function based on the prior knowledge of ultrasonic signal characteristics, and apply it to the neural network training to enhance the quality and stability of the solution. This embodiment applies the following three constraint loss functions:

[0092] 1) Sparsity constraint function in wavelet domain

[0093] Ultrasonic signals can usually be represented relatively sparsely in the wavelet domain. The present invention implements a sparse averaging algorithm with the help of the PyWavelets toolkit, thereby sparsely representing the high-quality solution output by the network. The sparse averaging operator consists of eight Daubechies (db) wavelet transforms (based on the db1 to db8 wavelet bases):

[0094]

[0095] Compared with single wavelet transform, sparse average operator has been shown to achieve better results in wavelet-based SR methods. The constraint term based on sparsity in the wavelet domain is expressed as follows:

[0096] L Sp-W =‖ψx′‖1 (5)

[0097] where ‖·‖1 represents the l1-norm of the vector.

[0098] 2) Smoothness constraint function in the spatial domain

[0099] Total variation (TV) constraints have the effect of enhancing the local smoothness of the image. This embodiment applies TV constraints to the high-quality solution x′ output by the network to enhance its smoothness and achieve a better balance between image smoothness (better preservation of background speckles) and sparseness (higher contrast):

[0100] L Sm =‖D1Env(x′)‖1+‖D2Env(x′)‖1 (6)

[0101] Among them, D1 and D2 are finite difference operators along the transverse and axial directions (vertical and along the direction of the ultrasonic beam), respectively. Env(·) is the Hilbert transform operator for extracting the signal envelope. In addition to the axial and transverse differences, the difference of pixels along a 45° angle can also be added. Since the difference operator is to calculate the difference between all adjacent elements of a matrix, where this difference operator can be divided into angles, for example, the transverse and axial difference operators are to calculate the difference between adjacent elements on the left and right or adjacent elements on the top and bottom. If it is a 45° difference, it is to calculate the difference between adjacent elements on the diagonal line.

[0102] 3) Sparse constraint function of signal envelope

[0103] Based on the prior knowledge of the differences in tissue components and their scattering intensities, the present invention imposes a sparse constraint based on the l1-norm on the envelope of the high-quality solution x' output by the network. This constraint will guide the network to suppress noise with smaller amplitude (such as side lobes or electronic noise in the echo-free area) while retaining signals with larger amplitude (such as scattered self-echoic tissue) to improve the contrast between different tissue components:

[0104] L Sp-E =‖Env(x′)‖1 (7)

[0105] In summary, the loss function in the present invention is the weighted sum of all the above constraints:

[0106]

[0107] Among them, coefficients λ1, λ2 and λ3 are the weights of the corresponding constraints.

[0108] S44. Train a network to solve the inverse beamforming problem

[0109] In order to solve the problems faced by traditional supervised learning methods, this embodiment proposes a self-supervised learning training strategy, which uses RF channel data as both input and label of the network during the training process, thereby bypassing the need for "ideal" ultrasound images in the training process.

[0110] like Figure 2 As shown, the training of the neural network in this embodiment is divided into two training stages. In each stage, the ultrasound channel data needs to be back-projected to generate an initial solution to the inverse beamforming problem. As the input of the network, it is then mapped to the target solution through the neural network.

[0111] The difference between the two training phases is that in training phase 1, the initial solution is used as both input and label for the network and is solved by minimizing the initial solution and recovery solution The mean square error between them is trained as shown in formula (2) to obtain a signal recovery network, namely an autoencoder (the output is approximately equal to the input).

[0112] In the training stage 2, the high-quality solution x′ obtained by the network reconstruction is used to recover the corresponding channel data y′=Hx′, and the network for solving the inverse beamforming problem is trained by minimizing the MSE between the measured channel data y and the restored channel data y′, as shown in formula (3). In this training stage, in addition to using the MSE loss function, additional constraint loss functions can be designed and introduced based on the prior knowledge of the signal to be reconstructed (data after beamforming) to further improve the quality of image reconstruction, such as formulas (3) and (5).

[0113] Among them, the initial parameters of the network training in training stage 2 are obtained by network transfer learning obtained by training in training stage 1, and the mean square error loss is suitable for solving the inverse problem when assuming that the noise obeys a Gaussian distribution.

[0114] Furthermore, as a means to significantly improve the quality of PWI images, if the frame rate allows, multi-angle coherent composite technology should be used as much as possible. When reconstructing PWI images at different angles, it is necessary to first establish a corresponding linear measurement model for the PW transmission at that angle. Since the mapping process of the neural network is independent of the measurement matrix, the reconstruction of images at different angles will be completed by a common neural network. In other words, the initial solutions obtained by the back-projection method after PW transmission at different angles will be sent to the same network in turn, and their corresponding outputs (high-quality solutions) will be coherently composited to obtain the final image.

[0115] The ultrasonic image reconstruction method of the present invention is described in detail below through specific embodiments.

[0116] In this embodiment, the measurement matrix is ​​established, the simulation data set is generated, and the quantitative evaluation is performed in Matlab 2015. The training and testing of the neural network are based on Python 3.5.2 and Pytorch 1.2.0. The hardware platform used in this embodiment has an Intel Xeon E5-2620 CPU, 128GB memory, and 4 NVIDIA Titan V graphics cards with 12GB video memory. This embodiment uses an L10-5 linear array probe to collect experimental data, and its parameters are shown in Table 3. The probe simulated in the computer simulation also has the same parameters. After full-aperture PW transmission and reception, the RF channel data is recorded at a sampling frequency of 20.832MHz.

[0117] Table 3 Parameters of the linear array probe used in PWI

[0118]

[0119] This embodiment makes a qualitative and quantitative comparison of the images reconstructed by this method (single angle, 11 angles and 75 angles emission) with the traditional DAS method (single angle, 11 angles and 75 angles emission; the trace window is 25% Tukey, F# is 1.75) and the traditional SR method (single angle emission; solved using the SPGL1 toolkit, with a maximum number of iterations of 50). The images after beam synthesis by all methods contain 256 imaging lines, each of which contains 1300 sampling points in the depth direction. The present invention demonstrates the quality and robustness of the images reconstructed by the trained network through sufficient simulation, phantom and in vivo experiments. Compared with the traditional SR method, the use of neural network solution will also have two main advantages: (1) The neural network learns from a large amount of training data, and there is no need to manually adjust the parameters when reconstructing each image like the SR method, and better reconstruction effect will be achieved; (2) Because the deep learning technology greatly shortens the reconstruction time of single-angle images, it is possible to reconstruct multi-angle plane wave images and perform coherent recombination in a short time, thereby further improving the image quality.

[0120] Embodiment 2: Embodiment 1 above provides an ultrasonic image reconstruction method, and correspondingly, this embodiment provides an ultrasonic image reconstruction system. The system provided in this embodiment can implement the ultrasonic image reconstruction method of embodiment 1, and the system can be implemented by software, hardware, or a combination of software and hardware. For the convenience of description, this embodiment is described by dividing the functions into various units and describing them separately. Of course, the functions of each unit can be implemented in the same or more software and / or hardware during implementation. For example, the system may include integrated or separate functional modules or functional units to perform the corresponding steps in each method of embodiment 1. Since the system of this embodiment is basically similar to the method embodiment, the description process of this embodiment is relatively simple, and the relevant parts can refer to the partial description of embodiment 1. The embodiment of the image reconstruction system provided by the present invention is only schematic.

[0121] Specifically, the ultrasound image reconstruction system provided in this embodiment includes:

[0122] A first unit is configured to establish a measurement model and / or a measurement matrix describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beam synthesis;

[0123] A second unit is configured to obtain RF channel data reflecting acoustic characteristics of the target by transmitting ultrasonic waves to the imaging target and receiving echo signals;

[0124] The third unit is configured to obtain high-quality beamforming data based on the RF channel data and the measurement matrix and / or the measurement matrix and based on a pre-trained neural network for solving the inverse beamforming problem to complete ultrasound image reconstruction.

[0125] Embodiment 3: This embodiment provides an electronic device corresponding to the ultrasound image reconstruction method provided in Embodiment 1. The electronic device may be an electronic device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of Embodiment 1.

[0126] like Figure 3 As shown, the electronic device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Component (EISA) bus, etc. The memory stores a computer program that can be run on the processor, and the processor executes the ultrasound image reconstruction method provided in the first embodiment when running the computer program. Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computing device to which the scheme of the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] In some implementations, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), optical disk and other media that can store program codes.

[0128] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0129] Embodiment 4: The image reconstruction method of the embodiment 1 may be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing the image reconstruction method described in the embodiment 1.

[0130] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0131] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some implementations", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing an ultrasonic image, characterized in that include: A measurement model and / or measurement matrix describing the relationship between the RF channel data received by the ultrasound system and the data after the desired beam synthesis is established, including: The steps to establish a measurement model and / or measurement matrix include: 1) Place a point scattering sub-target in the imaging area; 2) Obtaining ultrasound channel data; 3) Calculate the intensity histogram and cumulative distribution function of the ultrasound channel data; 4) Generate a binary mask based on the cumulative distribution function; 5) Pair the mask with the point target and apply the mask to the channel data to extract the echo signal corresponding to the point scattering sub-target; 6) Repeat steps 1)-5) until the echo signals corresponding to all grid points in the imaging area are completed; 7) The echo signal of each grid point in the imaging area is sequentially rearranged into each column of the measurement matrix to obtain the measurement matrix, wherein: the linear measurement model describing the RF channel data received by the ultrasound system and the data after the beam synthesis to be required is: y=Hx+n in, is the measurement matrix, is additive Gaussian noise, r = D × K is the total number of sampling points of RF channel data, there are K channels in total, each channel contains D sampling points, l = N z ×N x is the total number of grid points of the data after beam synthesis, a total of N x imaging lines, each imaging line contains N z pixels; By transmitting ultrasonic waves to an imaging target and receiving echo signals, RF channel data reflecting the acoustic characteristics of the target are obtained; Based on the RF channel data and the measurement model and / or the measurement matrix, high-quality beamforming data is obtained based on a pre-trained neural network that solves the inverse beamforming problem to complete ultrasound image reconstruction.

2. The ultrasonic image reconstruction method according to claim 1, characterized in that: Training of a neural network to solve the inverse beamforming problem, including: Set the relevant parameters of the neural network; Set the mean square error loss function; According to the prior knowledge of ultrasonic signal characteristics, relevant constraint loss functions are set; Based on the set neural network parameters, mean square error loss function and related constraint loss function, a self-supervised learning strategy is used to complete the training of the neural network for solving the inverse beamforming problem.

3. The ultrasonic image reconstruction method according to claim 2, characterized in that: The related constraint loss function is set based on the prior knowledge of ultrasonic signal characteristics, including: Sparsity constraint function in the wavelet domain; Smoothness constraint function in the spatial domain; Sparse constraint function for the signal envelope.

4. The image reconstruction method according to claim 2, characterized in that: The self-supervised learning strategy is used to complete the training of the neural network to solve the inverse beamforming problem, including: Training phase 1: initial solution is used as both input and label for the neural network and is solved by minimizing the initial solution and recovery solution The signal recovery network, also known as the autoencoder, is obtained by training the mean square error between the two networks. Training stage 2: The high-quality solution x′ obtained by network reconstruction is used to recover the corresponding channel data y′=Hx′, and the network for solving the inverse beamforming problem is trained by minimizing the mean square error loss function between the measured channel data y and the restored channel data y′, and introducing the relevant constraint loss function. In each stage, the ultrasound channel data is first back-projected to generate the initial solution of the inverse beamforming problem. As the input of the network, it is then mapped to the target solution through the neural network.

5. An ultrasonic image reconstruction system, characterized in that: The system includes: The first unit is configured to establish a measurement model and / or a measurement matrix describing the relationship between the RF channel data received by the ultrasound system and the data after the beam synthesis to be required, including: The steps to establish a measurement model and / or measurement matrix include: 1) Place a point scattering sub-target in the imaging area; 2) Obtaining ultrasound channel data; 3) Calculate the intensity histogram and cumulative distribution function of the ultrasound channel data; 4) Generate a binary mask based on the cumulative distribution function; 5) Pair the mask with the point target and apply the mask to the channel data to extract the echo signal corresponding to the point scattering sub-target; 6) Repeat steps 1)-5) until the echo signals corresponding to all grid points in the imaging area are completed; 7) The echo signal of each grid point in the imaging area is sequentially rearranged into each column of the measurement matrix to obtain the measurement matrix, wherein the linear measurement model describing the RF channel data received by the ultrasound system and the data after the beam synthesis to be required is: y=Hx+n in, is the measurement matrix, is additive Gaussian noise, r = D × K is the total number of sampling points of RF channel data, there are K channels in total, each channel contains D sampling points, l = N z ×N x is the total number of grid points of the data after beam synthesis, a total of N x imaging lines, each imaging line contains N z pixels; A second unit is configured to obtain RF channel data reflecting acoustic characteristics of the target by transmitting ultrasonic waves to the imaging target and receiving echo signals; The third unit is configured to obtain high-quality beamforming data based on the RF channel data and the measurement model and / or the measurement matrix and based on a pre-trained neural network for solving the inverse beamforming problem, so as to complete ultrasound image reconstruction.

6. An electronic device, characterized in that: The method comprises computer program instructions, wherein the program instructions are used to implement the ultrasonic image reconstruction method according to any one of claims 1 to 4 when executed by a processor.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, wherein the program instructions are used to implement the ultrasound image reconstruction method according to any one of claims 1 to 4 when executed by a processor.

Images