A three-dimensional ultrasonic CT image reconstruction method based on a neural network proxy model

Abstract

The application discloses a three-dimensional ultrasonic CT image reconstruction method based on a neural network agent model. The application constructs an anatomically real organ model, simulates a wave field by means of a numerical solver, and generates a multi-organ training data set; constructs and trains a strong scattering neural operator (S 2 NO) based on an iterative structure of a convergent Borel series, which is used to solve a partial differential equation with a large calculation domain and high oscillation; collects time-domain wave field observation data of a part to be measured of a patient, and converts the time-domain wave field observation data into frequency-domain wave field observation data in a set frequency range through Fourier transform; uses the trained S 2 NO as an agent model to perform a full waveform inversion algorithm and iteratively reconstructs ultrasonic CT images; the application reconstructs three-dimensional ultrasonic CT images of in-vivo clinical samples, and the speed is about 10 times that of a traditional ultrasonic CT full waveform inversion reconstruction algorithm, and an imaging quality similar to that of standard magnetic resonance imaging (MRI) is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional medical image reconstruction, in particular to a three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model. BACKGROUND

[0002] Ultrasound Computed Tomography (Ultrasound CT) is an emerging medical imaging technology that combines the portability and safety of traditional ultrasound with the high-resolution advantages of CT. However, high-precision Ultrasound CT reconstruction algorithms based on partial differential equation (PDE) solutions have high computational requirements and numerical instability. This problem is more pronounced when imaging three-dimensional strongly scattering organs such as the musculoskeletal system.

[0003] Specifically, Ultrasound CT reconstruction can be modeled as a nonlinear partial differential equation-constrained optimization inverse problem, known as Full Waveform Inversion (FWI). Its goal is to recover the acoustic properties of the medium under test, such as sound speed, density, and acoustic attenuation, with high resolution from the observed wave field. FWI iteratively updates the medium parameters by solving a large number of wave equations in its forward and inverse processes to simulate how ultrasound waves propagate in the medium under test, until the results match the observed data. This process requires capturing the high-frequency characteristics of sound waves on a large number of grid points, which is a high requirement for numerical calculation methods, thus hindering the clinical application of Ultrasound CT.

[0004] In the prior art, there are schemes that use neural networks to improve this problem, which can be mainly summarized into two categories.

[0005] The core of the first method is to learn an end-to-end network, aiming to directly establish a mapping relationship from the observed wave field to the medium parameters under test. Common network architectures include convolutional neural networks (CNN) and physically inspired neural operators. However, this type of method often focuses too much on fitting the inherent patterns of training images, and fails to fully learn the basic physical laws that describe the interaction between wave fields and media.

[0006] The second method is based on the Physics-Informed Neural Network (PINN) framework. This method explicitly encodes the partial differential equation constraints (such as control equations, boundary conditions, and conservation relations) describing wave field propagation into the training loss function, while solving the PDE inverse problem solution and unknown coefficients, thus facilitating the learning of solutions that conform to the underlying physical mechanisms. However, this method has an inherent tendency to preferentially learn low-frequency features, making it difficult to capture high-oscillation wave phenomena, thus limiting its ability to achieve high-resolution imaging on large fields of view in clinical Ultrasound CT.

[0007] Therefore, there is an urgent need for a new method to solve the two key challenges: the difference between simulation data and real data, and the physical field characterization of high-frequency oscillation waves. SUMMARY

[0008] In view of the above problems of the prior art, the present application provides a three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model.

[0009] The three-dimensional ultrasound CT image reconstruction method based on the neural network proxy model of the present application comprises the following steps:

[0010] 1) Construct an anatomically realistic organ model, and simulate the scattered wave field with the help of a numerical solver to generate a multi-organ training data set containing the medium parameter distribution and the scattered wave field corresponding to the medium parameter distribution;

[0011] 2) Construct a strong scattering neural operator (S 2 NO) based on the iterative structure of the convergent Borne series, which is used to solve the partial differential equation in the full waveform inversion algorithm with large calculation domain and high oscillation;

[0012] 3) Train the strong scattering neural operator using the multi-organ training data set in step 1) to obtain a trained strong scattering neural operator;

[0013] 4) Use an ultrasound CT device to collect time-domain wave field observation data of the part of the patient to be measured, and convert it into frequency-domain wave field observation data in a specified frequency range through Fourier transform;

[0014] 5) Use the trained S 2 NO as a proxy model to perform full waveform inversion algorithm on the frequency-domain wave field observation data, and iteratively reconstruct the ultrasound CT three-dimensional image.

[0015] In step 1), the medium parameters are sound speed, density or acoustic attenuation.

[0016] An anatomically realistic organ model is constructed based on the original image of the organ, and the organ is segmented into multiple different tissues, such as skin, fat, gland and other tissues for the breast, and skin, fat, muscle, bone and other tissues for the upper arm and thigh. First, select the type of medium parameter, then assign the medium parameter value of the corresponding position to different tissues, and the medium parameter value is a two-dimensional distribution. The medium parameter value assignment method: use a linear or polynomial function to map the pixel value of the original image to the reference range of the medium parameter of human tissue. For the breast, the original image is a two-dimensional image of the digital simulation model slice; for the upper arm and thigh, the original image is a clinical image of other modalities other than ultrasound CT; other modalities other than ultrasound CT are X-ray CT or magnetic resonance imaging (MRI).

[0017] In the simulation of the scattered wave field by means of the numerical solver, a plurality of sound sources fixed in position are provided in the numerical solver, and each sound source respectively emits a plurality of sound waves of different frequencies to the organ model. The frequency of the sound source is 0.2-1.2MHz, and the interval is 0.01-0.1MHz. Through the numerical solver simulation, the sound waves emitted by the fixed sound source pass through the organ model to obtain the corresponding output scattered wave field.

[0018] In step 2), the strong scattering neural operator is constructed based on the iterative structure of the convergent Born series, including the following steps:

[0019] a) The strong scattering neural operator includes an encoder, an iterative calculation layer, and a decoder; the iterative calculation layer is constructed based on the iterative structure of the convergent Born series, and the iterative calculation layer includes a plurality of same calculation modules connected in a loop;

[0020] b) The medium parameter distribution is encoded into the hidden space by the encoder to obtain the preprocessor , the scattering potential , and the initial wave field and the hidden state representation of the initial wave field :

[0021]

[0022] wherein, is the encoder of the preprocessor, is the encoder of the scattering potential, is the encoder of the initial wave field;

[0023] c) In the hidden space, the wave field hidden state is iteratively updated between the iterative calculation layers to obtain the final wave field hidden state , and the final wave field hidden state , =1,…,N, N is the number of calculation modules in the iterative calculation layer, and N is a natural number of 3-8;

[0024]

[0025] wherein, is one calculation module in the iterative calculation layer;

[0026] d) The final wave field hidden state is projected back to the physical space by the decoder to obtain the output scattered wave field :

[0027] .

[0028] In step 4), the frequency range needs to be consistent with the sound source frequency in the numerical solver simulation of the scattered wave field in step 1).

[0029] In step 5), S 2 NO as a proxy model, performs a full waveform inversion algorithm to iteratively reconstruct an ultrasound CT image, including the following steps:

[0030] a) input the frequency domain wave field observation data and the initial medium parameter distribution image into the full waveform inversion algorithm;

[0031] b) based on the current medium parameter distribution image, S 2 NO solves the partial differential equation to perform forward propagation and adjoint state calculation, and the output scattered wave field and adjoint state are solved; then the loss function gradient is calculated;

[0032] c) using the obtained loss function gradient, using an optimization method to minimize the optimization objective, iteratively updating the medium parameter distribution image until convergence, obtaining an iteratively reconstructed ultrasound CT three-dimensional image.

[0033] Advantages of the present application:

[0034] The present application reconstructs a three-dimensional ultrasound CT image of a clinical sample in vivo, and the speed is about 10 times that of a traditional ultrasound CT full waveform inversion reconstruction algorithm, and achieves an imaging quality similar to that of a standard magnetic resonance imaging (MRI). BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A flowchart of the three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model of the present application;

[0036] Figure 2 A result graph of one embodiment of the three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described below with reference to the accompanying drawings and specific embodiments.

[0038] As Figure 1 shown, the three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model of the upper arm and thigh of the present embodiment includes the following steps:

[0039] 1) Construct an anatomically realistic organ model based on the original image of the organ, segment the organ into different types of tissues using an artificial intelligence segmentation model: skin, fat, muscle, bone and other tissues, assign the corresponding medium parameter value of the position to different tissues, in this embodiment, the medium parameter is sound speed; set 256 fixed-position sound sources in the numerical solver, each sound source respectively emits sound waves of frequencies from 0.25MHz to 0.60MHz at equal intervals of 0.05MHz to the organ model, simulate the scattered wave field by means of the numerical solver, obtain the corresponding scattered wave field, and generate a multi-organ training data set containing the medium parameter distribution and the scattered wave field corresponding to the medium parameter distribution;

[0040] 2) Construct a strong scattering neural operator (S 2 NO) to solve the partial differential equation with large calculation domain and high oscillation of the full waveform inversion algorithm:

[0041] a) The strong scattering neural operator includes an encoder, an iterative calculation layer and a decoder; the iterative calculation layer is constructed based on the iterative structure of the convergent Bessel series, and the iterative calculation layer includes four same calculation modules connected in a loop;

[0042] b) The medium parameter distribution and the sound source are encoded into the hidden space through the encoder to obtain the hidden state representation of the preprocessor , the scattering potential and the initial wave field :

[0043]

[0044] Wherein, is the encoder of the preprocessor, is the encoder of the scattering potential, is the encoder of the initial wave field;

[0045] c) In the hidden space, the wave field hidden state is iteratively updated between the iterative calculation layers to obtain the final wave field hidden state , =1,…,N=4;

[0046]

[0047] Wherein, is one calculation module in the iterative calculation layer;

[0048] d) The final wave field hidden state is projected back to the physical space through the decoder to obtain the output scattered wave field :

[0049] ;

[0050] 3) Using the multi-organ training dataset in step 1), training the strong scattering neural operator to obtain a trained strong scattering neural operator;

[0051] 4) Using the ultrasound CT device, collecting the time-domain wave field observation data of the part to be measured of the patient, and converting it into frequency-domain wave field observation data in the set frequency range through Fourier transform:

[0052] The ultrasound CT acquisition instrument uses a ring transducer array composed of 256 transducers, each of which can act as a transmitter and a receiver. The transmitter corresponds to the sound source in step 1). During data acquisition, one transducer emits a sound wave signal, and all transducers act as receivers, repeating the process for all 256 elements. By gradually vertically scanning, multiple two-dimensional slices are obtained, and then stacked to obtain a complete three-dimensional structure. The center frequency of the transducer is 0.9 MHz, the signal is sampled at 25 MHz, and the sampling time is 0.198 ms. In order to facilitate frequency-domain full waveform inversion and improve signal-to-noise ratio, a Butterworth band-pass filter is used to filter the collected time-domain signal, and then the filtered time-domain signal is converted into multiple single-frequency components through discrete Fourier transform;

[0053] 5) Using the trained S 2 NO as a surrogate model, performing full waveform inversion algorithm to iteratively reconstruct the ultrasound CT image:

[0054] a) Input the frequency-domain wave field observation data and the initial medium parameter distribution image into the full waveform inversion algorithm:

[0055] The medium parameter is the sound speed, and the initial value of the sound speed can be set to the sound speed of water 1500 m / s. Under stable conditions, the full waveform inversion is represented as an optimization problem constrained by the following Helmholtz equation:

[0056] Minimize the optimization objective:

[0057] Subject to the constraint of the Helmholtz equation: ;

[0058] b) Based on the current medium parameter distribution image, solve the partial differential equation through S 2 NO to perform forward propagation and adjoint state calculation of the full waveform inversion algorithm, and solve the output scattered wave field and adjoint state:

[0059] Using the Lagrange multiplier method, this problem is represented as minimizing the Lagrange function:

[0060]

[0061] where, denotes the sound speed distribution of the medium to be measured in the computational domain, and denotes the wavefield in the computational domain and the transducer positions, respectively, denotes the observed wavefield, defines the spatial coordinates of the computational domain, denotes the transducer positions of the ultrasound CT device, denotes the angular frequency of the wavefield, denotes the wave source term in the computational domain, = 1,..., K, K is the number of transducers, is the Laplace operator; and is the Lagrangian function, defines the function and is the real part of the inner product in space; the loss function gradient is proportional to the wavefield and the product of the adjoint state , is the real part, denotes the conjugate:

[0062]

[0063] where, and are both solved by S 2 NO to obtain the Helmholtz equation:

[0064]

[0065] where, denotes the position of the th sound source, denotes the observed wavefield of the th transducer, denotes the point wave source function; then the loss function gradient is calculated; using cluster multi-card parallel computing, the calculation is performed simultaneously for each slice of the upper arm and the lower leg;

[0066] c) Using the obtained loss function gradient, the nonlinear conjugate gradient method (NCG) is used to minimize the residual error between the observed data and the proxy model prediction data, iteratively updating the image of the medium parameter distribution until convergence, obtaining the reconstruction results of each layer of the upper arm and the lower leg, and then stacking them to obtain the iterative reconstruction ultrasound CT three-dimensional image after interpolation.

[0067] The reconstruction results of the upper arm and the lower leg are as follows: Figure 2The first and second columns are the three-dimensional reconstruction results of the upper arm and the thigh, respectively, and the third and fourth columns are the two-dimensional cross-sectional views of the three-dimensional reconstruction results of the upper arm and the thigh, respectively. 2 NO-FWI is the FWI reconstruction using S 2 NO proxy model, MRI is the magnetic resonance imaging, FWI is the FWI reconstruction using traditional numerical solver, and DAS is the reconstruction result using the delay and sum method. The S 2 The NO-FWI method realizes high definition close to the magnetic resonance imaging, and the reconstruction speed is significantly faster than the traditional numerical FWI method: the upper arm S 2 The NO-FWI reconstruction time is 8.50 minutes, which is 7 times faster than the traditional numerical FWI; the thigh S 2 The NO-FWI reconstruction time is 9.46 minutes, which is 14 times faster than the traditional numerical FWI.

[0068] Finally, it should be noted that the purpose of the disclosed embodiments is to help further understand the present application, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present application and the appended claims. Therefore, the present application should not be limited to the disclosed embodiments, and the scope of the present application is defined by the scope of the claims.

Claims

1. A method for reconstructing three-dimensional ultrasound CT images based on a neural network surrogate model, characterized in that, The method includes the following steps: 1) Construct realistic anatomical organ models and use a numerical solver to simulate the scattered wave field, generating a multi-organ training dataset containing the distribution of medium parameters and the scattered wave field corresponding to the distribution of medium parameters; 2) Constructing a strong scattering neural operator S based on the iterative structure of a convergent Born series. 2 NO, is used to solve the partial differential equations in the full waveform inversion algorithm; The construction of a strong scattering neural operator based on the iterative structure of a convergent Born series includes the following steps: a) The strong scattering neural operator includes an encoder, an iterative computation layer, and a decoder; the iterative computation layer is constructed based on the iterative structure of the convergent Born series, and the iterative computation layer includes multiple layers of the same computation modules connected end to end; b) Distribute medium parameters With the sound source via encoder Encode into the hidden space to obtain the preprocessor Scattering potential and initial wave field Hidden state representation: in, The encoder for the preprocessor, For encoders with scattering potential, The encoder for the initial wave field; c) In the latent space, iteratively update the wavefield latent state between iterative computation layers. Solving for the final hidden wave field state yields the solution. , =1,…,N, where N is the number of computation modules in the iterative computation layer; in, It is a computation module in the iterative computation layer; d) The final hidden state of the wave field via decoder Projecting back into physical space yields the output scattered wave field. : ； 3) Using the multi-organ training dataset from step 1), train the strong scattering neural operator to obtain the trained strong scattering neural operator; 4) Using an ultrasound CT device, time-domain wave field observation data of the patient's target area is acquired and converted into frequency-domain wave field observation data within a set frequency range through Fourier transform. 5) For frequency domain wavefield observation data, use the trained S 2 NO is used as a surrogate model to execute the full waveform inversion algorithm and iteratively reconstruct the ultrasound CT three-dimensional image.

2. The method as described in claim 1, characterized in that, In step 1), an anatomically realistic organ model is constructed based on the original image of the organ, the organ is segmented into different types of tissues, and media parameter values ​​are assigned to different tissues at corresponding locations.

3. The method as described in claim 1, characterized in that, In step 1), a numerical solver is used to simulate the scattered wave field. Multiple sound sources with fixed positions are set in the numerical solver, and each sound source emits multiple sound waves of different frequencies to the organ model.

4. The method as described in claim 3, characterized in that, In step 4), the frequency range needs to be consistent with the frequency of the sound source in the scattered wave field simulated by the numerical solver in step 1).

5. The method as described in claim 1, characterized in that, In step 5), S 2 NO is used as a surrogate model to execute a full waveform inversion algorithm and iteratively reconstruct ultrasound CT images, including the following steps: a) Input the frequency domain wavefield observation data and the initial medium parameter distribution image into the full waveform inversion algorithm; b) Based on the current medium parameter distribution, the image is obtained through S 2 NO solves partial differential equations by performing forward propagation and adjoint state calculations, solving for the output scattered wave field and adjoint state; then it calculates the gradient of the loss function. c) Using the obtained loss function gradient, an optimization method is adopted to minimize the optimization objective and iteratively update the image of the medium parameter distribution until convergence, thus obtaining the iteratively reconstructed ultrasound CT three-dimensional image.

Images