Three-dimensional ultrasonic CT image reconstruction method based on neural network agent model

Abstract

The invention discloses a three-dimensional ultrasonic CT image reconstruction method based on a neural network agent model. The method comprises the following steps: constructing an anatomical real organ model, simulating a wave field by virtue of a numerical solver, and generating a multi-organ training data set; constructing and training a strong scattering neural operator (S2NO) based on an iterative structure of convergence Born series, so as to solve a partial differential equation with a large computational domain and high oscillation; collecting time domain wave field observation data of a to-be-detected part of a patient, and converting the time domain wave field observation data into frequency domain wave field observation data in a set frequency range through Fourier transform; using the trained S2NO as an agent model, executing a full waveform inversion algorithm, and iteratively reconstructing an ultrasonic CT image; according to the method, a three-dimensional ultrasonic CT image of an in-vivo clinical sample is reconstructed, the speed reaches about 10 times that of a traditional ultrasonic CT full-waveform inversion reconstruction algorithm, and the imaging quality similar to the standard magnetic resonance imaging (MRI) resolution is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional medical image reconstruction, and particularly relates to a three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model. BACKGROUND

[0002] Ultrasound Computed Tomography (ultrasound CT) is an emerging medical imaging technology that combines the portability and safety of traditional ultrasound with the high-resolution advantages of CT. However, high-precision ultrasound CT reconstruction algorithms based on partial differential equation (PDE) solutions have high computational requirements and numerical instability. This problem is more pronounced when imaging three-dimensional strongly scattering organs such as the musculoskeletal system.

[0003] Specifically, ultrasound CT reconstruction can be modeled as a nonlinear partial differential equation-constrained optimization inverse problem, known as full waveform inversion (FWI). Its goal is to recover the acoustic properties of the medium under test, such as sound speed, density, and acoustic attenuation, at high resolution from the observed wave field. FWI iteratively updates the medium parameters by solving a large number of wave equations in its forward and inverse processes to simulate how ultrasound waves propagate in the medium under test, until the results match the observed data. This process requires capturing the high-frequency characteristics of sound waves at a large number of grid points, which is a high requirement for numerical calculation methods, thereby hindering the clinical application of ultrasound CT.

[0004] In the prior art, there are schemes that use neural networks to improve this problem, which can be mainly summarized into two categories.

[0005] Among them, the core of the first method is to learn an end-to-end network, aiming to directly establish a mapping relationship from the observed wave field to the medium parameters under test. Common network architectures include convolutional neural networks (CNN) and physically inspired neural operators, etc. However, this type of method often focuses too much on fitting the inherent patterns of training images, and fails to fully learn the basic physical laws that describe the interaction between wave fields and media.

[0006] The second method is based on the physical information neural network (PINN) framework. This type of method explicitly encodes the partial differential equation constraints (such as control equations, boundary conditions, and conservation relations) that describe wave field propagation into the training loss function, while solving the solution and unknown coefficients of the PDE inverse problem, thereby facilitating the learning of solutions that conform to the underlying physical mechanisms. However, this method has an inherent tendency to preferentially learn low-frequency features, making it difficult to capture high-oscillation wave phenomena, thus limiting its ability to achieve high-resolution imaging in large fields of view in clinical ultrasound CT.

[0007] Therefore, there is an urgent need for a new method to solve the two key challenges: the difference between simulation data and real data, and the physical field characterization of high-frequency oscillation waves. SUMMARY

[0008] In view of the problems existing in the prior art, the present application provides a three-dimensional ultrasound CT image reconstruction method based on a neural network proxy model.

[0009] The three-dimensional ultrasound CT image reconstruction method based on the neural network proxy model of the present application comprises the following steps:

[0010] 1) Construct an anatomically realistic organ model, and simulate the scattered wave field with the aid of a numerical solver to generate a multi-organ training data set containing the medium parameter distribution and the scattered wave field corresponding to the medium parameter distribution;

[0011] 2) Construct a strong scattering neural operator (S 2 NO) based on the iterative structure of the convergent Borne series, which is used to solve the partial differential equation in the full waveform inversion algorithm with large calculation domain and high oscillation;

[0012] 3) Train the strong scattering neural operator using the multi-organ training data set in step 1) to obtain a trained strong scattering neural operator;

[0013] 4) Use an ultrasound CT device to collect time-domain wave field observation data of the part of the patient to be measured, and convert it into frequency-domain wave field observation data in a set frequency range through Fourier transform;

[0014] 5) Use the trained S 2 NO as a proxy model to perform full waveform inversion algorithm to iteratively reconstruct the three-dimensional ultrasound CT image.

[0015] In step 1), the medium parameters are sound speed, density or acoustic attenuation.

[0016] An anatomically realistic organ model is constructed based on the original image of the organ, and the organ is segmented into multiple different tissues, such as skin, fat, gland and other tissues for the breast, and skin, fat, muscle, bone and other tissues for the upper arm and thigh. First, select the type of medium parameter, then assign the medium parameter value of the corresponding position to different tissues, and the medium parameter value is a two-dimensional distribution. The medium parameter value assignment method: use a linear or polynomial function to map the pixel value of the original image to the reference range of the medium parameter of human tissue. For the breast, the original image is a two-dimensional image of the digital simulation model slice; for the upper arm and thigh, the original image is a clinical image of other modalities other than ultrasound CT; other modalities other than ultrasound CT are X-ray CT or magnetic resonance imaging (MRI).

[0017] In the simulation of the scattered wave field by means of the numerical solver, a plurality of sound sources fixed in position are provided in the numerical solver, and each sound source respectively emits a plurality of sound waves of different frequencies to the organ model. The frequency of the sound source is 0.2-1.2MHz, and the interval is 0.01-0.1MHz. Through the numerical solver simulation, the sound waves emitted by the fixed sound source pass through the organ model, and the corresponding output scattered wave field is obtained.

[0018] In step 2), the strong scattering neural operator is constructed based on the iterative structure of the convergent Born series, including the following steps:

[0019] a) The strong scattering neural operator includes an encoder, an iterative calculation layer and a decoder; the iterative calculation layer is constructed based on the iterative structure of the convergent Born series, and the iterative calculation layer includes a plurality of same calculation modules connected in a loop;

[0020] b) The medium parameter distribution c and the sound source p are encoded into the latent space through the encoder ε θ to obtain the latent state representation of the preprocessor q, the scattering potential v and the initial wave field u0:

[0021]

[0022] wherein, is the encoder of the preprocessor, is the encoder of the scattering potential, is the encoder of the initial wave field;

[0023] c) In the latent space, the wave field latent state u n is iteratively updated between the iterative calculation layers to obtain the final wave field latent state u N , n = 1, …, N, N is the number of calculation modules in the iterative calculation layer, and N is a natural number of 3-8;

[0024]

[0025] wherein, is one calculation module in the iterative calculation layer;

[0026] d) The final wave field latent state u N is projected back to the physical space through the decoder to obtain the output scattered wave field u out :

[0027]

[0028] In step 4), the frequency range needs to be consistent with the sound source frequency in the numerical solver simulation of the scattered wave field in step 1).

[0029] In step 5), S 2As a proxy model, NO performs the full waveform inversion algorithm to iteratively reconstruct the ultrasound CT image, including the following steps:

[0030] a) input the frequency domain wave field observation data and the initial medium parameter distribution image into the full waveform inversion algorithm;

[0031] b) based on the current medium parameter distribution image, calculate the scattering wave field and the adjoint state by solving the partial differential equation using S 2 NO solves the partial differential equation to perform forward propagation and adjoint state calculation, and the output scattering wave field and adjoint state are obtained; then the loss function gradient is calculated;

[0032] c) using the obtained loss function gradient, using an optimization method to minimize the optimization objective, iteratively updating the medium parameter distribution image until convergence, obtaining the iteratively reconstructed ultrasound CT three-dimensional image.

[0033] Advantages of the present application:

[0034] The present application reconstructs the three-dimensional ultrasound CT image of the in-vivo clinical sample, the speed is about 10 times of the traditional ultrasound CT full waveform inversion reconstruction algorithm, and the imaging quality similar to that of the standard magnetic resonance imaging (MRI) is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 Flowchart of the three-dimensional ultrasound CT image reconstruction method based on the neural network proxy model of the present application;

[0036] Figure 2 Result graph of one embodiment of the three-dimensional ultrasound CT image reconstruction method based on the neural network proxy model of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described below by specific embodiments in conjunction with the accompanying drawings.

[0038] As shown in the drawings, the three-dimensional ultrasound CT image reconstruction method based on the neural network proxy model of the upper arm and thigh of the present embodiment includes the following steps: Figure 1

[0039] 1) construct an anatomically real organ model based on the original image of the organ, use an artificial intelligence segmentation model to segment the organ into different types of tissues: skin, fat, muscle, bone and other tissues, assign the medium parameter values at the corresponding positions to different tissues, in the present embodiment, the medium parameter is the sound speed; set 256 sound sources with fixed positions in the numerical solver, each sound source respectively emits sound waves with frequencies of 0.25MHz to 0.60MHz at equal intervals of 0.05MHz to the organ model, simulate the scattering wave field by means of the numerical solver, obtain the corresponding scattering wave field, and generate a multi-organ training data set containing the medium parameter distribution and the scattering wave field corresponding to the medium parameter distribution;​

[0040] 2) Construct a strong scattering neural operator (S 2 NO) to solve the partial differential equation of large computational domain and high oscillation of full waveform inversion algorithm:

[0041] a) The strong scattering neural operator includes an encoder, an iterative calculation layer and a decoder; the iterative calculation layer is constructed based on the iterative structure of the convergent Borel series, and the iterative calculation layer includes four identical calculation modules connected in a loop;

[0042] b) The medium parameter distribution c and the sound source v are encoded into the latent space through the encoder ε θ , to obtain the latent state representation of the preprocessor q, the scattering potential v and the initial wave field u0:

[0043]

[0044] wherein, is the encoder of the preprocessor, is the encoder of the scattering potential, is the encoder of the initial wave field;

[0045] c) In the latent space, the wave field latent state u n is iteratively updated between the iterative calculation layers to obtain the final wave field latent state u N , n = 1, …, N, N = 4;

[0046]

[0047] wherein, is one calculation module in the iterative calculation layer;

[0048] d) The final wave field latent state u N is projected back to the physical space through the decoder to obtain the output scattering wave field u out :

[0049]

[0050] 3) Using the multi-organ training data set in step 1), training the strong scattering neural operator to obtain the trained strong scattering neural operator;

[0051] 4) Using an ultrasonic CT device, collecting time-domain wave field observation data of the part to be measured of the patient, and converting the time-domain wave field observation data into frequency-domain wave field observation data in a set frequency range through Fourier transform:

[0052] The ultrasonic CT acquisition instrument uses a ring transducer array, which is composed of 256 transducers, each of which can act as a transmitter and a receiver. The transmitter corresponds to the sound source in step 1). During data acquisition, one transducer emits a sound wave signal, and all transducers act as receivers. This process is repeated for all 256 elements. By gradually vertically scanning, multiple two-dimensional slices are obtained, and then stacked to obtain a complete three-dimensional structure. The center frequency of the transducer is 0.9 MHz, the signal is sampled at 25 MHz, and the sampling time is 0.198 ms. In order to facilitate frequency domain full waveform inversion and improve signal-to-noise ratio, a Butterworth band-pass filter is used to filter the collected time domain signal, and then the filtered time domain signal is converted into multiple single frequency components through discrete Fourier transform;

[0053] 5) Using the trained S 2 NO as a surrogate model, performing full waveform inversion algorithm, iterative reconstruction of ultrasonic CT image: a) input the frequency domain wave field observation data and the initial medium parameter distribution image into the full waveform inversion algorithm:

[0054] The medium parameter is the sound velocity, and the initial value of the sound velocity can be set to the sound velocity of water 1500 m / s. Under stable conditions, the full waveform inversion is represented as an optimization problem constrained by the following Helmholtz equation:

[0055] The optimization objective is minimized:

[0056] The Helmholtz equation is used as a constraint: b) based on the current medium parameter distribution image, the forward propagation and adjoint state calculation of the full waveform inversion algorithm are solved by S 2 NO to solve the partial differential equation, the output of the scattering wave field and the adjoint state are solved:

[0057] Using the Lagrange multiplier method, this problem is represented as minimizing the Lagrange function:

[0058]

[0059] where c(x) represents the sound velocity distribution of the medium to be measured in the calculation domain, u k (x) and u k (x f ) represent the wave fields in the calculation domain and the transducer position respectively, y k represents the observed wave field, x defines the spatial coordinates of the calculation domain, x f represents the transducer position of the ultrasonic CT device, ω represents the wave field angular frequency, ρ k (x) represents the wave source term in the calculation domain, k = 1, …, K, K is the number of transducers, is the Laplace operator; With is the Lagrangian function, <f,g> defines the real part of the inner product of functions f and g in

[0060] space; loss function gradient is proportional to the wave field u k and the product of the adjoint state λ k , Re is

[0061] the real part, denotes the conjugate of λ k :

[0062]

[0063] where u k and λ k are obtained by solving the Helmholtz equation using S 2 NO:

[0064]

[0065] where, denotes the position of the i-th sound source, denotes the observed wave field of the i-th transducer, and δ denotes the point source function; then the loss function gradient is calculated; using cluster multi-card parallel computing, the calculation is performed simultaneously for each slice of the upper arm and the thigh;

[0066] c) Using the obtained loss function gradient, using the nonlinear conjugate gradient method (NCG), minimizing the residual of the observed data and the proxy model prediction data, iteratively updating the image of the medium parameter distribution until convergence, obtaining the reconstruction results of each slice of the upper arm and the thigh, and stacking them after interpolation to obtain the iterative reconstruction ultrasound CT three-dimensional image.

[0067] The reconstruction results of the upper arm and the thigh are shown in Figure 2 . Among them, the first and second columns are the three-dimensional reconstruction results of the upper arm and the thigh, respectively, and the third and fourth columns are two-dimensional cross-sectional views of the three-dimensional reconstruction results of the upper arm and the thigh, respectively. 2 NO-FWI is the FWI reconstruction using S 2 NO proxy model, MRI is nuclear magnetic resonance imaging, FWI is the FWI reconstruction using a traditional numerical solver, and DAS is the reconstruction result using the delay and sum method. The S 2 NO-FWI method proposed in the present application realizes high definition close to nuclear magnetic resonance imaging, and the reconstruction speed is significantly faster than the traditional numerical FWI method: the upper arm S 2 NO-FWI reconstruction time is 8.50 minutes, which is 7 times faster than the traditional numerical FWI; the thigh S 2The NO-FWI reconstruction time is 9.46 minutes, which is 14 times faster than the traditional numerical FWI.

[0068] Finally, it is important to note that the embodiments disclosed herein are intended to be illustrative only and that the true scope of the present application should be determined by reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. Accordingly, the application should not be limited to the embodiments disclosed herein, but should be given the full scope of the appended claims.

Claims

1. A method for three-dimensional ultrasound CT image reconstruction based on a neural network proxy model, characterized in that, The method comprises the following steps: 1) constructing an anatomically realistic organ model, simulating a scattered wave field by means of a numerical solver, and generating a multi-organ training data set containing a medium parameter distribution and a scattered wave field corresponding to the medium parameter distribution; 2) Constructing strong scattering neural operator S based on the iterative structure of convergent Borel series 2 NO, to solve partial differential equations in full waveform inversion algorithms; 3) training a strong scattering neural operator by using the multi-organ training data set in step 1), to obtain a trained strong scattering neural operator; 4) using an ultrasound CT device, collecting time-domain wave field observation data of a patient's to-be-measured part, and converting the time-domain wave field observation data into frequency-domain wave field observation data of a set frequency range by Fourier transform; 5) For frequency domain wavefield observation data, use the trained S 2 NO performs full waveform inversion algorithm as a surrogate model, iteratively reconstructing ultrasound CT 3D images.

2. The method of claim 1, wherein, In step 1), an anatomically realistic organ model is constructed based on an original image of the organ, the organ is segmented into different types of tissues, and the medium parameter values of different tissues are assigned to the corresponding positions.

3. The method of claim 1, wherein, In step 1), the numerical solver is used to simulate the scattered wave field, and a plurality of sound sources with fixed positions are set in the numerical solver, each sound source emitting sound waves of different frequencies to the organ model.

4. The method of claim 1, wherein, In step 2), the strong scattering neural operator is constructed based on the iterative structure of the convergent Bessel series, comprising the following steps: a) the strong scattering neural operator comprises an encoder, an iterative calculation layer and a decoder; the iterative calculation layer is constructed based on the iterative structure of the convergent Bessel series, and the iterative calculation layer comprises a plurality of identical calculation modules connected in a loop; b) the medium parameter distribution c is combined with the sound source p by an encoder ε θ Encode into the latent space, obtaining the latent state representation of the preprocessor q, the scattering potential v, and the initial wavefield u0: wherein, an encoder for a pre-processor, an encoder for a scattering potential, an encoder for an initial wavefield; c) In the hidden space, iteratively calculate the iterative update wave field hidden state u between layers n , solve to get the final wave field hidden state u N , n = 1, …, N, N is the number of calculation modules in the iterative calculation layer; wherein, is a computing module in the iterative computation layer; d) the final wavefield hidden state u N by the decoder projected back to the physical space, resulting in the output scattered wavefield u out :

5. The method of claim 3, wherein, In step 4), the frequency range needs to be consistent with the sound source frequency in the numerical solver simulating the scattered wave field in step 1).

6. The method of claim 1, wherein, In step 5) S 2 NO as a proxy model, performs a full waveform inversion algorithm, iteratively reconstructing the ultrasound CT image, comprising the following steps: a) input the frequency-domain wave field observation data and the initial medium parameter distribution image into the full waveform inversion algorithm; b) based on the current image of the medium parameter distribution, by S 2 NO solves the partial differential equation to perform forward propagation and adjoint state calculation, and solve the output of the scattered wave field and the adjoint state; then calculate the loss function gradient; c) using the obtained loss function gradient, using an optimization method to minimize the optimization target, and iteratively updating the medium parameter distribution image until convergence, to obtain an iterative reconstructed ultrasound CT three-dimensional image.

Images