Cardiac ultrasound image three-dimensional reconstruction method, device and equipment and storage medium

Abstract

The invention discloses a cardiac ultrasound image three-dimensional reconstruction method, device and equipment and a storage medium, and relates to the technical field of image processing. The three-dimensional reconstruction method comprises the following steps: constructing a sample data set; an image reconstruction model is constructed, the image reconstruction model comprises a physical parameter prediction module and a physical rendering module, and the physical parameter prediction module performs feature extraction on the position of each point in the real ultrasonic image to obtain physical parameters of each point in the real ultrasonic image; the physical rendering module generates predicted ultrasonic images according to the physical parameters of each point in the real ultrasonic image, and generates a three-dimensional ultrasonic image according to all the predicted ultrasonic images; training and evaluating a physical parameter prediction module in the image reconstruction model by using the sample data set to obtain a trained image reconstruction model; and realizing three-dimensional reconstruction of the cardiac ultrasound image by using the trained image reconstruction model. The problem that a traditional three-dimensional reconstruction method is not accurate enough is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular, to a three-dimensional reconstruction method, device, equipment and storage medium for cardiac ultrasound images. Background Art

[0002] Ultrasonic imaging is a medical imaging technology widely used in clinical diagnosis. Due to its advantages such as portability, real-time performance, and radiation-free, ultrasound plays an important role in various clinical applications. Currently, the ultrasound equipment clinically used for cardiovascular diseases mainly provides two-dimensional images of a single cross-section. However, the heart itself is a three-dimensional spatial structure. During actual diagnosis, doctors need to fuse these two-dimensional images in their minds through long-term training and clinical experience to generate a three-dimensional anatomical structure model. This subjective fusion is easily restricted by personal experience and has a relatively high risk of error.

[0003] Currently, the three-dimensional reconstruction of cardiac ultrasound has the following technical difficulties:

[0004] 1) Due to the different propagation characteristics of ultrasonic waves in different tissues, the same anatomical structure may present different image manifestations at different probe positions. This perspective dependence makes three-dimensional reconstruction more difficult, and conventional three-dimensional reconstruction methods are difficult to accurately restore the true structure of tissues.

[0005] 2) Existing three-dimensional ultrasound reconstructions mainly rely on special wobbling probes, two-dimensional probe arrays, or probes with position tracking to synthesize three-dimensional images. Although this method can obtain three-dimensional data, the reconstruction quality is not ideal. For example, when observing the reconstructed volume from different perspectives, unreasonable acoustic shadows may appear; at certain perspectives, anatomical structures that are actually impossible to be reached by ultrasonic waves may be displayed; the lack of accurate modeling of the physical characteristics of ultrasound results in the reconstructed results not being realistic enough.

[0006] In summary, there is an urgent need for a more advanced ultrasound three-dimensional reconstruction method that not only considers the physical characteristics of ultrasound imaging but also can accurately reconstruct the three-dimensional structure from sparse-view two-dimensional images, providing more realistic and reliable three-dimensional ultrasound imaging results and more accurate information for clinical diagnosis and treatment. Summary of the Invention

[0007] Aiming at the deficiencies of the existing technology, the present invention provides a three-dimensional reconstruction method, device, equipment and storage medium for cardiac ultrasound images, which deeply integrates deep learning technology with the physical characteristics of ultrasound, uses deep learning technology to predict ultrasound physical parameters from spatial coordinates, and realizes ultrasound image prediction through physical rendering to ensure the accuracy of three-dimensional ultrasound images and the physical consistency of images from different perspectives.

[0008] In a first aspect, the present invention provides a three-dimensional reconstruction method for cardiac ultrasound images, including:

[0009] Construct a sample data set; wherein, the sample data set includes real ultrasonic images of the heart at different scanning angles and the positions of each point in each frame of real ultrasonic image;

[0010] Construct an image reconstruction model; wherein, the image reconstruction model includes a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasonic image to obtain the physical parameters of each point in the real ultrasonic image; the physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, and generate a three-dimensional ultrasonic image according to all the predicted ultrasonic images;

[0011] Use the sample data set to train and evaluate the physical parameter prediction module in the image reconstruction model to obtain a trained image reconstruction model;

[0012] Use the trained image reconstruction model to realize the three-dimensional reconstruction of cardiac ultrasound images.

[0013] Furthermore, the physical parameter prediction module adopts a neural network.

[0014] Furthermore, the physical parameters include attenuation coefficient, reflection coefficient, boundary probability, scattering density and scattering intensity.

[0015] Furthermore, the physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, specifically including:

[0016] Calculate the remaining energy of each point according to the physical parameters of each point in the real ultrasonic image;

[0017] Calculate the total echo intensity of each point according to the remaining energy and physical parameters of each point;

[0018] Generate a predicted ultrasonic image according to the total echo intensity of each point.

[0019] Furthermore, the calculation formula of the remaining energy is:

[0020]

[0021] wherein, I(r, t) represents the remaining energy at the depth t on the scan line r; I0 represents the initial energy; β(r, n) represents the reflection coefficient at the depth n on the scan line r; G(r, n) represents the boundary indication function at the depth n on the scan line r, which is obtained by sampling the boundary probability at the depth n on the scan line r; α represents the attenuation coefficient; f represents the frequency factor; dt represents the depth step;

[0022] The calculation formula of the total echo intensity is:

[0023]

[0024] E(r, t) = R(r, t) + B(r, t);

[0025] Wherein, R(r, t) represents the reflected energy at depth t on scan line r; PSF(r) represents the point spread function; represents the convolution operation; G(r′, t′) represents the boundary indication function at depth t′ on scan line r′, where scan line r′ is the offset of scan line r and depth t′ is the offset of depth t; B(r, t) represents the scattered energy at depth t on scan line r; r(r′, t′) represents the scattering characteristics at depth t′ on scan line r′; H(r′, t′) represents the scattering probability at depth t′ on scan line r′, which is obtained by sampling the scattering density at depth t′ on scan line r′; represents the scattering intensity at depth t′ on scan line r′; E(r, t) represents the total echo intensity at depth t on scan line r.

[0026] Further, generating a predicted ultrasound image according to the total echo intensity of each point specifically includes:

[0027] Obtaining the brightness value of the corresponding pixel according to the total echo intensity of each point;

[0028] Generating a predicted ultrasound image according to the brightness values of all pixels.

[0029] Further, when training the physical parameter prediction module, the specific calculation formula of the loss value is:

[0030] L = λ1L SS + λ2L2;

[0031] L SS = 1 - SSIM(U′(i), U(i)), L2 = ∑(U′(i) - U(i)) 2 ;

[0032] Wherein, L represents the mixed loss; λ1 represents the proportion coefficient of the structural similarity loss; λ2 represents the proportion coefficient of the mean square error; L SS represents the structural similarity loss; L2 represents the mean square error; SSIM() represents the structural similarity function; U′(i) represents the i-th frame of the predicted ultrasound image; U(i) represents the i-th frame of the real ultrasound image.

[0033] In a second aspect, the present invention provides a three-dimensional reconstruction device for cardiac ultrasound images, including:

[0034] A dataset construction unit for constructing a sample dataset; wherein, the sample dataset includes real ultrasound images of the heart at different scanning angles and the positions of each point in each frame of real ultrasound image.

[0035] A model construction unit for constructing an image reconstruction model; wherein, the image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image and generate a three-dimensional ultrasound image based on all the predicted ultrasound images.

[0036] A training and evaluation unit for training and evaluating the physical parameter prediction module in the image reconstruction model by using the sample dataset to obtain a trained image reconstruction model.

[0037] A reconstruction unit for implementing three-dimensional reconstruction of cardiac ultrasound images by using the trained image reconstruction model.

[0038] In a third aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored on the memory. The processor executes the computer program / instructions to implement the steps in the three-dimensional reconstruction method of cardiac ultrasound images as described above in this application.

[0039] In a fourth aspect, the present invention also provides a computer-readable storage medium, on which a computer program / instructions is stored. When the computer program / instructions is executed by a processor, it implements the steps in the three-dimensional reconstruction method of cardiac ultrasound images as described above in this application.

[0040] The beneficial effects of the present invention are:

[0041] The present invention deeply integrates deep learning technology with ultrasonic physical characteristics, uses the physical parameter prediction module to predict ultrasonic physical parameters from spatial coordinates, and realizes ultrasonic image prediction and generation through physical rendering, which not only ensures the accuracy of three-dimensional reconstruction of ultrasonic images, but also ensures the physical consistency of images from different perspectives, and solves the problem that traditional three-dimensional reconstruction methods are not accurate enough. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only one embodiment of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1Flowchart of the three-dimensional reconstruction method for cardiac ultrasound images in an embodiment of the present invention;

[0044] Figure 2 Block diagram of the structure of the three-dimensional reconstruction device for cardiac ultrasound images in an embodiment of the present invention;

[0045] Figure 3 Exemplary structural schematic diagram of an electronic device in an embodiment of the present invention. Detailed implementation manners

[0046] The following describes the technical solutions in the present invention clearly and completely with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0047] The following uses specific embodiments to elaborate on the technical solutions of the present application in detail. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0048] Embodiment 1

[0049] To solve the problem of insufficient accuracy in the three-dimensional reconstruction of traditional cardiac ultrasound images, the present invention provides a three-dimensional reconstruction method for cardiac ultrasound images. As Figure 1 shown, the three-dimensional reconstruction method includes the following steps:

[0050] Step S1: Construct a sample data set.

[0051] To obtain sample data, each scanning object (i.e., the heart) is scanned from multiple angles to obtain the true ultrasound images of the heart at different scanning angles. The true ultrasound images are two-dimensional images. In this embodiment, the heart is scanned at at least 6 sets of tilt angles and 1 set of vertical angles, which can better reflect the true structure of the heart and improve the accuracy of the three-dimensional reconstruction of cardiac ultrasound images. Each set of scans includes multiple frames (e.g., 150 frames) of true ultrasound images. Therefore, for each scanning object, at least 7 * 150 frames of true ultrasound images can be obtained, and the resolution of each frame of true ultrasound image is 512 × 512 pixels.

[0052] For each frame of real ultrasonic image, its pose matrix is synchronously recorded. The pose matrix of each frame of real ultrasonic image is used to describe the precise position and orientation of the ultrasonic probe in the world coordinate system. According to each frame of real ultrasonic image, its pose matrix, and the relative position of each point in this frame of real ultrasonic image, the position of each point in each frame of real ultrasonic image in the world coordinate system can be determined. The position of each point in the real ultrasonic image in the world coordinate system represents the sampling points on the ultrasonic wave propagation path. In this embodiment, a linear array ultrasonic probe is used for the ultrasonic probe.

[0053] Construct a sample data set based on the real ultrasonic images of the heart at different scanning angles and the positions of each point in each frame of real ultrasonic image in the world coordinate system, providing a data basis for the training of the subsequent image reconstruction model.

[0054] Step S2: Construct an image reconstruction model.

[0055] The image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasonic image to obtain the physical parameters of each point in the real ultrasonic image; the physical rendering module is configured to generate a predicted ultrasonic image based on the physical parameters of each point in the real ultrasonic image, and generate a three-dimensional ultrasonic image based on all the predicted ultrasonic images. The predicted ultrasonic image is a two-dimensional image.

[0056] In this embodiment, the physical parameter prediction module uses a neural network. The architecture of the neural network is an existing architecture, specifically including an input layer, a main structure, and an output layer. The input layer is used to receive an ultrasonic image (such as a real ultrasonic image) and perform position encoding on the position components of each point in the real ultrasonic image. Suppose the position of a certain point in the real ultrasonic image is (x, y, z), where x, y, and z respectively represent the abscissa, ordinate, and vertical coordinate of the point. The abscissa corresponds to the lateral scanning direction of the ultrasonic probe, the ordinate corresponds to the depth direction of the ultrasonic probe, and the vertical coordinate corresponds to the slice direction of the ultrasonic probe. Position encoding is performed on the coordinate components x, y, and z of the point respectively.

[0057] The main structure adopts an 8-layer fully connected network structure, with each layer containing 256 neurons. The activation function of the intermediate layer uses the ReLU function. The output layer contains multiple output nodes, and each output node corresponds to a physical parameter. In this embodiment, the physical parameters include attenuation coefficient, reflection coefficient, boundary probability, scattering density, and scattering intensity. Therefore, the output layer contains 5 output nodes. After being trained with a large number of ultrasonic images from different perspectives, the physical parameter prediction module has learned to implicitly express the ultrasonic physical characteristics of the corresponding region in the entire three-dimensional space, that is, to form a continuous expression:

[0058] F: (x, y, z) → (α′, β′, ρb′, ρs′, φ′) (1)

[0059] Among them, (x, y, z) represents the position of a point in the real ultrasonic image, F represents the implicit expression of the physical parameter prediction module, α' represents the original attenuation coefficient, β' represents the original reflection coefficient, ρb' represents the original boundary probability, ρs' represents the original scattering density, and φ' represents the original scattering intensity.

[0060] To achieve the positive or normalized value of the physical parameters, the output layer of the present invention also performs positive or normalized processing on the original physical parameters. Specifically, positive processing is performed on the original attenuation coefficient α', and the specific formula is:

[0061] α = |α'| (2)

[0062] Among them, α represents the attenuation coefficient after positive processing.

[0063] Normalized processing is respectively performed on the original reflection coefficient β', boundary probability ρb', scattering density ρs', and scattering intensity φ', and the specific formula is:

[0064]

[0065] Among them, g represents the reflection coefficient, boundary probability, scattering density, or scattering intensity after normalized processing; g' represents the original reflection coefficient, boundary probability, scattering density, or scattering intensity.

[0066] The physical rendering module of the present invention simulates the propagation process of ultrasonic waves in cardiac tissue based on the ray tracing principle. One emission-reception cycle of a linear array ultrasonic probe forms a scan line, and the scan line represents the propagation path of ultrasonic waves. Each scan line is defined as a ray (or ray tracing path): r(t) = o + td, where r represents the scan line or ray, o represents the position of the ultrasonic probe, that is, the starting point of the scan line or ray, d represents the propagation direction of ultrasonic waves, and t represents the depth. Each scan line corresponds to a set of scans. Therefore, each scan line contains multiple frames of real ultrasonic images, such as 150 frames of real ultrasonic images, and each frame of real ultrasonic image corresponds to a depth t. The position of points is sampled along the scan line, and the positions of these points are input into the physical parameter prediction module to obtain the physical parameters of the points.

[0067] In the specific implementation manner of the present invention, the physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, specifically including:

[0068] Step S2.1: Calculate the remaining energy of each point according to the physical parameters of each point in the real ultrasonic image, and the specific calculation formula is:

[0069]

[0070] Among them, I(r, t) represents the remaining energy at depth t on scan line r; I0 represents the initial energy; β(r, n) represents the reflection coefficient at depth n on scan line r; G(r, n) represents the boundary indication function at depth n on scan line r, which is obtained by sampling the boundary probability at depth n on scan line r (output by the physical parameter prediction module); α represents the attenuation coefficient (output by the physical parameter prediction module); f represents the frequency factor; dt represents the depth step size.

[0071] The remaining energy represents the energy remaining when the ultrasonic wave propagates to a certain point. There are reflection loss, attenuation loss, and scattering loss during the propagation of the ultrasonic wave. Therefore, the physical parameters include the attenuation coefficient, reflection coefficient, scattering density, and scattering intensity. Formula (4) reflects the cumulative attenuation effect from depth 0 to t - 1. The second term on the right side of formula (4) represents the cumulative reflection loss, and the third term represents the cumulative attenuation loss. That is, formula (4) simulates the reflection loss mechanism (represented by the product term) and the attenuation loss mechanism (represented by the exponential term) during the propagation of the ultrasonic wave. The remaining energy I(r, t) reflects how much energy remains after reflection loss and attenuation loss when the ultrasonic wave propagates to depth t.

[0072] Step S2.2: Calculate the total echo intensity of each point according to the remaining energy I(r, t) of each point and the physical parameters. The specific calculation formula is:

[0073]

[0074] E(r, t) = R(r, t) + B(r, t) (8)

[0075] Among them, R(r, t) represents the reflection energy at depth t on scan line r; PSF(r) represents the point spread function, which is an existing function; represents the convolution operation; G(r′, t′) represents the boundary indication function at depth t′ on scan line r′, which is obtained by sampling the boundary probability at depth t′ on scan line r′. Scan line r′ is the offset of scan line r, and depth t′ is the offset of depth t; B(r, t) represents the scattering energy at depth t on scan line r; T(r′, t′) represents the scattering characteristic (or scattering pattern) at depth t′ on scan line r′; H(r′, t′) represents the scattering probability at depth t′ on scan line r′, which is obtained by sampling the scattering density at depth t′ on scan line r′; represents the scattering intensity at depth t′ on scan line r′; E(r, t) represents the total echo intensity at depth t on scan line r. G(r′, t′) indicates that when performing the convolution with the point spread function, the boundary indication function is evaluated at each offset position (r′, t′) around the original point (r, t).

[0076] Step S2.3: Generate a predicted ultrasound image based on the total echo intensity E(r, t) of each point.

[0077] Sum up the total echo intensities E(r, t) of all points along the scan line in a certain integral or accumulation manner to obtain the pixel intensity corresponding to the scan line, which is the brightness value in the final B-mode ultrasound image. In ultrasound imaging, the B-mode (Brightness Mode) is a commonly used two-dimensional imaging method. Its main principle is as follows: The ultrasound probe emits high-frequency sound waves into the human tissue. When the sound waves encounter the interfaces of different media (such as soft tissues, liquids, or bones), reflection and scattering will occur; the probe receives the reflected ultrasonic signals (i.e., echo signals), and the amplitudes (intensities) of these echo signals are related to the physical properties of the tissue (such as density and acoustic impedance differences). The B-mode converts the intensity of each echo signal into the brightness value of the corresponding pixel on the image. Usually, the stronger the echo signal, the brighter the corresponding pixel (higher gray-scale value), and vice versa. The finally generated image is a grayscale image, showing the different reflection characteristics of the tissue structure.

[0078] After training, the image reconstruction model can generate B-mode ultrasound images from any position and direction. That is, given a new pose of the ultrasound probe, scan lines are emitted from this position, and a two-dimensional ultrasound image is generated using the physical parameter prediction and physical rendering processes of the image reconstruction model, namely, the predicted ultrasound image is obtained. The image reconstruction model can not only generate ultrasound images with "new perspectives", but also ensure the consistency of ultrasound images from different perspectives because the physical properties of the entire three-dimensional space of the heart are uniformly modeled.

[0079] The two-dimensional predicted ultrasound image contains the physical parameters of each point. The physical parameters of each sampling point in the entire heart region are obtained based on the physical parameters of each point in all predicted ultrasound images. Since the predicted ultrasound images are ultrasound images from different perspectives, a dense three-dimensional ultrasound physical parameter field is constructed. According to the three-dimensional ultrasound physical parameter field, the isosurface is extracted using a post-processing algorithm (such as the Marching Cubes algorithm), and then the three-dimensional shape of the heart can be reconstructed, that is, the three-dimensional ultrasound image is obtained.

[0080] Step S3: Use the sample data set to train and evaluate the physical parameter prediction module in the image reconstruction model to obtain the trained image reconstruction model.

[0081] Input the real ultrasound image and the positions of each point in the real ultrasound image into the physical parameter prediction module. The physical parameter prediction module extracts features from the positions of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module generates the corresponding predicted ultrasound image according to the physical parameters of each point in the real ultrasound image. Calculate the loss value based on the corresponding predicted ultrasound image and the real ultrasound image, and realize the training of the physical parameter prediction module through backpropagation of the loss value. During the training process, only the trainable parameters of the physical parameter prediction module are updated. The specific calculation formula of the loss value is:

[0082] L = λ1L SS + λ2L2 (9)

[0083] L SS = 1 - SSIM(U′(i), U(i)) (10)

[0084] L2 = ∑(U′(i) - U(i)) 2 (11)

[0085] Among them, L represents the mixed loss; λ1 represents the proportion coefficient of the structural similarity loss, and the value in this embodiment is 0.9; λ2 represents the proportion coefficient of the mean square error, and the value in this embodiment is 0.1; L SS represents the structural similarity loss; L2 represents the mean square error; SSIM() represents the structural similarity function, which is used to calculate the brightness, contrast and structural similarity of the image; U′(i) represents the i-th frame of the predicted ultrasound image; U(i) represents the i-th frame of the real ultrasound image.

[0086] The specific settings of the training parameters are as follows: the batch size is 4, the number of training epochs is 50, the initial value of the learning rate is 1e -4 , and the learning rate decays to 0.5 times the original every 20 epochs. The Adam optimizer is used.

[0087] The evaluation of the image reconstruction model includes quantitative evaluation and qualitative evaluation. The quantitative evaluation indicators include SSIM score, reconstruction consistency and geometric accuracy. The SSIM score is used to calculate the structural similarity between the predicted ultrasound image and the real ultrasound image. The reconstruction consistency is used to calculate the standard deviation of the prediction results at different scanning angles. The geometric accuracy is used to calculate the position error of the key anatomical landmark points. The qualitative evaluation includes visual comparison, parameter analysis and three-dimensional visualization. Visual comparison can realize the intuitive comparison between the predicted ultrasound image and the real ultrasound image. Draw the spatial distribution map of each physical parameter for parameter analysis, and draw the three-dimensional ultrasound image to realize three-dimensional visualization. Through the evaluation of the image reconstruction model, the reconstruction quality of the three-dimensional ultrasound image is guaranteed, which can guide the performance improvement of the image reconstruction model.

[0088] Step S4: Use the trained image reconstruction model to achieve 3D reconstruction of cardiac ultrasound images.

[0089] Input the ultrasound images at different scanning angles and the positions of each point in the ultrasound images into the trained image reconstruction model, and the corresponding 3D ultrasound image can be obtained, achieving 3D reconstruction of cardiac ultrasound images.

[0090] The present invention innovatively introduces a physical parameter prediction module to accurately predict the acoustic characteristics of cardiac tissues. The rendering algorithm based on ray tracing ensures that the images conform to the laws of ultrasound physics, achieving highly realistic ultrasound image generation and 3D reconstruction.

[0091] The physical parameter prediction module uses a neural network. The implicit neural representation realizes continuous spatial sampling ability and has strong generalization performance, being applicable to different anatomical structures.

[0092] The physical rendering module ensures the consistency of images at different scanning angles. The reconstruction results completely retain the acoustic characteristic information of cardiac tissues and support the generation of ultrasound images at any new scanning angle. The rendering process is completely based on the principles of ultrasound physics, and the reconstruction results have good physical interpretability, supporting parameter-level analysis and verification of the reconstruction results, with stable and reliable performance.

[0093] Embodiment 2

[0094] As Figure 2 shown, the 3D reconstruction device for cardiac ultrasound images provided by the embodiment of the present invention includes a dataset construction unit, a model construction unit, a training and evaluation unit, and a reconstruction unit.

[0095] The dataset construction unit is used to construct a sample dataset. Among them, the sample dataset includes the real ultrasound images of the heart at different scanning angles and the positions of each point in each frame of real ultrasound image.

[0096] The model construction unit is used to construct an image reconstruction model. Among them, the image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module is configured to generate a predicted ultrasound image according to the physical parameters of each point in the real ultrasound image and generate a 3D ultrasound image according to all the predicted ultrasound images.

[0097] The training and evaluation unit is used to train and evaluate the physical parameter prediction module in the image reconstruction model using the sample dataset to obtain the trained image reconstruction model.

[0098] The reconstruction unit is used to use the trained image reconstruction model to achieve 3D reconstruction of cardiac ultrasound images.

[0099] In some embodiments, the three-dimensional reconstruction device for cardiac ultrasound images may incorporate the features of the three-dimensional reconstruction method for cardiac ultrasound images in Embodiment 1 of the present application, and vice versa, which will not be elaborated herein.

[0100] Embodiment 3

[0101] The embodiment of the present invention further provides an electronic device, as Figure 3 shown, the electronic device includes: a memory, a processor, and a computer program / instructions stored on the memory, and the processor executes the computer program / instructions to implement the three-dimensional reconstruction method for cardiac ultrasound images in Embodiment 1 of the present application.

[0102] Although not shown, the electronic device includes a processor, which can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) and / or the programs and / or data loaded from the storage section into the random access memory (RAM). The processor can be a multi-core processor or include multiple processors. In some embodiments, the processor may include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), and so on. In the RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0103] The above-mentioned processor and memory are jointly used to execute the programs / instructions stored in the memory, and when the programs / instructions are executed by a computer, they can implement the methods, steps, or functions described in the above embodiments.

[0104] Although not shown, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program / instructions are stored, and when the computer program / instructions are executed by a processor, they implement the three-dimensional reconstruction method for cardiac ultrasound images in Embodiment 1 of the present application.

[0105] A readable storage medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media do not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0106] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A three-dimensional reconstruction method for cardiac ultrasound images, characterized in that, The three-dimensional reconstruction method includes: Constructing a sample data set; wherein, the sample data set includes real ultrasonic images of the heart at different scanning angles and the positions of each point in each frame of real ultrasonic image; Constructing an image reconstruction model; wherein, the image reconstruction model includes a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasonic image to obtain the physical parameters of each point in the real ultrasonic image; the physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, and generate a three-dimensional ultrasonic image according to all the predicted ultrasonic images; Training and evaluating the physical parameter prediction module in the image reconstruction model by using the sample data set to obtain a trained image reconstruction model; Implementing three-dimensional reconstruction of cardiac ultrasound images by using the trained image reconstruction model.

2. The three-dimensional reconstruction method of cardiac ultrasound images according to claim 1, wherein The physical parameter prediction module adopts a neural network.

3. The three-dimensional reconstruction method of cardiac ultrasound images according to claim 1, wherein The physical parameters include attenuation coefficient, reflection coefficient, boundary probability, scattering density, and scattering intensity.

4. The three-dimensional reconstruction method of cardiac ultrasound images according to claim 3, wherein The physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, specifically including: Calculating the remaining energy of each point according to the physical parameters of each point in the real ultrasonic image; Calculating the total echo intensity of each point according to the remaining energy and physical parameters of each point; Generating a predicted ultrasonic image according to the total echo intensity of each point.

5. The three-dimensional reconstruction method of cardiac ultrasound images according to claim 4, wherein, The calculation formula of the remaining energy is: wherein, I(r, t) represents the remaining energy at the depth t on the scan line r; I0 represents the initial energy; β(r, n) represents the reflection coefficient at the depth n on the scan line r; G(r, n) represents the boundary indication function at the depth n on the scan line r, which is obtained by sampling the boundary probability at the depth n on the scan line r; α represents the attenuation coefficient; f represents the frequency factor; dt represents the depth step; The calculation formula of the total echo intensity is: E(r, t) = R(r, t) + B(r, t); Among them, R(r, t) represents the reflection energy at depth t on scan line r; PSF(r) represents the point spread function; represents the convolution operation; G(r′, t′) represents the boundary indication function at depth t′ on scan line r′, where scan line r′ is the offset of scan line r and depth t′ is the offset of depth t; B(r, t) represents the scattering energy at depth t on scan line r; T(r′, t′) represents the scattering characteristic at depth t′ on scan line r′; H(r', t′) represents the scattering probability at depth t′ on scan line r′, which is obtained by sampling the scattering density at depth t′ on scan line r′; represents the scattering intensity at depth t′ on scan line r′; E(r, t) represents the total echo intensity at depth t on scan line r.

6. The three-dimensional reconstruction method of cardiac ultrasound images according to claim 4, characterized in that The generating of the predicted ultrasonic image according to the total echo intensity of each point specifically includes: Obtaining the brightness value of the corresponding pixel according to the total echo intensity of each point; Generating a predicted ultrasonic image according to the brightness values of all pixels.

7. The three-dimensional reconstruction method of cardiac ultrasound images according to any one of claims 1 to 6, characterized in that, When training the physical parameter prediction module, the specific calculation formula of the loss value is: L = λ1L SS + λ2L2; L SS = 1 - SSIM(U′(i), U(i)), L2 = ∑(U′(i) - U(i)) 2 ; Among them, L represents the hybrid loss; λ1 represents the proportion coefficient of the structural similarity loss; λ2 represents the proportion coefficient of the mean square error; L ss represents the structural similarity loss; L2 represents the mean square error; SSIM() represents the structural similarity function; U′(i) represents the predicted ultrasound image of the i-th frame; U(i) represents the true ultrasound image of the i-th frame.

8. A three-dimensional reconstruction device for cardiac ultrasound images, characterized in that, The three-dimensional reconstruction device includes: A data set construction unit for constructing a sample data set; wherein, the sample data set includes real ultrasonic images of the heart at different scanning angles and the positions of each point in each frame of real ultrasonic image; A model construction unit for constructing an image reconstruction model; wherein, the image reconstruction model includes a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features from the positions of each point in the real ultrasonic image to obtain the physical parameters of each point in the real ultrasonic image; the physical rendering module is configured to generate a predicted ultrasonic image according to the physical parameters of each point in the real ultrasonic image, and generate a three-dimensional ultrasonic image according to all the predicted ultrasonic images; A training and evaluation unit for training and evaluating the physical parameter prediction module in the image reconstruction model by using the sample data set to obtain a trained image reconstruction model; A reconstruction unit for implementing three-dimensional reconstruction of cardiac ultrasound images by using the trained image reconstruction model.

9. An electronic device, comprising a memory, a processor, and a computer program / instructions stored on the memory, characterized in that, The processor executes the computer program / instructions to implement the steps in the three-dimensional reconstruction method of cardiac ultrasound images according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, the steps in the three-dimensional reconstruction method of cardiac ultrasound images according to any one of claims 1 to 7 are implemented.

Images