Heart ultrasound image three-dimensional reconstruction method, device and equipment and storage medium

Abstract

The application discloses a heart ultrasound image three-dimensional reconstruction method, device and equipment and a storage medium, and relates to the technical field of image processing. The three-dimensional reconstruction method comprises the following steps: constructing a sample data set; constructing an image reconstruction model, wherein the image reconstruction model comprises a physical parameter prediction module and a physical rendering module, the physical parameter prediction module extracts the features of the positions of each point in a real ultrasound image, and obtains the physical parameters of each point in the real ultrasound image; the physical rendering module generates a predicted ultrasound image according to the physical parameters of each point in the real ultrasound image, and generates a three-dimensional ultrasound image according to all the predicted ultrasound images; the physical parameter prediction module in the image reconstruction model is trained and evaluated by using the sample data set, and a trained image reconstruction model is obtained; and the three-dimensional reconstruction of the heart ultrasound image is realized by using the trained image reconstruction model. The application solves the problem that the traditional three-dimensional reconstruction method is not accurate enough.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for three-dimensional reconstruction of cardiac ultrasound images. Background Technology

[0002] Ultrasound imaging is a widely used medical imaging technique in clinical diagnosis. Due to its advantages such as portability, real-time performance, and lack of radiation, ultrasound plays a vital role in various clinical applications. Currently, ultrasound equipment used in cardiovascular disease clinics primarily provides two-dimensional images of a single cross-section. However, the heart itself is a three-dimensional structure. In actual diagnosis, physicians need to use long-term training and clinical experience to mentally fuse these two-dimensional images into a three-dimensional anatomical model. This subjective fusion is easily limited by personal experience and carries a significant risk of error.

[0003] Currently, three-dimensional reconstruction of cardiac ultrasound presents the following technical challenges:

[0004] 1) Due to the different propagation characteristics of ultrasound waves in different tissues, the same anatomical structure may present different image appearances under different probe positions. This viewpoint dependence makes 3D reconstruction more difficult, and conventional 3D reconstruction methods are unable to accurately reproduce the true structure of the tissue.

[0005] 2) Existing 3D ultrasound reconstruction mainly relies on dedicated swing probes, 2D probe arrays, or probes with position tracking to synthesize 3D images. Although this method can obtain 3D data, the reconstruction quality is not ideal. For example, unreasonable acoustic shadows may appear when observing the reconstructed volume from different perspectives; anatomical structures that are actually impossible for ultrasound to reach may be displayed at certain perspectives; and the lack of accurate modeling of the physical properties of ultrasound leads to unrealistic reconstruction results.

[0006] In conclusion, there is an urgent need for a more advanced ultrasound 3D reconstruction method that takes into account the physical characteristics of ultrasound imaging and can accurately reconstruct 3D structures from sparse 2D images, providing more realistic and reliable 3D ultrasound imaging results and offering more accurate information for clinical diagnosis and treatment. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method, apparatus, device, and storage medium for three-dimensional reconstruction of cardiac ultrasound images. It deeply integrates deep learning technology with ultrasound physical characteristics, uses deep learning technology to predict ultrasound physical parameters from spatial coordinates, and achieves ultrasound image prediction through physical rendering, ensuring the accuracy of three-dimensional ultrasound images and the physical consistency of images from different perspectives.

[0008] In a first aspect, the present invention provides a method for three-dimensional reconstruction of cardiac ultrasound images, comprising:

[0009] Construct a sample dataset; wherein, the sample dataset includes real ultrasound images of the heart at different scanning angles and the position of each point in each frame of the real ultrasound image;

[0010] An image reconstruction model is constructed; wherein the image reconstruction model includes a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features of the position of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image; the physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, and generate a three-dimensional ultrasound image based on all the predicted ultrasound images.

[0011] The physical parameter prediction module in the image reconstruction model is trained and evaluated using the sample dataset to obtain the trained image reconstruction model.

[0012] Three-dimensional reconstruction of cardiac ultrasound images is achieved using a trained image reconstruction model.

[0013] Furthermore, the physical parameter prediction module employs a neural network.

[0014] Furthermore, the physical parameters include attenuation coefficient, reflection coefficient, boundary probability, scattering density, and scattering intensity.

[0015] Furthermore, the physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, specifically including:

[0016] Calculate the remaining energy at each point based on the physical parameters of each point in the real ultrasound image;

[0017] Calculate the total echo intensity at each point based on the remaining energy and physical parameters at each point;

[0018] A predictive ultrasound image is generated based on the total echo intensity at each point.

[0019] Furthermore, the formula for calculating the remaining energy is as follows:

[0020]

[0021] Where I(r,t) represents the remaining energy at depth t on scan line r; I0 represents the initial energy; β(r,n) represents the reflection coefficient at depth n on scan line r; G(r,n) represents the boundary indicator function at depth n on scan line r, obtained by boundary probability sampling at depth n on scan line r; α represents the attenuation coefficient; f represents the frequency factor; and dt represents the depth step size.

[0022] The formula for calculating the total echo intensity is:

[0023]

[0024] E(r,t)=R(r,t)+B(r,t);

[0025] Where R(r,t) represents the reflected energy at depth t on scan line r; PSF(r) represents the point spread function; The convolution operation is represented by G(r′, t′); G(r′, t′) represents the boundary indicator function at depth t′ on scan line r′, where scan line r′ is the offset of scan line r and depth t′ is the offset of depth t; B(r, t) represents the scattering energy at depth t on scan line r; r(r′, t′) represents the scattering characteristics at depth t′ on scan line r′; H(r′, t′) represents the scattering probability at depth t′ on scan line r′, obtained by sampling the scattering density at depth t′ on scan line r′. E(r,t) represents the scattering intensity at a depth of t′ on the scan line r′; E(r,t) represents the total echo intensity at a depth of t on the scan line r.

[0026] Furthermore, the step of generating a predicted ultrasound image based on the total echo intensity at each point specifically includes:

[0027] The brightness value of the corresponding pixel is obtained based on the total echo intensity of each point;

[0028] A predictive ultrasound image is generated based on the brightness values ​​of all pixels.

[0029] Furthermore, during the training of the physical parameter prediction module, the specific formula for calculating the loss value is as follows:

[0030] L=λ1L SS +λ2L2;

[0031] L SS =1-SSIM(U′(i),U(i)), L2=∑(U′(i)-U(i)) 2 ;

[0032] Where L represents the mixed loss; λ1 represents the proportion coefficient of structural similarity loss; λ2 represents the proportion coefficient of mean square error; L SS denoted by ; L2 represents the mean squared error; SSIM() represents the structural similarity function; U′(i) represents the predicted ultrasound image of the i-th frame; U(i) represents the actual ultrasound image of the i-th frame.

[0033] In a second aspect, the present invention provides a three-dimensional reconstruction device for cardiac ultrasound images, comprising:

[0034] A dataset construction unit is used to construct a sample dataset; wherein, the sample dataset includes real ultrasound images of the heart at different scanning angles and the position of each point in each frame of the real ultrasound image;

[0035] A model building unit is used to build an image reconstruction model. The image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features of the position of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, and generate a three-dimensional ultrasound image based on all the predicted ultrasound images.

[0036] The training and evaluation unit is used to train and evaluate the physical parameter prediction module in the image reconstruction model using the sample dataset, so as to obtain the trained image reconstruction model.

[0037] The reconstruction unit is used to realize the three-dimensional reconstruction of cardiac ultrasound images using a trained image reconstruction model.

[0038] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the steps in the cardiac ultrasound image three-dimensional reconstruction method described above in this application.

[0039] Fourthly, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps in the cardiac ultrasound image three-dimensional reconstruction method described above.

[0040] The beneficial effects of this invention are:

[0041] This invention deeply integrates deep learning technology with ultrasound physical properties. It uses a physical parameter prediction module to predict ultrasound physical parameters from spatial coordinates and realizes ultrasound image prediction and generation through physical rendering. This ensures both the accuracy of ultrasound image 3D reconstruction and the physical consistency of images from different perspectives, solving the problem of inaccuracy in traditional 3D reconstruction methods. Attached Figure Description

[0042] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1This is a flowchart of the three-dimensional reconstruction method of cardiac ultrasound images in an embodiment of the present invention;

[0044] Figure 2 This is a structural block diagram of the cardiac ultrasound image three-dimensional reconstruction device in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of an exemplary structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0046] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0048] Example 1

[0049] To address the issue of insufficient accuracy in three-dimensional reconstruction of traditional cardiac ultrasound images, this invention provides a method for three-dimensional reconstruction of cardiac ultrasound images. For example... Figure 1 As shown, the three-dimensional reconstruction method includes the following steps:

[0050] Step S1: Construct the sample dataset.

[0051] To obtain sample data, each scanned object (i.e., the heart) is scanned from multiple angles to obtain real ultrasound images of the heart at different scanning angles. These real ultrasound images are two-dimensional. In this embodiment, the heart is scanned at at least six sets of tilt angles and one set of vertical angles to better reflect the true structure of the heart and improve the accuracy of three-dimensional reconstruction of cardiac ultrasound images. Each set of scans contains multiple frames (e.g., 150 frames) of real ultrasound images; therefore, for each scanned object, at least 7*150 frames of real ultrasound images can be obtained, with each frame having a resolution of 512×512 pixels.

[0052] For each frame of a real ultrasound image, its pose matrix is ​​recorded synchronously. The pose matrix of each frame of a real ultrasound image is used to describe the precise position and orientation of the ultrasound probe in the world coordinate system. Based on each frame of a real ultrasound image, its pose matrix, and the relative position of each point in that frame of a real ultrasound image, the position of each point in the world coordinate system in each frame of a real ultrasound image can be determined. The position of each point in the real ultrasound image in the world coordinate system represents a sampling point on the ultrasound propagation path. In this embodiment, a linear array ultrasound probe is used.

[0053] A sample dataset is constructed based on real ultrasound images of the heart at different scanning angles and the position of each point in each real ultrasound image in the world coordinate system, providing a data foundation for the training of subsequent image reconstruction models.

[0054] Step S2: Construct an image reconstruction model.

[0055] The image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features from the location of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, and then generate a three-dimensional ultrasound image based on all the predicted ultrasound images. The predicted ultrasound image is a two-dimensional image.

[0056] In this embodiment, the physical parameter prediction module employs a neural network. The neural network architecture is an existing architecture, specifically including an input layer, a main structure, and an output layer. The input layer receives ultrasound images (e.g., real ultrasound images) and encodes the position components of each point in the real ultrasound image. Let the position of a point in the real ultrasound image be (x, y, z), where x, y, and z represent the horizontal, vertical, and abscissa coordinates of the point, respectively. The horizontal coordinate corresponds to the lateral scanning direction of the ultrasound probe, the vertical coordinate corresponds to the depth direction of the ultrasound probe, and the vertical coordinate corresponds to the slicing direction of the ultrasound probe. The position components x, y, and z of the point are then encoded.

[0057] The main structure employs an 8-layer fully connected network, with each layer containing 256 neurons. The ReLU activation function is used in the intermediate layers. The output layer contains multiple output nodes, each corresponding to a physical parameter. In this embodiment, the physical parameters include attenuation coefficient, reflection coefficient, boundary probability, scattering density, and scattering intensity; therefore, the output layer contains 5 output nodes. After training with a large number of ultrasound images from different perspectives, the physical parameter prediction module learns to implicitly express the ultrasound physical characteristics of the corresponding region throughout the entire three-dimensional space, forming a continuous expression:

[0058] F: (x, y, z)→(α′, β′, ρb′, ρs′, φ′) (1)

[0059] Where (x, y, z) represents the position of a point in the real ultrasound image, F represents the implicit expression of the physical parameter prediction module, α′ represents the original attenuation coefficient, β′ represents the original reflection coefficient, ρb′ represents the original boundary probability, ρs′ represents the original scattering density, and φ′ represents the original scattering intensity.

[0060] To achieve positive or normalized physical parameters, the output layer of this invention also performs positive or normalized processing on the original physical parameters. Specifically, the original attenuation coefficient α′ is positiveized using the following formula:

[0061] α=|α′| (2)

[0062] Where α represents the attenuation coefficient after positiveization.

[0063] The original reflection coefficient β′, boundary probability ρb′, scattering density ρs′, and scattering intensity φ′ are normalized respectively, and the specific formulas are as follows:

[0064]

[0065] Where g represents the normalized reflection coefficient, boundary probability, scattering density, or scattering intensity; g′ represents the original reflection coefficient, boundary probability, scattering density, or scattering intensity.

[0066] The physical rendering module of this invention simulates the propagation of ultrasound waves in heart tissue based on the principle of ray tracing. A linear ultrasound probe's transmit-receive cycle forms a scan line, representing the propagation path of the ultrasound wave. Each scan line is defined as a ray (or ray tracing path): r(t) = o + td, where r represents the scan line or ray, o represents the position of the ultrasound probe (i.e., the starting point of the scan line or ray), d represents the direction of ultrasound propagation, and t represents the depth. Each scan line corresponds to a set of scans; therefore, each scan line contains multiple frames of real ultrasound images, for example, 150 frames of real ultrasound images, each frame corresponding to a depth t. The positions of points are sampled along the scan line, and these positions are input into the physical parameter prediction module to obtain the physical parameters of the points.

[0067] In a specific embodiment of the present invention, the physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, specifically including:

[0068] Step S2.1: Calculate the remaining energy at each point based on the physical parameters of each point in the real ultrasound image. The specific calculation formula is as follows:

[0069]

[0070] Where I(r,t) represents the remaining energy at depth t on scan line r; I0 represents the initial energy; β(r,n) represents the reflection coefficient at depth n on scan line r; G(r,n) represents the boundary indicator function at depth n on scan line r, which is obtained by sampling the boundary probability at depth n on scan line r (output by the physical parameter prediction module); α represents the attenuation coefficient (output by the physical parameter prediction module); f represents the frequency factor; and dt represents the depth step size.

[0071] Residual energy represents the energy remaining when an ultrasonic wave propagates to a certain point. During propagation, ultrasonic waves experience reflection loss, attenuation loss, and scattering loss. Therefore, physical parameters include attenuation coefficient, reflection coefficient, scattering density, and scattering intensity. Equation (4) reflects the cumulative attenuation effect from depth 0 to depth t-1. The second term on the right side of Equation (4) represents the cumulative reflection loss, and the third term represents the cumulative attenuation loss. That is, Equation (4) simulates the reflection loss mechanism (represented by the multiplication term) and the attenuation loss mechanism (represented by the exponential term) during ultrasonic wave propagation. Residual energy I(r,t) reflects how much energy remains after reflection loss and attenuation loss when the ultrasonic wave propagates to depth t.

[0072] Step S2.2: Calculate the total echo intensity at each point based on the remaining energy I(r,t) and physical parameters. The specific calculation formula is as follows:

[0073]

[0074] E(r,t)=R(r,t)+B(r,t)(8)

[0075] Where R(r,t) represents the reflected energy at depth t on scan line r; PSF(r) represents the point spread function, which is an existing function; The convolution operation is represented by G(r′, t′); G(r′, t′) represents the boundary indicator function at depth t′ on scan line r′, obtained by sampling the boundary probability at depth t′ on scan line r′, where scan line r′ is the offset of scan line r and depth t′ is the offset of depth t; B(r, t) represents the scattering energy at depth t on scan line r; T(r′, t′) represents the scattering characteristics (or scattering map) at depth t′ on scan line r′; H(r′, t′) represents the scattering probability at depth t′ on scan line r′, obtained by sampling the scattering density at depth t′ on scan line r′. Let represent the scattering intensity at a depth of t′ on scan line r′; E(r,t) represents the total echo intensity at a depth of t on scan line r. G(r′,t′) indicates the boundary indicator function evaluated at each offset position (r′,t′) around the original point (r,t) when performing convolution with the point spread function.

[0076] Step S2.3: Generate a predicted ultrasound image based on the total echo intensity E(r,t) at each point.

[0077] The total echo intensity E(r, t) of all points along the scan line is summed using a certain integration or accumulation method to obtain the pixel intensity of the corresponding scan line, which is the brightness value in the final B-mode ultrasound image. In ultrasound imaging, B-mode (Brightness Mode) is a commonly used two-dimensional imaging method. Its main principle is as follows: The ultrasound probe emits high-frequency sound waves into human tissue. When the sound waves encounter the interface of different media (such as soft tissue, liquid, or bone), reflection and scattering occur. The probe receives the reflected ultrasound signals (i.e., echo signals). The amplitude (intensity) of these echo signals is related to the physical properties of the tissue (such as density and acoustic impedance differences). B-mode converts the intensity of each echo signal into the brightness value of the corresponding pixel on the image. Generally, the stronger the echo signal, the brighter the corresponding pixel (higher grayscale value), and vice versa. The final generated image is a grayscale image that shows the different reflective characteristics of the tissue structure.

[0078] After training, the image reconstruction model can generate B-mode ultrasound images from any position and orientation. That is, given a new ultrasound probe pose, a scan line is emitted from that position, and a two-dimensional ultrasound image is generated using the physical parameters predicted and the physical rendering process of the image reconstruction model—this is the predicted ultrasound image. The image reconstruction model not only generates ultrasound images from "new perspectives," but also ensures consistency of ultrasound images from various perspectives because the physical properties of the entire three-dimensional space of the heart are modeled uniformly.

[0079] Two-dimensional predictive ultrasound images contain the physical parameters of each point. Based on the physical parameters of each point in all predictive ultrasound images, the physical parameters of each sampling point in the entire heart region are obtained. Since each predictive ultrasound image is an ultrasound image from a different perspective, a dense three-dimensional ultrasound physical parameter field is constructed. Based on the three-dimensional ultrasound physical parameter field, the isosurface is extracted using a post-processing algorithm (such as the Marching Cubes algorithm), and the three-dimensional morphology of the heart can be reconstructed, thus obtaining a three-dimensional ultrasound image.

[0080] Step S3: Use the sample dataset to train and evaluate the physical parameter prediction module in the image reconstruction model to obtain the trained image reconstruction model.

[0081] The physical parameter prediction module inputs a real ultrasound image and the position of each point within that image. This module extracts features from the position of each point in the real ultrasound image to obtain the physical parameters for that point. The physical rendering module then generates a corresponding predicted ultrasound image based on these physical parameters. A loss value is calculated between the predicted and real ultrasound images, and this loss value is used for backpropagation to train the physical parameter prediction module. During training, only the trainable parameters of the physical parameter prediction module are updated. The specific formula for calculating the loss value is as follows:

[0082] L=λ1L SS +λ2L2 (9)

[0083] L SS =1-SSIM(U′(i),U(i)) (10)

[0084] L2=∑(U′(i)-U(i)) 2 (11)

[0085] Where L represents the hybrid loss; λ1 represents the proportion coefficient of structural similarity loss, which is 0.9 in this embodiment; λ2 represents the proportion coefficient of mean square error, which is 0.1 in this embodiment; L SS L2 represents the structural similarity loss; SSIM() represents the structural similarity function, used to calculate the brightness, contrast, and structural similarity of an image; U′(i) represents the predicted ultrasound image of the i-th frame; U(i) represents the actual ultrasound image of the i-th frame.

[0086] The specific training parameters are set as follows: batch size of 4, training epochs of 50, and initial learning rate of 1e. -4 The learning rate decays to 0.5 times its original value every 20 rounds, using the Adam optimizer.

[0087] The evaluation of image reconstruction models includes quantitative and qualitative assessments. Quantitative assessment metrics include the SSIM score, reconstruction consistency, and geometric accuracy. The SSIM score calculates the structural similarity between the predicted and actual ultrasound images. Reconstruction consistency calculates the standard deviation of the prediction results at different scanning angles. Geometric accuracy calculates the positional errors of key anatomical landmarks. Qualitative assessments include visual comparison, parametric analysis, and 3D visualization. Visual comparison allows for an intuitive comparison between the predicted and actual ultrasound images. Spatial distribution maps of various physical parameters are plotted for parametric analysis. 3D ultrasound images are then visualized. The evaluation of image reconstruction models ensures the reconstruction quality of 3D ultrasound images and can guide performance improvements of the image reconstruction model.

[0088] Step S4: Use the trained image reconstruction model to achieve three-dimensional reconstruction of cardiac ultrasound images.

[0089] By inputting ultrasound images from different scanning angles and the position of each point in the ultrasound images into the trained image reconstruction model, the corresponding three-dimensional ultrasound images can be obtained, thus realizing the three-dimensional reconstruction of cardiac ultrasound images.

[0090] This invention innovatively introduces a physical parameter prediction module to accurately predict the acoustic properties of cardiac tissue. The ray tracing-based rendering algorithm ensures that the image conforms to the physical laws of ultrasound, realizing highly realistic ultrasound image generation and three-dimensional reconstruction.

[0091] The physical parameter prediction module uses a neural network. The implicit neural representation enables continuous spatial sampling and has strong generalization performance, making it suitable for different anatomical structures.

[0092] The physical rendering module ensures the consistency of images under different scanning angles, and the reconstruction results fully preserve the acoustic characteristics of the heart tissue. It supports the generation of ultrasound images from any new scanning angle. The rendering process is based entirely on ultrasound physics principles, and the reconstruction results have good physical interpretability. It supports parameter-level analysis and verification of the reconstruction results, and its performance is stable and reliable.

[0093] Example 2

[0094] like Figure 2 As shown, the cardiac ultrasound image three-dimensional reconstruction device provided in this embodiment of the invention includes a dataset construction unit, a model construction unit, a training and evaluation unit, and a reconstruction unit.

[0095] The dataset construction unit is used to construct the sample dataset; the sample dataset includes real ultrasound images of the heart at different scanning angles and the position of each point in each frame of the real ultrasound image.

[0096] The model building unit is used to build an image reconstruction model. The image reconstruction model includes a physical parameter prediction module and a physical rendering module. The physical parameter prediction module is configured to extract features of the position of each point in the real ultrasound image to obtain the physical parameters of each point in the real ultrasound image. The physical rendering module is configured to generate a predicted ultrasound image based on the physical parameters of each point in the real ultrasound image, and generate a three-dimensional ultrasound image based on all the predicted ultrasound images.

[0097] The training and evaluation unit is used to train and evaluate the physical parameter prediction module in the image reconstruction model using the sample dataset, so as to obtain the trained image reconstruction model.

[0098] The reconstruction unit is used to realize the three-dimensional reconstruction of cardiac ultrasound images using a trained image reconstruction model.

[0099] In some embodiments, the cardiac ultrasound image three-dimensional reconstruction device can incorporate the features of the cardiac ultrasound image three-dimensional reconstruction method in Embodiment 1 of this application, and vice versa, which will not be elaborated here.

[0100] Example 3

[0101] This invention also provides an electronic device, such as... Figure 3 As shown, the electronic device includes: a memory, a processor, and a computer program / instructions stored in the memory. The processor executes the computer program / instructions to implement the three-dimensional reconstruction method of cardiac ultrasound images in Embodiment 1 of this application.

[0102] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) or loaded from a storage portion into random access memory (RAM). The processor can be a multi-core processor or may contain multiple processors. In some embodiments, the processor may include a general-purpose main processor and one or more specialized coprocessors, such as a central processing unit, graphics processing unit (GPU), neural network processor (NPU), digital signal processor (DSP), etc. Various programs and data required for device operation are also stored in RAM. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0103] The processor and memory described above are used together to execute programs / instructions stored in the memory. When the program / instructions are executed by the computer, they can implement the methods, steps, or functions described in the above embodiments.

[0104] Although not shown, embodiments of the present invention also provide a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the three-dimensional reconstruction method of cardiac ultrasound images in Embodiment 1 of this application.

[0105] Readable storage media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0106] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for three-dimensional reconstruction of cardiac ultrasound images, characterized in that, The three-dimensional reconstruction method comprises: constructing a sample data set; wherein the sample data set comprises real ultrasound images of a heart at different scanning angles and positions of each point in each frame of real ultrasound images; constructing an image reconstruction model; wherein the image reconstruction model comprises a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features of the positions of each point in the real ultrasound images to obtain physical parameters of each point in the real ultrasound images, the physical parameters comprising an attenuation coefficient, a reflection coefficient, a boundary probability, a scattering density and a scattering intensity; the physical rendering module is configured to calculate a residual energy and a total echo intensity of each point based on a ray tracing principle according to the physical parameters of each point in the real ultrasound images, generate a predicted ultrasound image according to the total echo intensities of all points, and further generate a three-dimensional ultrasound image according to all predicted ultrasound images; the calculation formula of the residual energy is: ； wherein, represents a scan line above and with a depth of the remaining energy at; represents an initial energy; represents a scan line above and with a depth of the reflection coefficient at; represents a scan line above and with a depth of the boundary indicator function at; by sampling the boundary probability at above and with a depth of ; and represents an attenuation coefficient; represents a frequency factor; represents a depth step size; the calculation formula of the total echo intensity is: ； ； ； in, Indicates scan line Above and depth is Reflected energy at the location; Represents the point spread function; This represents the convolution operation; Indicates scan line Above and depth is Boundary indicator function, scan line For scan lines offset, depth For depth The offset; Indicates scan line Above and depth is Scattered energy at that location; Indicates scan line Above and depth is Scattering characteristics at that location; Indicates scan line Above and depth is The scattering probability at that point, through the scan line Above and depth is The scattering density at that location was obtained by sampling. Indicates scan line Above and depth is Scattering intensity at that location; Indicates scan line Above and depth is Total echo intensity at the location; training and evaluating the physical parameter prediction module in the image reconstruction model by using the sample data set to obtain a trained image reconstruction model; realizing three-dimensional reconstruction of a heart ultrasound image by using the trained image reconstruction model.

2. The method of claim 1, wherein, The physical parameter prediction module adopts a neural network.

3. The method of claim 1, wherein, The physical rendering module is specifically configured to: calculate the residual energy of each point based on a ray tracing principle according to the physical parameters of each point in the real ultrasound images; calculate the total echo intensity of each point according to the residual energy and the physical parameters of each point; generate a predicted ultrasound image according to the total echo intensity of each point.

4. The method of claim 3, wherein, The generation of the predicted ultrasound image according to the total echo intensity of each point specifically comprises: obtaining a brightness value of a corresponding pixel according to the total echo intensity of each point; generating a predicted ultrasound image according to the brightness values of all pixels.

5. The method according to any one of claims 1 to 4, wherein, When the physical parameter prediction module is trained, the specific calculation formula of the loss value is: ； ； wherein, denotes a mixed loss; denotes a proportionality coefficient of structural similarity loss; denotes a proportionality coefficient of mean square error; denotes a structural similarity loss; denotes a mean square error; denotes a structural similarity function; denotes the i-th frame predicted ultrasound image; denotes the i-th frame real ultrasound image.

6. A device for three-dimensional reconstruction of cardiac ultrasound images, characterized in that it comprises: The three-dimensional reconstruction device comprises: a data set construction unit configured to construct a sample data set; wherein the sample data set comprises real ultrasound images of a heart at different scanning angles and positions of each point in each frame of real ultrasound images; a model construction unit configured to construct an image reconstruction model; wherein the image reconstruction model comprises a physical parameter prediction module and a physical rendering module, the physical parameter prediction module is configured to extract features of the positions of each point in the real ultrasound images to obtain physical parameters of each point in the real ultrasound images, the physical parameters comprising an attenuation coefficient, a reflection coefficient, a boundary probability, a scattering density and a scattering intensity; the physical rendering module is configured to calculate a residual energy and a total echo intensity of each point based on a ray tracing principle according to the physical parameters of each point in the real ultrasound images, generate a predicted ultrasound image according to the total echo intensities of all points, and further generate a three-dimensional ultrasound image according to all predicted ultrasound images; the calculation formula of the residual energy is: ； wherein, denotes a scan line above and with a depth of the remaining energy at; denotes an initial energy; denotes a scan line above and with a depth of the reflection coefficient at; denotes a scan line above and with a depth of the boundary indicator function at, obtained by sampling the boundary probability at above and with a depth of above and with a depth of denotes an attenuation coefficient; denotes a frequency factor; denotes a depth step size; the calculation formula of the total echo intensity is: ； ； ； in, Indicates scan line Above and depth is Reflected energy at the location; Represents the point spread function; This represents the convolution operation; Indicates scan line Above and depth is Boundary indicator function, scan line For scan lines offset, depth For depth The offset; Indicates scan line Above and depth is Scattered energy at that location; Indicates scan line Above and depth is Scattering characteristics at that location; Indicates scan line Above and depth is The scattering probability at that point, through the scan line Above and depth is The scattering density at that location was obtained by sampling. Indicates scan line Above and depth is Scattering intensity at that location; Indicates scan line Above and depth is Total echo intensity at the location; a training and evaluating unit configured to train and evaluate the physical parameter prediction module in the image reconstruction model by using the sample data set to obtain a trained image reconstruction model; A reconstruction unit is configured to implement three-dimensional reconstruction of the cardiac ultrasound image by using the trained image reconstruction model.

7. An electronic device comprising a memory, a processor, and a computer program / instructions stored on the memory, wherein, The processor executes the computer program / instructions to implement the steps in the method of three-dimensional reconstruction of the cardiac ultrasound image according to any one of claims 1-5.

8. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps in the method of three-dimensional reconstruction of the cardiac ultrasound image according to any one of claims 1-5.

Images