Ultrasonic image reconstruction method and device for extremely sparse space sampling and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of medical image processing, in particular to an ultrasonic image reconstruction method and device for extremely sparse space sampling and electronic equipment. The ultrasonic image reconstruction method for extremely sparse space sampling comprises the following steps: acquiring an extremely sparse space observation sequence acquired under an extremely sparse sampling condition; inputting the extremely sparse space observation sequence into a pre-trained generative model to generate predicted Gaussian dot matrix distribution corresponding to the extremely sparse space observation sequence; the generative model has a mapping relation between the full-space sampling ultrasonic data and the full-space Gaussian dot matrix distribution after being pre-trained; and performing image rendering according to the predicted Gaussian dot matrix distribution to obtain an ultrasonic image reconstructed based on the extremely sparse space observation sequence. The method can achieve the purpose of realizing high-quality three-dimensional reconstruction by fully utilizing sparse ultrasonic data under the condition of low-cost hardware.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of medical image processing, in particular to an ultrasound image reconstruction method and device for extremely sparse spatial sampling and electronic equipment. BACKGROUND

[0002] At present, the field of medical image intelligent reconstruction and high-resolution modeling is developing rapidly, especially in the direction of ultrasound / optical acoustic imaging. Ultrasound imaging has been widely used in clinical diagnosis and intraoperative navigation due to its advantages of no radiation, strong portability, low cost, etc. With the progress of deep learning technology, many studies have introduced data-driven algorithms into imaging systems to improve image quality, automation level and generalization ability.

[0003] Traditional ultrasound image reconstruction methods mainly include algorithms based on synthetic aperture, time delay and additive reconstruction, and back projection method, etc. Such methods rely on approximate modeling of sound propagation path and device response, and are only suitable for image reconstruction under regular scanning trajectories. The reconstruction effect is poor in clinical actual scenes such as sparse sampling or free scanning. In addition, some systems use non-focused transducers or high-channel number hardware to perform full spatial spectrum sampling in order to improve image quality, but such devices are usually expensive and bulky, which is difficult to deploy in a regular clinical environment.

[0004] Therefore, how to fully utilize sparse ultrasound data to achieve high-quality three-dimensional reconstruction under the condition of low-cost hardware is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide at least an ultrasound image reconstruction method and device for extremely sparse spatial sampling and electronic equipment, which can achieve the purpose of fully utilizing sparse ultrasound data to achieve high-quality three-dimensional reconstruction under the condition of low-cost hardware.

[0006] To solve the above technical problems, at least one embodiment of the present application provides an ultrasound image reconstruction method for extremely sparse spatial sampling, comprising: obtaining an extremely sparse spatial observation sequence collected under extremely sparse sampling conditions; inputting the extremely sparse spatial observation sequence into a pre-trained generative model to generate a predicted Gaussian point array distribution corresponding to the extremely sparse spatial observation sequence; the generative model has a mapping relationship between full spatial sampling ultrasound data and full spatial Gaussian point array distribution after pre-training; performing image rendering according to the predicted Gaussian point array distribution to obtain an ultrasound image reconstructed based on the extremely sparse spatial observation sequence.

[0007] In an embodiment, the generative model obtains the mapping relationship between the full spatial sampling ultrasound data and the full spatial Gaussian point array distribution, comprising: Using an ultrasound imaging platform with translation-rotation scanning capabilities, full-space sampling ultrasound data corresponding to the phantom were acquired; The full-space sampled ultrasound data is parametrically represented and transformed using a trained physical mapping model to obtain the full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data. Determine the mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution.

[0008] In one embodiment, training the physical mapping model includes: A forward acoustic propagation model based on a Gaussian ellipsoid is constructed, and the full-space sampled ultrasound data is input into the forward acoustic propagation model to determine the initial full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data; An optimization strategy based on gradient backpropagation is adopted. Based on the biological tissue morphology characteristics corresponding to the full-space sampled ultrasound data, the forward acoustic propagation model is adaptively optimized to determine the full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data. The parameters include at least one of the position, covariance, and density of the Gaussian ellipsoid.

[0009] In one embodiment, the generative model includes a latent space encoder, a latent space diffusion model, and a latent space decoder, wherein the step of basing the mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution includes: The latent space encoder is used to map the full-space Gaussian lattice distribution to the latent space to obtain a low-dimensional latent vector; The low-dimensional latent vectors in the latent space are trained using the latent space diffusion model to obtain generative model parameters that conform to the statistical law of the distribution of Gaussian points in the whole space. Based on the trained latent space diffusion model, the low-dimensional latent vector is denoised in reverse to obtain the completed latent vector; The latent space decoder is used to reconstruct the full-space Gaussian lattice distribution based on the completed latent vector; Based on the reconstructed full-space Gaussian lattice distribution, a mapping relationship is generated between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution.

[0010] In one embodiment, after reconstructing the full-space Gaussian lattice distribution using the latent space decoder based on the completed latent vector, the method further includes: The reconstructed full-space Gaussian point distribution is input into the latent space encoder in the pre-trained latent space generation and completion model. The latent space encoder is used to perform low-dimensional latent vector mapping of the three-dimensional structural information to obtain the mapping vector. extracting an observation mask corresponding to the extremely sparse spatial observation sequence, and labeling the positions of sparse sampling and missing areas in the observation mask; inputting the labeled observation mask and the encoding vector into the latent space diffusion model as a constraint condition of a diffusion inverse process, and performing diffusion inverse process calculation on the mapping vector based on the constraint condition through the latent space diffusion model to obtain a completed mapping vector; inputting the completed mapping vector into the latent space decoder to reconstruct three-dimensional structure information, and generating a completed Gaussian lattice; According to the reconstructed full-space Gaussian lattice distribution, the mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian lattice distribution is generated, specifically: according to the completed Gaussian lattice, the mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian lattice distribution is generated.

[0011] In an embodiment, before the full-space Gaussian lattice distribution is mapped to the latent space using the latent space encoder, it includes: Based on the Gaussian probability field and the Gaussian attribute field, the implicit structured modeling is performed according to the full-space sampling ultrasound data to obtain the full-space Gaussian lattice probability representation corresponding to the full-space sampling ultrasound data; According to the full-space Gaussian lattice probability representation, a full-space Gaussian continuous probability field corresponding to the full-space sampling ultrasound data is generated; According to the full-space Gaussian continuous probability field, the full-space Gaussian lattice distribution is obtained.

[0012] In an embodiment, the extremely sparse spatial observation sequence is input into the pre-trained generative model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence, including: The extremely sparse spatial observation sequence is input into the pre-trained generative model to generate a predicted Gaussian continuous field corresponding to the extremely sparse spatial observation sequence through the pre-trained latent space diffusion model in the generative model; The predicted Gaussian continuous field is converted into the initial predicted Gaussian lattice distribution using the pre-trained latent space decoder in the generative model; The initial predicted Gaussian lattice distribution is fine-tuned using the mapping relationship to obtain the predicted Gaussian lattice distribution.

[0013] In an embodiment, the image rendering is performed according to the predicted Gaussian lattice distribution to obtain the reconstructed ultrasound image based on the extremely sparse spatial observation sequence, including: screening, from the predicted Gaussian lattice distribution, a high-weight Gaussian point with a learnable weight reaching a preset threshold and a low-weight Gaussian point not reaching the preset threshold; performing Gaussian adaptive filtering on the high-weight Gaussian point and performing Gaussian smoothing filtering on the low-weight Gaussian point; performing pixel-level weighted fusion on the filtered high-weight Gaussian point corresponding fine-scale image and the filtered low-weight Gaussian point corresponding coarse-scale image to obtain a reconstructed ultrasound image.

[0014] At least one embodiment of the present application also provides an ultrasound image reconstruction device for extremely sparse spatial sampling, comprising: a data acquisition module configured to acquire an extremely sparse spatial observation sequence collected under extremely sparse sampling conditions; a Gaussian lattice generation module configured to input the extremely sparse spatial observation sequence into a pre-trained generation model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence; the generation model is pre-trained to have a mapping relationship between full-space sampling ultrasound data and full-space Gaussian lattice distribution an image rendering module configured to perform image rendering according to the predicted Gaussian lattice distribution to obtain an ultrasound image reconstructed based on the extremely sparse spatial observation sequence.

[0015] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned ultrasound image reconstruction method for extremely sparse spatial sampling.

[0016] The ultrasound image reconstruction method for extremely sparse spatial sampling provided by the embodiments of the present application maps the extremely sparse spatial observation sequence collected under extremely sparse sampling conditions through a generation model, the generation model is pre-trained to have a mapping relationship between full-space sampling ultrasound data and full-space Gaussian lattice distribution, the structure priori contained in the full-space data can be used to accurately supplement the structural details missing in the sparse data under the constraints of observation mask and system response coding, to complete the spatial frequency spectrum information missing in the extremely sparse data, to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence, and to use the Gaussian lattice as a three-dimensional structure representation method, which can greatly reduce data storage and computing consumption compared with the traditional voxel grid, adapt to the computing power requirement of low-cost hardware, realize the conversion from sparse data to high-quality structure representation, and finally render an image based on the optimized predicted Gaussian lattice distribution to ensure that the output result meets the clinical diagnosis standard.

[0017] The method does not rely on expensive non-focused transducer arrays, adapts to the computing power and storage limitations of low-cost hardware, and still maintains high-fidelity reconstruction capability in extremely sparse sampling scenarios. Ultimately, the method outputs visualization results that meet clinical standards through image rendering, forms an efficient closed loop from the original signal to the diagnostic image, and balances reconstruction accuracy, physical consistency and clinical practicability, providing an innovative technical path for low-cost, low-sampling pressure ultrasound imaging. BRIEF DESCRIPTION OF DRAWINGS

[0018] One or more embodiments are illustrated by way of example in the figures that form a part of this patent document, and which do not limit the scope of embodiments.

[0019] Figure 1 is a working principle diagram of a non-focused system; Figure 2 is an implementation principle diagram of a multi-angle fusion strategy; Figure 3 is an input and output effect diagram of improving the resolution of the slice direction through an algorithm; Figure 4 is a flowchart of an ultrasound image reconstruction method for extremely sparse spatial sampling provided by an embodiment of the present application; Figure 5 is a full spatial sampling ultrasound data set construction scheme diagram provided by an embodiment of the present application; Figure 6 is a flow conversion diagram of an extremely sparse spatial sampling ultrasound reconstruction method provided by an embodiment of the present application; Figure 7 is a schematic diagram of an ultrasound image reconstruction device for extremely sparse spatial sampling provided by an embodiment of the present application; Figure 8 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that in the embodiments of the present application, many technical details are proposed in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present application. The embodiments can be combined and referenced with each other without contradiction.

[0021] In the field of ultrasound image reconstruction, traditional methods mainly include synthetic aperture-based algorithms, time delay and additive reconstruction, and back-projection methods. The core of these methods is to approximately model the sound propagation path and device response. Although these methods can meet the image reconstruction requirements under regular scanning trajectories, the reconstruction effect will be greatly reduced in the actual clinical scenario of sparse sampling or free scanning. At the same time, some systems that pursue high image quality use non-focused transducers or high-channel number hardware to perform full-space spectral acquisition, Figure 1 However, such devices generally have the problems of high price and large size, which makes it difficult to deploy in a regular clinical environment.

[0022] However, some potential key technologies have emerged in the field, such as Gaussian Splatting, which is a new sparse three-dimensional data representation method that can encode the geometry and intensity attributes of objects in three-dimensional space through Gaussian distribution, achieving efficient and continuous volume representation; generative deep learning models such as Variational Auto-Encoders (VAE) and Latent Diffusion Model (LDM) can learn the statistical distribution of complex structures from large-scale training data and generate high-fidelity images under sparse input conditions, which are widely used in medical image restoration and reconstruction tasks; and differentiable physical modeling, which models the physical propagation process as a differentiable operator, helps to optimize the structural features directly from the signal layer, and has become a research hotspot in the field of multi-modal medical imaging. Even so, there is still a lack of a general method that can fully utilize sparse ultrasound data to achieve high-quality three-dimensional reconstruction under low-cost hardware conditions, especially in terms of image reconstruction stability and cross-device generalization ability, which has not yet broken through the existing technical bottlenecks. Therefore, it is an urgent need in the field to develop a new ultrasound image reconstruction method that integrates Gaussian Splatting, generative learning, and physical modeling to achieve efficient restoration and high-resolution recovery of sparse data.

[0023] It should be noted that in this embodiment, only Variational Auto-Encoders (VAE) and Latent Diffusion Model (LDM) are used as examples, and conditional generative adversarial networks or autoregressive model-based generators can also be used. The model type can be selected according to the requirements of the actual application scenario, and will not be described here.

[0024] In the practical application of ultrasound and photoacoustic image reconstruction, technical solutions at the hardware and algorithm levels have their own characteristics.

[0025] In terms of hardware, imaging systems based on cylindrical focusing transducer arrays are most widely used, which are usually combined with aperture imaging technology or filtered back-projection algorithm for image reconstruction. Such systems are suitable for standard two-dimensional image acquisition paths, but are limited by the focusing characteristics of the acoustic beam, resulting in a significant lack of resolution in the slice direction. To improve resolution, some studies have shifted to developing hardware devices that can achieve full sampling of spatial spectrum, such as using two-dimensional arrangements (e.g., planar, cylindrical, or spherical arrangements) of non-focused transducer arrays to build systems (referred to as non-focused systems). Although such systems can generate high-quality three-dimensional images, they rely on expensive and bulky hardware, which cannot be deployed in ordinary clinical environments. In contrast, cylindrical focusing transducer arrays have a mature supply chain system, and some research teams have used multi-angle mechanical scanning methods and multi-angle fusion strategies to more completely acquire three-dimensional spatial spectrum information, such as Figure 2 The implementation principle diagram of a multi-angle fusion strategy is shown. This strategy improves the imaging resolution to some extent, but at the cost of imaging speed.

[0026] In terms of algorithms, as shown in Figure 3 The input and output effect diagram of an algorithm for improving the resolution in the slice direction is shown. Traditional methods such as deconvolution (using simulated or measured point spread functions for post-processing optimization) and physical modeling optimization (combining the physical model of the imaging process with image features for optimized reconstruction) can improve the visual effect of the image to some extent, but they do not solve the core problem of spatial frequency missing. The introduction of deep learning technology brings new ideas to this field. Regression networks based on U-Net and ResNet are used for ultrasound image reconstruction or enhancement, and some studies have also tried to use VAE and GAN generative models to realize image domain repair. However, these methods mostly rely on image post-processing and do not model the physical propagation process of ultrasound waves from the signal source, lacking physical constraints and interpretability.

[0027] Although current medical ultrasound image reconstruction technology has been extensively researched and applied, there are still many technical defects that restrict its development in terms of high precision, real-time performance, and universality.

[0028] Firstly, the imaging resolution is limited, and the structural details are blurred. Existing systems mostly use cylindrical focusing transducer arrays as the core, and the focusing ability of the acoustic beam in the slice direction is weak, resulting in a serious unevenness in the imaging resolution in three-dimensional space, especially in the out-of-plane direction, making it difficult to accurately restore the anatomical structure, directly affecting the doctor's judgment of the lesion boundary and internal structure, and increasing the risk of clinical diagnosis errors.

[0029] Secondly, the reconstruction ability is poor under sparse sampling, and it depends on high-density scanning. Traditional algorithms and existing deep learning methods usually need relatively dense spatial sampling to obtain clear images. When the sampling is extremely sparse (such as free-hand scanning or rapid scanning), the target structure cannot be accurately restored, the image artifacts are obvious, and the information loss is serious, which limits its application in clinical rapid imaging scenarios such as intraoperative navigation and rapid screening.

[0030] Thirdly, it cannot be modeled from the physical signal source, and the reconstruction stability is poor. Most image enhancement or reconstruction methods only process in the image domain, without modeling the physical process of acoustic signal propagation (such as time delay, attenuation, and point spread effect). The reconstruction effect fluctuates greatly under different devices, different tissue characteristics, or different operating conditions, making it difficult to ensure consistency and robustness, and unable to meet the requirements of clinical diagnosis for result stability.

[0031] Fourthly, the generalization ability is weak, and the model migration is poor. Most current deep learning-based reconstruction algorithms are optimized for specific devices and acquisition conditions, and the model parameters are deeply coupled with device characteristics, scan trajectory. It is difficult to adapt to different probes, different tissue types, or different scan trajectories. Once the imaging conditions change (such as sound speed change, angle offset), the reconstruction quality will decrease significantly, and large-scale data needs to be collected and the model needs to be trained, which greatly increases the clinical deployment cost and technical barriers.

[0032] Fifthly, the storage and computing resource consumption is large, and it is difficult to run in real time. In the process of high-resolution three-dimensional image reconstruction, the traditional voxel grid representation method will produce a large amount of data, and the training and inference process requires large amount of calculation and high memory consumption, which is difficult to meet the clinical real-time imaging demand. Even with deep networks, existing solutions are difficult to balance reconstruction accuracy and efficiency, resulting in long imaging latency and affecting clinical use experience, especially not suitable for intraoperative navigation and other scenarios with high real-time requirements.

[0033] In summary, the existing technology has obvious bottlenecks in key technical links such as low-cost high-resolution three-dimensional reconstruction, high-fidelity image restoration under extremely sparse data, and physical consistency and generalization modeling. It is necessary to propose a new technical solution that integrates sparse modeling, physical modeling, and generative model to break through the limitations of existing technology and promote the further application of ultrasound image reconstruction technology in the clinical field.

[0034] In view of this, the present application proposes an ultrasound image reconstruction method for extremely sparse spatial sampling. The implementation details of the ultrasound image reconstruction method for extremely sparse spatial sampling of the present embodiment will be described in detail below. The following content is only provided for the implementation details for easy understanding, and is not necessary for implementing the present solution.

[0035] Embodiment one: The specific process of the ultrasound image reconstruction method for extremely sparse space sampling in the embodiment can be as shown in Figure 4 Step 101, acquiring an extremely sparse space observation sequence acquired under an extremely sparse sampling condition.

[0036] The extremely sparse space observation sequence is acquired by a conventional ultrasound acquisition device, that is, the original signal of the target tissue acquired by the ultrasound acquisition device, such as a conventional cylindrical focusing ultrasound transducer (or a portable ultrasound probe) and the like. The method does not need to rely on expensive hardware, so the ultrasound acquisition device can be a conventional, low-cost ultrasound hardware capable of outputting an original beam signal (RF signal) or a time domain sampling sequence after time delay compensation, and providing corresponding probe pose / angle information (not necessarily), and the type of the ultrasound acquisition device is not limited in the embodiment.

[0037] Step 102, inputting the extremely sparse space observation sequence into a pre-trained generative model to generate a predicted Gaussian dot array distribution corresponding to the extremely sparse space observation sequence.

[0038] The existing ultrasound reconstruction technology has problems of missing spatial frequency and broken structure information under extremely sparse sampling (such as 1 / 30 sampling rate) due to relying on discretization representation methods such as voxel grid, resulting in obvious image artifacts. In view of this, the Gaussian dot array is proposed as a sparse representation carrier of three-dimensional structure in the method, and the extremely sparse space observation sequence is converted into a Gaussian dot array.

[0039] Specifically, under the condition of extremely sparse ultrasound sampling, the preprocessed extremely sparse space observation sequence (containing a normalized signal tensor and an observation mask of the labeled sampling position / missing region) is input into a pre-trained generative model. The generative model has a mapping relationship between the full space sampling ultrasound data and the full space Gaussian dot array distribution after pre-training. The model generates a predicted Gaussian dot array distribution corresponding to the extremely sparse space observation sequence based on the full space sampling ultrasound data-full space Gaussian dot array distribution mapping relationship learned in the pre-training stage, and the dot array needs to meet both the full space Gaussian dot array statistical rule and the physical characteristics of the extremely sparse observation sequence.

[0040] Based on the full space mapping relationship of the pre-trained model, the missing structure information can be supplemented under the condition of extremely sparse sampling (such as 1 / 30 sampling rate), and a predicted Gaussian dot array distribution (such as SSIM>0.9) similar to the full sampling quality can be generated, which can solve the problems of many artifacts and fuzzy structure details in the reconstruction of the traditional method under sparse sampling, and realize isotropic three-dimensional imaging.

[0041] ​It should be noted that before the extremely sparse spatial observation sequence is input into the pre-trained generation model, the extremely sparse spatial observation sequence can be pre-processed (such as band-pass filtering, envelope denoising, coarse alignment based on probe pose, and normalization) first to eliminate the interference of noise on the judgment of signal strength. Of course, whether to perform pre-processing and the pre-processing means adopted can be determined according to the data state in the actual application scene, and this embodiment does not limit this.

[0042] In step 103, an image is rendered according to the predicted Gaussian point array distribution, and an ultrasonic image reconstructed based on the extremely sparse spatial observation sequence is obtained.

[0043] The optimized predicted Gaussian point array distribution is a set composed of tens of thousands to hundreds of thousands of Gaussian points, each Gaussian point encodes the spatial distribution and acoustic reflection characteristics (such as amplitude corresponding to reflection intensity, position corresponding to anatomical structure coordinates) of the target tissue through position, amplitude, scale and other attributes. Through image rendering, the dispersed structure and acoustic information in the Gaussian point array are integrated into a continuous and intuitive image, and finally an ultrasonic image (such as a B-mode two-dimensional slice image and a three-dimensional volume image) conforming to the clinical habit is output, realizing the reconstruction target of sparse sampling data to high-fidelity diagnostic image.

[0044] It should be noted that the Gaussian point array can be represented by an isotropic or anisotropic covariance, or small voxel clusters can be used instead of single points to take into account specific hardware limitations, which can be referred to the introduction of the present embodiment, and will not be repeated here.

[0045] Based on the above introduction, the ultrasonic image reconstruction method for extremely sparse spatial sampling provided in the present embodiment maps the extremely sparse spatial observation sequence collected under the condition of extremely sparse sampling through a generation model. The generation model learns the mapping relationship between the full-space sampling ultrasonic data and the full-space Gaussian point array distribution through pre-training of full-space data, and can supplement the structural details missing in the sparse data and complete the spatial frequency spectrum information missing in the extremely sparse data by means of the structure prior contained in the full-space data under the constraints of observation mask and system response coding, to generate the predicted Gaussian point array distribution corresponding to the extremely sparse spatial observation sequence. Using the Gaussian point array as a three-dimensional structure representation method, compared with the traditional voxel grid, the data storage and calculation consumption can be greatly reduced, the computing power requirement of low-cost hardware can be adapted, the conversion from sparse data to high-quality structure representation can be realized, and finally the image is rendered based on the optimized predicted Gaussian point array distribution to ensure that the output result meets the clinical diagnosis standard.

[0046] The method does not need to rely on expensive non-focusing transducer array, adapts the computing power and storage limit of low-cost hardware, and still maintains high fidelity reconstruction ability in extremely sparse sampling scene. Finally, the visualization result meeting the clinical standard is output through image rendering, forming an efficient closed loop from the original signal to the diagnostic image, and taking into account the reconstruction accuracy, physical consistency and clinical practicability, which provides an innovative technical path for low-cost and low-sampling pressure ultrasound imaging.

[0047] Embodiment two: The generation model is pre-trained to have a mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian point array distribution. However, the specific implementation of the generation model obtaining the mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian point array distribution from the sample data is not limited in the above embodiments. To ensure the physical consistency of the mapping relationship and the spatial prior of the model, and to ensure the high-quality reconstruction of sparse data, a mapping relationship learning method is proposed in this embodiment. Specifically, the generation model obtains the mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian point array distribution, which can include the following sub-steps: Step 104, using an ultrasound imaging platform with translation-rotation scanning capability, obtaining full-space sampling ultrasound data corresponding to the phantom.

[0048] Relying on the ultrasound imaging platform with translation-rotation scanning capability (including conventional cylindrical focused ultrasound probe, translation stage, rotation stage and imaging cavity), full-space dense sampling is performed on the calibration standard phantom (simulating the acoustic properties of human tissues such as sound speed and acoustic impedance), and ultrasound data (including original RF signal and corresponding probe pose information) covering the complete spatial spectrum are obtained.

[0049] Among them, the acoustic properties and structural parameters of the phantom are known (such as preset lesion size and position), which can avoid data fluctuations caused by individual differences of human tissues, provide uniform and controllable training samples for mapping relationship learning, and ensure that the mapping rule learned by the model has universality, rather than relying on specific individual tissue data.

[0050] Step 105, using the trained physical mapping model to perform parameterized representation conversion on the full-space sampling ultrasound data to obtain the full-space Gaussian point array distribution corresponding to the full-space sampling ultrasound data.

[0051] The traditional pure data-driven model lacks physical constraints and is prone to generate point array distributions that do not conform to the law of ultrasonic propagation. In view of this, the trained physical mapping model is called in the embodiment to convert the full-space sampling ultrasonic data into a parameterized full-space Gaussian point array distribution. The physical mapping model incorporates the acoustic propagation physical mechanism, can establish an explicit physical association between the signal domain (full-space sampling data) and the structure domain (Gaussian point array), ensure that the mapping relationship conforms to the principle of ultrasonic wave propagation, and avoid the occurrence of physical contradiction structure artifacts in subsequent extremely sparse reconstruction.

[0052] Step 106, determining the mapping relationship between the full-space sampling ultrasonic data and the full-space Gaussian point array distribution.

[0053] Through a large number of matching training of phantom full-space sampling data and corresponding Gaussian point array, the full-space mapping relationship is used as a structure template, so that the generative model masters the corresponding rules of data and point array in the full space, and the stable corresponding relationship between the two is solidified, providing full-space structure prior for subsequent extremely sparse sampling reconstruction.

[0054] The mapping relationship learning method provided by the embodiment relies on the platform of translation-rotation scanning capability + conventional cylindrical focused probe to obtain full-space data, without relying on expensive non-focused transducer array (such as spherical and two-dimensional array), and can build a system by using existing mature hardware supply chain, which can greatly reduce the hardware threshold of full-space data acquisition. The mapping relationship is trained based on phantom data, and the standardized characteristics of the phantom enable the mapping relationship to adapt to ultrasonic equipment in different clinical scenarios, reducing the problem of insufficient generalization ability of the model caused by data differences. At the same time, the Gaussian point array distribution is generated by the physical mapping model, which can greatly reduce data storage and computing consumption compared with the traditional dense voxel grid representation.

[0055] Embodiment three: In the mapping relationship between the generative model and the full-space sampling ultrasonic data and the full-space Gaussian point array distribution, a trained physical mapping model is needed to perform parameterized representation conversion on the full-space sampling ultrasonic data. The physical model therein refers to a differentiable modeling system that integrates the physical law of ultrasonic wave propagation and is used to establish the association between ultrasonic signals and three-dimensional organizational structure (represented by Gaussian point array as the core), and its core is to simulate the acoustic behavior in the real ultrasonic imaging process through mathematical formulas and physical parameters. The specific model structure and training method of the physical mapping model are not limited, such as a differentiable forward model based on wave equation approximation, a simplified differentiable model based on ray tracing, etc.

[0056] In order to ensure the physical rationality of the physical mapping model and avoid deviating from the real imaging law, the embodiment proposes a training method for the physical mapping model, which specifically includes the following steps: In step 107, a forward acoustic propagation model based on a Gaussian ellipsoid is constructed, and the full-space sampled ultrasound data is input into the forward acoustic propagation model to determine an initial full-space Gaussian point cloud distribution corresponding to the full-space sampled ultrasound data.

[0057] A forward acoustic propagation model taking a Gaussian ellipsoid as a core carrier is constructed. The Gaussian ellipsoid can flexibly represent the non-uniform geometric shape of the biological tissue (such as irregular lesions, tissue boundaries), and the forward acoustic propagation model incorporates the physical mechanism of ultrasound wave propagation (time delay, attenuation, etc.), which can simulate the complete process of ultrasound wave emission from the probe, reflection / scattering through the tissue, and reception by the probe. This can avoid the generation of point cloud distributions that do not conform to acoustic laws by traditional pure data-driven models due to the lack of physical constraints, ensuring that the initial Gaussian point cloud distribution is consistent with the real ultrasound imaging process. Based on the forward acoustic propagation model of the Gaussian ellipsoid, the fine structure of the biological tissue (such as small lesions, tissue interfaces) can be described by the flexible parameters of the Gaussian ellipsoid, and the point cloud distribution can be ensured to conform to the ultrasound imaging rules by relying on the physical mechanism of acoustic propagation, solving the problem of difficult trade-off between structure accuracy and physical rationality in traditional modeling, and providing a standard of high-quality full-space point cloud for subsequent extremely sparse reconstruction.

[0058] The forward acoustic propagation model can be a differentiable acoustic propagation model. The differentiable acoustic propagation model is a mathematical model that incorporates the hardware characteristics of the ultrasound acquisition system, simulates the physical propagation process of sound waves, and supports back propagation. By simulating the physical process of real sound wave propagation, the Gaussian point cloud (representing the geometric shape and acoustic reflectivity of the tissue structure) is converted into a transducer-receivable signal (synthetic observation signal). The differentiable forward calculation is based on the real sound wave propagation physical model, ensuring that the Gaussian point cloud obtained through subsequent loss optimization not only matches the signal characteristics but also conforms to the physical laws of acoustic propagation. With the differentiable characteristic, the model can back-propagate the signal domain loss to the Gaussian point cloud attributes (position, amplitude, scale) and system encoding vectors, realize the closed loop of synthetic signal error→model parameter adjustment→structure representation optimization, fill the gap between the mathematical representation of the tissue structure and the actual ultrasound acquisition signal, and obtain a synthetic observation signal that matches the actual acquisition scenario, providing a signal domain benchmark for subsequent comparison of real extremely sparse space observation sequences and optimization of structure parameters. Of course, other types of forward acoustic propagation models can also be selected, which are not limited in the present embodiment.

[0059] It should be noted that the construction steps of the forward acoustic propagation model in the embodiment are not limited, and the system response coding model can be called to code the system parameters to obtain a coding vector. Then, the forward acoustic propagation model is constructed based on the coding vector. Through the special encoder of the system response coding model, the system parameters describing the hardware characteristics of the ultrasonic equipment and the acquisition configuration are converted into a low-dimensional and structured coding vector. Then, the vector is injected as a core parameter into the differentiable acoustic propagation model, so that the model can accurately reproduce the current device's acoustic wave propagation physical law.

[0060] In the process of coding the system parameters, the device-specific parameters in the system parameters can be obtained. After coding the device-specific parameters, the differential acoustic propagation model is constructed based on the coding. The device-specific parameters refer to the parameters that are directly bound to the ultrasonic acquisition hardware and determine the core physical characteristics of the device, which are different from the general acquisition configuration parameters (such as adjustable software configurations such as sampling rate and scan angle). Specifically, it can include probe hardware inherent attributes, device hardware limitation parameters, and exclusive calibration parameters. The device-specific parameters are the core factors that determine the differences in acoustic wave propagation (such as different array arrangements that will result in completely different beam directions), and only coding them can ensure that the coding vector focuses on the essential differences between devices, so that the differentiable acoustic propagation model constructed based on the vector can accurately reproduce the physical behavior of the current device, avoid generalization error, and also improve the efficiency and robustness of the model through dimension reduction and redundancy reduction.

[0061] After inputting the full-space sampling ultrasonic data (containing RF signals with complete spatial spectrum and probe pose information) into the forward acoustic propagation model based on the Gaussian ellipsoid, the initial Gaussian point array distribution of the full space that can preliminarily represent the three-dimensional structure of the tissue is generated by calculating the contribution of each spatial position Gaussian ellipsoid to the ultrasonic signal, including the basic parameters such as the position, covariance, and density of the Gaussian ellipsoid.

[0062] In step 108, an optimization strategy based on gradient backpropagation is adopted to adaptively optimize the parameters of the forward acoustic propagation model according to the full-space sampling ultrasonic data and the corresponding biological tissue morphology characteristics of the phantom, and to determine the full-space Gaussian point array distribution corresponding to the full-space sampling ultrasonic data.

[0063] The optimization strategy based on gradient backpropagation can directly calculate the parameter adjustment direction based on the signal difference and the structure prior, avoiding the inefficiency of traditional trial-and-error optimization.

[0064] An optimization strategy based on gradient backpropagation is adopted to constrain the full-space sampled ultrasound data and the phantom biological tissue morphology features: on the one hand, the difference (such as L1 / L2 loss, frequency domain loss) between the synthesized ultrasound signal (generated by the initial Gaussian point array through the forward model) output by the model and the original full-space sampled ultrasound data is calculated, and the gradient is calculated and backpropagated; on the other hand, the Gaussian ellipsoid parameters (at least one of position, covariance, density, etc.) associated with the forward acoustic propagation model are adaptively adjusted in combination with the known biological tissue morphology features (such as the preset lesion geometry, position boundary) of the phantom. For example, the Gaussian ellipsoid position and covariance are optimized in the lesion boundary area of the phantom to improve the structural resolution, and the density parameter is adjusted in the uniform tissue area to match the signal intensity, and finally the full-space Gaussian point array distribution highly consistent with the full-space sampled ultrasound data is determined, and the training of the physical mapping model is completed.

[0065] The full-space sampled ultrasound data can provide accurate constraints at the signal level to ensure that the point array mapping signal is consistent with the original data, and the known biological tissue morphology features of the phantom can provide prior guidance at the structure level to avoid structures that are inconsistent with the real tissue morphology after optimization, such as false lesions and blurred boundaries. In this embodiment, the full-space Gaussian point array distribution output by the trained physical mapping model is adaptively parameter-optimized according to the full-space sampled ultrasound data and the corresponding biological tissue morphology features of the phantom, and can be used as the core sample for the generation model to learn the full-space data-point array mapping relationship; in subsequent extremely sparse reconstruction, the generation model can rely on this mapping relationship to accurately complete the missing structure from a small amount of sparse data, greatly improving the reconstruction quality in the extremely sparse scene, and providing key technical support for realizing high-quality ultrasound reconstruction with low-cost hardware.

[0066] Embodiment Four In order to establish a stable mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian point array distribution, the generation model in this embodiment can specifically include a latent space encoder, a latent space diffusion model, and a latent space decoder, and a mapping relationship determination method based on the generation model architecture is proposed, which includes encoding, training, completion, and decoding.

[0067] Specifically, step 106 can be performed according to the following sub-steps: Step 61, mapping the full-space Gaussian point array distribution to the latent space by using the latent space encoder to obtain a low-dimensional latent vector.

[0068] With the help of the latent space encoder, the three-dimensional structural features (such as Gaussian point position, covariance, and density correlation law) and acoustic properties of the full-space Gaussian point array distribution are extracted, and the high-dimensional and complex point array parameters are compressed into low-dimensional latent vectors, which reduces the complexity of subsequent processing while retaining the core information.

[0069] Step 62, train the low-dimensional latent vector in the latent space using the latent space diffusion model to obtain the generation model parameters consistent with the statistical law of the full space Gaussian lattice distribution.

[0070] The low-dimensional latent vector is trained using the latent space diffusion model, and through the step-by-step injection of noise-reverse denoising mechanism, combined with the signal domain constraints of full space sampling ultrasound data (such as the loss of synthetic signals and original data) and the structural constraints of full space Gaussian lattice (such as geometric consistency), the model parameters are optimized, so that the generation model parameters conform to the statistical law of the full space Gaussian lattice distribution.

[0071] Step 63, based on the trained latent space diffusion model, reverse denoising is performed on the low-dimensional latent vector to obtain the completed latent vector.

[0072] Based on the trained latent space diffusion model, reverse denoising is performed on the low-dimensional latent vector to supplement the missing structural details in the latent vector, and the completed latent vector with complete information is obtained.

[0073] Step 64, reconstruct the full space Gaussian lattice distribution based on the completed latent vector using the latent space decoder.

[0074] The completed latent vector is reversely mapped to the high-dimensional full space Gaussian lattice parameters through the latent space decoder, the key attributes of the Gaussian ellipsoid are recovered point by point, and the lattice accuracy and efficiency are optimized through dynamic adjustment strategies (such as key area copy splitting, redundant area pruning).

[0075] Step 65, according to the reconstructed full space Gaussian lattice distribution, generate the mapping relationship between the full space sampling ultrasound data and the full space Gaussian lattice distribution.

[0076] By verifying the matching degree of the reconstructed lattice and the full space sampling ultrasound data (such as the consistency of the synthetic signals generated by the differentiable acoustic propagation model and the original data), the mapping relationship between the full space sampling ultrasound data and the full space Gaussian lattice distribution is solidified, and it is ensured that when the full space sampling data is input, the model can accurately output the corresponding lattice distribution.

[0077] The mapping relationship determination method proposed in this embodiment compresses the high-dimensional full-space Gaussian lattice distribution into a low-dimensional latent vector by means of the latent space encoder, greatly reduces the data storage and the calculation consumption of model training and inference, and accurately reconstructs the high-dimensional lattice through the latent space decoder, avoiding the efficiency bottleneck of high-dimensional direct modeling and adapting to the demand of clinical reconstruction speed; the latent space diffusion model can deeply learn the distribution rule of the full-space Gaussian lattice through step-by-step noise injection-reverse denoising training, so that the generated model parameters fit the statistical characteristics of the full-space data, and the reverse denoising process can actively complete the potential information loss in the latent vector, laying a foundation for subsequent structure information completion based on a small amount of observed data in the extremely sparse sampling scenario, and breaking through the limitation of traditional deterministic mapping that cannot cope with data sparseness. Through the closed-loop process of encoding-training-completion-decoding, combined with the signal domain constraint of full-space sampling ultrasound data and the structure constraint of full-space Gaussian lattice, the matching degree of the reconstructed lattice and the original data is verified before generating the mapping relationship, ensuring the accuracy and reliability of the mapping relationship; at the same time, the model learns the general statistical rule of full-space data, rather than relying on the exclusive characteristics of specific tissues or devices, which can adapt to the full-space data mapping needs of different biological tissue types and different ultrasound probe configurations, has stronger generalization ability, and reduces the repeated training cost in cross-scene application.

[0078] Embodiment five In order to solve the problem of incomplete reconstruction caused by structure information loss in the extremely sparse sampling scenario, in the determination of the mapping relationship between the full-space sampling ultrasound data and the full-space Gaussian lattice distribution, the latent space generation completion mechanism is introduced in this embodiment to fill in the structural details of the un-covered area of the sparse signal by using the information completion ability of the generative model. Then, after the step 64 of reconstructing the full-space Gaussian lattice distribution based on the completed latent vector by using the latent space decoder, and before the step 65 is executed, the following steps can be further executed: Step 66, input the reconstructed full-space Gaussian lattice distribution into the latent space encoder in the pre-trained latent space generation completion model, map the three-dimensional structure information into a low-dimensional latent vector through the latent space encoder, and obtain a mapping vector.

[0079] The reconstructed full-space Gaussian lattice distribution is a three-dimensional discrete set composed of tens of thousands to hundreds of thousands of Gaussian points, each Gaussian point contains position, amplitude, scale and other attributes, and the whole represents the spatial distribution and acoustic characteristics of the target tissue. However, this discrete point set form has the problems of high dimension (the total parameter dimension can reach millions), high computational complexity, and difficulty in direct use for generative modeling.

[0080] The latent space encoder, as the core component of the latent space generation and completion model, its core function is to convert this high-dimensional discrete three-dimensional structure information into a low-dimensional continuous mapping vector (usually hundreds of dimensions, such as 256 dimensions, 512 dimensions) through feature extraction + dimension compression. The vector is not simply a parameter reduction, but through deep learning algorithms (such as convolution, attention mechanism) to extract global structural features (such as the overall shape of the organ, the relative position of the lesion) and local detail features (such as tissue boundary texture, reflectivity gradient) from the Gaussian point array, and condense these features into a structured numerical vector. For example, a certain segment of the vector may correspond to the overall contour features of the liver, and another segment may correspond to the distribution features of the blood vessels on the surface of the liver, forming a compressed description of the tissue structure.

[0081] The final generated low-dimensional mapping vector not only reduces the computational load of the model, enabling diffusion completion to be completed within a clinically acceptable time, but also has a clear structural meaning in space, for example, a small adjustment of the latent vector may correspond to a subtle deformation of the tissue structure, which enables the diffusion model to generate continuous and reasonable structural features for the missing area through continuous sampling and iteration within the latent space, avoiding structural discontinuity or discrete artifacts after completion.

[0082] Step 67, extract the observation mask corresponding to the extremely sparse spatial observation sequence, and mark the positions of sparse sampling and missing areas in the observation mask.

[0083] The extremely sparse spatial observation sequence is the original signal collected by the ultrasound device under the extremely sparse sampling mode (such as only 1 / 30 of the regular sampling points), and its signal distribution presents the characteristics of discrete distribution and large amount of blank. For example, in the two-dimensional imaging plane, only part of the coordinate points have actual signal values, and the rest of the coordinate points have no signal (because they are not sampled).

[0084] The observation mask is a binary label matrix (or three-dimensional tensor for three-dimensional imaging) with the same spatial dimension as the extremely sparse spatial observation sequence. Its core function is to visually mark the sampling positions (known areas) and missing areas (unknown areas) by distinguishing between 0 and 1 (or other label values). The elements corresponding to the positions with signals in the extremely sparse spatial observation sequence in the mask are marked as 1 (or a specific value), indicating that this area has real collected signals, and the completion process must strictly follow the known information in these areas and cannot be modified arbitrarily. The elements corresponding to the positions without signals in the mask are marked as 0 (or another specific value), indicating that this area is a signal blank area and needs to be completed by the subsequent latent space diffusion model.

[0085] For example, in abdominal ultrasound imaging, if sparse sampling only covers the right half of the liver, the observation mask will mark 1 (known signal area) in the right half and 0 (missing area) in the left half, clearly defining the boundary of the completion and the target.

[0086] At step 68, the labeled observation mask and the encoding vector are input into the latent space diffusion model as constraint conditions for the diffusion inverse process. The latent space diffusion model calculates the diffusion inverse process of the mapping vector based on the constraint conditions to obtain the completed mapping vector.

[0087] Although the mapping vector condenses the core features of the predicted Gaussian lattice distribution, it is limited by the information missing in the extremely sparse spatial observation sequence, and its structure representation in the missing area is still incomplete (such as the latent vector features of a certain segment of blood vessels being blurred due to signal missing). The latent space diffusion model is a generative model with constraint-based generation capability. By inputting two types of key constraints (observation mask and encoding vector), the model is guided to complete the mapping vector in the diffusion inverse process.

[0088] Among them, the observation mask constraint explicitly tells the model which structure area has reliable signal (needs to be consistent) and which area is missing (needs to be completed), ensuring that the completion does not violate the known signal facts; the encoding vector constraint injects device physical characteristics (such as probe beam characteristics, sampling rate) into the completion process, ensuring that the structure features of the completion conform to the current device imaging rules (such as the reflection intensity of the completion needs to adapt to the device sensitivity).

[0089] Through the diffusion inverse process guided by these two types of constraints, the missing structure information in the mapping vector is reasonably filled, and the final output of the completed mapping vector not only retains the reliable features of the original predicted Gaussian lattice distribution, but also contains the complete structure features of the missing area.

[0090] Among them, the method of calculating the diffusion inverse process of the mapping vector based on the constraint conditions by the latent space diffusion model can be executed according to the following sub-steps: based on the constraint conditions, the latent space diffusion model calculates the diffusion inverse process of the mapping vector in stages from coarse to fine according to the scale; after each stage iteration ends, the loss value of the structural similarity between the current completed mapping vector and the predicted Gaussian lattice distribution is calculated; if the loss value is lower than the preset threshold, terminate the iteration of the current stage and execute the iteration of the next stage; if the iteration number reaches the preset iteration step number, output the current completed mapping vector.

[0091] Specifically, the completion of the mapping vector by the latent space diffusion model is not completed at one time, but is executed in stages in the order from coarse scale to fine scale: first, the global outline of the tissue structure is constructed (such as the overall shape of the organ, the direction of the large blood vessels), and then the local details are gradually refined (such as tissue texture, small branches). In each stage iteration, the structural similarity loss between the completed mapping vector and the predicted Gaussian point array distribution is calculated (measuring the matching degree of the completion result and the known structure), and it is dynamically judged whether to terminate the current stage: if the loss value is lower than the preset threshold (indicating that the completion at the current scale has met the accuracy requirement), then enter the next scale optimization; if the maximum iteration step is reached, then output the completed mapping vector optimized by multiple scales. The optimization order from coarse to fine of the calculation method of the diffusion inverse process ensures that the global structure (such as the position of the organ, the direction of the main stem of the blood vessel) is matched with the known signal first, and then the details are gradually refined, avoiding the contradiction of accurate local details but misplaced global structure, which can enhance the anatomical consistency of the completion result. The preset iteration step can be calculated by using a sampling strategy such as DDIM or exponential spectrum sampling, and the control of the iteration step of the diffusion inverse process can be set to not more than 50 steps.

[0092] Further, after each round of stage iteration of the latent space diffusion model, the gradient can be calculated based on the observation mask and the encoding vector; the pixel distribution of the mapping vector is updated according to the gradient. Based on the observation mask (marking the known / missing area) and the encoding vector (device physical characteristics), the deviation gradient of the current completed mapping vector and the ideal completion result is calculated, the gradient direction reflects how to adjust the mapping vector to better conform to the known signal facts (observation mask constraint) and better adapt to the device imaging law (encoding vector constraint), and the gradient size represents the deviation degree. Adjust the numerical distribution (i.e. pixel distribution, corresponding to the numerical representation of the structural features in the latent space) of the mapping vector along the gradient direction, so that the features of the mapping vector in the missing area tend to a reasonable state that conforms to both the known signal correlation and the device physical characteristics. For example, if the gradient shows that the blood vessel features in a missing area deviate greatly from the known blood vessel segment direction, the vector values of the area are adjusted along the gradient direction to make the completed blood vessel direction more continuous. Through gradient calculation based on the observation mask, it is ensured that the known area features of the mapping vector are highly consistent with the predicted Gaussian point array distribution (reducing structural deviation); based on the gradient adjustment of the encoding vector, the completion features strictly adapt to the device acoustic characteristics (such as the reflection intensity range conforming to the device sensitivity), greatly reducing the false structures that conform to the statistical law but violate the physical facts. Of course, the gradient calculation and updating steps can also not be performed, which is not limited in this embodiment.

[0093] Step 69, input the completed mapping vector into the latent space decoder to reconstruct the three-dimensional structure information and generate the completed Gaussian point array.

[0094] The completed mapping vector is a low-dimensional vector output by the latent space diffusion model, which contains reliable structure information of the known sampling area and structure information of the completed missing area. The latent space decoder is called to reversely reconstruct the low-dimensional abstract features in the completed mapping vector into a high-dimensional and discrete completed Gaussian dot array: that is, from the latent vector of hundreds of dimensions, the specific attributes (position (x, y, z), amplitude, scale, etc.) of tens of thousands to hundreds of thousands of Gaussian dots are decoded, and the spatial distribution of these Gaussian dots completely corresponds to the three-dimensional anatomical structure of the target tissue (such as the shape of the liver, the running of the blood vessels, and the position and reflection intensity of the lesion), realizing the conversion of abstract features to concrete structures. Then step 65 is adaptively adjusted to generate a mapping relationship between the full-space sampling ultrasound data and the distribution of the full-space Gaussian dot array according to the completed Gaussian dot array. The input of image rendering is adjusted to the completed Gaussian dot array, solving the problem of incomplete and fuzzy details in the rendered image caused by the lack of structure information in the extremely sparse sampling scenario, and through the generation of the completed complete structure representation, the final output ultrasound image is significantly improved in integrity, detail accuracy and clinical usability.

[0095] Based on the above introduction, the method provided by the embodiment maps the predicted Gaussian dot array distribution into a low-dimensional latent vector through the latent space encoder, realizes efficient compression and feature extraction of high-dimensional structure information, and lays a structured foundation for subsequent completion; then the observation mask accurately marks the sampling position and the missing area, which defines the factual boundary for the generation process and avoids false generation that deviates from the real signal; then the latent space diffusion model, under the dual constraints of the observation mask and the encoded vector (device physical characteristics), completes the latent vector through the diffusion inverse process, ensures the structure fidelity of the known area, and reasonably infers the missing area based on the tissue structure priori (such as blood vessel continuity and organ shape regularity) to fill the information gap caused by sparse sampling; finally, the latent space decoder is called to finally reconstruct the completed latent vector into a complete Gaussian dot array, so that the rendered image can not only retain the real signal characteristics of the extremely sparse spatial observation sequence, but also restore the missing three-dimensional structure details.

[0096] The method and the technology realize the following through the full-link design of compression-constraint-completion-reduction: in the 1 / 30 or extremely sparse sampling scenario, the image integrity is significantly improved, the structure is broken and the information is eliminated (such as the missing part of the completed blood vessel), the detail performance is enhanced, the key diagnostic features such as the micro-lesion and the tissue texture are restored, and the physical and clinical consistency is ensured, the completion result conforms to the device acoustic characteristics and follows the anatomical rules, and the false structure is avoided to interfere with the diagnosis. This design breaks through the dependence of traditional reconstruction on high-density sampling, reduces the acquisition pressure of the device, significantly improves the clinical diagnostic value of sparse ultrasound images, and provides an innovative solution for low-cost and high-fidelity ultrasound imaging.

[0097] Embodiment six: The full-space sampling ultrasound data is essentially a sequence of discrete signals (such as RF signals), and direct conversion to a Gaussian point array may cause a signal-structure mapping disconnection problem due to lack of structured association. To further solve the representation of the original ultrasound data and the point array distribution, before performing step 61 of mapping the full-space Gaussian point array distribution to the latent space by using the latent space encoder, the following steps can be performed first: Step 58, based on the Gaussian probability field and the Gaussian attribute field, implicit structured modeling is performed according to the full-space sampling ultrasound data to obtain a full-space Gaussian point array probability representation corresponding to the full-space sampling ultrasound data.

[0098] Before the latent space encoder maps the full-space Gaussian point array distribution, based on the Gaussian probability field (describing the probability of the structure at each position in the space) and the Gaussian attribute field (describing the acoustic properties of the structure, such as reflection intensity, acoustic impedance), the signal characteristics (such as RF signal amplitude, frequency domain distribution) of the full-space sampling ultrasound data are combined to perform implicit structured modeling on the three-dimensional structure of the biological tissue. By calculating the correlation degree of the Gaussian probability and the attribute at each position in the space, a full-space Gaussian point array probability representation is generated, which can represent the corresponding relationship between the structure existence probability and the acoustic attribute.

[0099] Step 59, according to the full-space Gaussian point array probability representation, a full-space Gaussian continuous probability field corresponding to the full-space sampling ultrasound data is generated.

[0100] According to the full-space Gaussian point array probability representation, a full-space Gaussian continuous probability field covering the entire imaging space with continuously changing parameters is constructed through interpolation, smoothing and other mathematical processing, so as to eliminate the local information discontinuity caused by discrete sampling. The specific processing method used in this embodiment is not limited, and can be set according to the use requirements of the actual application scene.

[0101] Step 60, according to the full-space Gaussian continuous probability field, a full-space Gaussian point array distribution is obtained.

[0102] From the continuous probability field, the high-probability region (corresponding to the position of the real tissue structure) is extracted, and the position, covariance, density and other core parameters of the Gaussian ellipsoid in each region are determined to generate a full-space Gaussian point array distribution that can accurately match the full-space sampling ultrasound data, providing high-quality input for subsequent latent space coding.

[0103] This method deeply binds the signal characteristics (such as amplitude, frequency domain) of the ultrasound data and the structure parameters (position, density) and acoustic parameters (covariance) of the point array through double modeling of the Gaussian probability field and the attribute field, and the matching degree (such as L1 / L2 loss, SSIM) of the synthesized signal and the original full-space sampling ultrasound data is significantly improved when the generated point array distribution is verified by the differentiable acoustic propagation model, which can solve the core problem of structure and signal disconnection in traditional modeling.

[0104] Example Seven: In order to ensure the spatial continuity and structural integrity of the prediction results, this embodiment proposes a specific implementation method for converting the extremely sparse spatial observation sequence into a precise predicted Gaussian lattice distribution. Step 102 inputs the extremely sparse spatial observation sequence into the pre-trained generative model to generate the predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence, which can be performed according to the following steps: Step 21, input the extremely sparse spatial observation sequence into the pre-trained generative model to generate the predicted Gaussian continuous field corresponding to the extremely sparse spatial observation sequence by the pre-trained latent space diffusion model in the generative model.

[0105] The preprocessed extremely sparse spatial observation sequence (including the standardized signal tensor and the observation mask marking the missing area) is input into the pre-trained generative model. The model calls the latent space diffusion model trained on the full space data, and performs reverse denoising iteration in the latent space with the extremely sparse observation signal features + system response encoding vector as constraints, to generate a predicted Gaussian continuous field that can represent the probability of the corresponding tissue structure and acoustic properties of the extremely sparse observation, covering the entire imaging space, and the parameters change continuously to avoid discrete discontinuity.

[0106] Step 22, use the pre-trained latent space decoder in the generative model to convert the predicted Gaussian continuous field into the initial predicted Gaussian lattice distribution.

[0107] The pre-trained latent space decoder in the generative model discretizes the continuous field, extracts the parameters (such as position, covariance, density) of the high-probability structure region in the continuous field, and converts them into the initial predicted Gaussian lattice distribution that reflects the initial tissue morphology.

[0108] Step 23, fine-tune the initial predicted Gaussian lattice distribution using the mapping relationship to obtain the predicted Gaussian lattice distribution.

[0109] Call the full-space sampling ultrasound data-full-space Gaussian lattice distribution mapping relationship established in the pre-training stage, and use the full-space structure prior (such as tissue geometric rules, acoustic property correlation) contained in the mapping relationship as a reference to fine-tune and calibrate the parameters (such as adjusting the Gaussian point position in the edge region, optimizing the density in the lesion area) of the initial predicted Gaussian lattice distribution. Finally, the predicted Gaussian lattice distribution that is highly matched with the extremely sparse observation and conforms to the full-space structure rules is obtained.

[0110] The method first generates a continuous field and then converts the lattice, which can fill the information gap of sparse observation relying on the spatial continuity of the continuous field, and further calibrates the deviation by mapping fine-tuning, so that the structural resolution (such as the sharpness of the lesion boundary) and the signal matching degree (the similarity between the synthesized signal and the original extremely sparse observation) of the predicted lattice are significantly improved, which can solve the core problem of low precision of sparse data reconstruction in traditional methods and improve the accuracy of the predicted lattice; at the same time, the method introduces the full-space physical prior through the mapping relationship fine-tuning, which ensures that the predicted lattice not only fits the extremely sparse observation signal, but also meets the physical laws of ultrasonic wave propagation and tissue morphology, effectively reducing false lesions, boundary misplacement and other artifacts, and improving the clinical reliability of the predicted results.

[0111] Embodiment eight: The image rendering step based on the Gaussian lattice in the above embodiments is not limited, and can be implemented according to related technologies. In order to realize the precise regulation and control of the clear key details and the smooth background noise, an embodiment of the present application proposes a fine image rendering method based on Gaussian lattice weight grading, which distinguishes the importance of Gaussian points and adopts a differential filtering strategy to realize high-quality reconstruction of the ultrasound image.

[0112] Specifically, step 103 performs image rendering according to the predicted Gaussian lattice distribution to obtain the ultrasound image reconstructed based on the extremely sparse spatial observation sequence, which can be performed according to the following steps: Step 31, from the predicted Gaussian lattice distribution, screening out high-weight Gaussian points with learnable weights reaching a preset threshold and low-weight Gaussian points not reaching the preset threshold.

[0113] According to the learnable weight of the Gaussian point (reflecting the importance of the point to the representation of the tissue structure, which is obtained by model training), the predicted Gaussian lattice distribution is divided into two categories, i.e. high-weight Gaussian points (corresponding to key structures such as lesion edges and blood vessel trunks) with weights higher than the threshold and low-weight Gaussian points (corresponding to secondary structures such as background tissues and blurred textures) with weights lower than the threshold.

[0114] Step 32, performing Gaussian adaptive filtering on the high-weight Gaussian points and Gaussian smoothing filtering on the low-weight Gaussian points.

[0115] GAF (Gaussian adaptive filtering) is adopted for the high-weight Gaussian points to retain their fine structure features (such as sharp boundaries and subtle morphological changes) by dynamically adjusting the filter kernel size and intensity; GPF (Gaussian smoothing filtering) is adopted for the low-weight Gaussian points to reduce noise by fixed wide kernel filtering and strengthen their smoothness as background.

[0116] Step 33, pixel-level weighted fusion of the fine-scale image corresponding to the filtered high-weight points and the coarse-scale image corresponding to the filtered low-weight points to obtain the reconstructed ultrasound image.

[0117] The filtered high-weight key fine-scale image (containing details) and the low-weight key coarse-scale image (containing background) are fused at the pixel level according to the weight, and finally a reconstructed ultrasound image considering the clarity of details and the smoothness of background is generated.

[0118] In order to further improve the image rendering efficiency and reduce the invalid calculation cost, a Gaussian point screening step can be added before step 51. The target voxel region coordinate is determined by performing step 50. The octree index is constructed according to the target voxel region coordinate. The Gaussian points in the predicted Gaussian point array distribution located in the target voxel region of interest are screened out according to the octree index, and the Gaussian points not in the target voxel region of interest are removed as the core Gaussian point array.

[0119] The target voxel region of interest coordinate refers to the coordinate boundary of a specific three-dimensional space range that needs to be paid attention to and finely rendered, which is clearly defined in the three-dimensional ultrasound imaging space according to the clinical diagnosis requirements or imaging target. In step 50, according to the clinical diagnosis requirements (such as the lesion position marked by the doctor, the preset organ imaging range), the voxel region coordinate range (such as the three-dimensional coordinate boundary of the liver lesion region) that needs to be rendered is determined in the imaging space. The octree is a data structure for efficient retrieval of three-dimensional space data. By recursively dividing the imaging space into 8 sub-cubes (voxel blocks), an index is established for each sub-cube to quickly locate the spatial block corresponding to the target region of interest. Based on the octree index, the Gaussian points in the predicted Gaussian point array distribution whose position coordinates fall within the target region of interest are quickly matched, and the background Gaussian points outside the region (such as the Gaussian points corresponding to the muscle and fat tissue around the lesion) are removed to form a core Gaussian point array containing only the core region structure information. After screening by the octree index, the data amount of the core Gaussian point array can be reduced by 60%-80%. The subsequent weight screening, differential filtering and other steps are only performed on the core region, and the calculation efficiency is significantly improved, especially suitable for clinical real-time reconstruction scenarios. Step 51 is correspondingly adjusted to screen out high-weight Gaussian points with learnable weights reaching a preset threshold and low-weight Gaussian points not reaching the preset threshold from the core Gaussian point array.

[0120] Based on the above introduction, the method provided in the embodiment implements fine processing according to the objective difference of the learnable weight of the Gaussian point, adopts Gaussian adaptive filtering for the high-weight Gaussian point bearing key structural information of the image, retains the sharpness and integrity of the subtle anatomical features (such as the lesion edge and the blood vessel branch) by dynamically adjusting the filtering parameters, and adopts Gaussian smoothing filtering for the low-weight Gaussian point constituting the background to suppress noise and sparse sampling artifacts by wide kernel processing, so as to ensure the visual consistency of the background area. Finally, the natural transition of the fine-scale key details and the coarse-scale smooth background is realized through pixel-level weighted fusion, which not only strengthens the detail performance of the diagnostic core area, but also reduces the background noise interference, and at the same time, the level differentiation of the tissue structure is enhanced through weight regulation. The method breaks through the limitation that the traditional uniform filtering cannot balance details and noise, and in the sparse sampling scene, it can not only accurately retain the key morphological features required for clinical diagnosis, but also ensure the visual coherence and noise robustness of the whole image, which significantly improves the clinical practical value of the reconstructed ultrasound image.

[0121] Embodiment Nine Under extremely sparse sampling, the information of the extremely sparse spatial observation sequence is limited, which may lead to a deviation that is physically reasonable but clinically unreasonable in the prediction of the Gaussian point array distribution in the signal-uncovered area. For example, due to the signal loss in a certain area, the optimization algorithm may adjust the Gaussian point amplitude of the normal liver tissue to be close to the high-reflection range of the tumor, which cannot be found from the signal matching perspective, but it violates the clinical tissue characteristics. In order to avoid the limitation of signal threshold optimization and improve the clinical credibility, in the embodiment, the following steps can be further performed before the step 103 of obtaining the ultrasound image reconstructed based on the extremely sparse spatial observation sequence according to the prediction of the Gaussian point array distribution: Step 109, performing structure decoding processing on the predicted Gaussian point array distribution based on a tissue structure prior model to obtain a decoded point array; and performing image rendering according to the decoded Gaussian point array.

[0122] The tissue structure prior model is a preset parameter independent of the device-specific parameter, contains the statistical law of the acoustic reflectivity of the human tissue, such as the reflectivity difference of the bone / muscle / fat, and the morphological characteristics of the anatomical structure, such as the continuity of the organ boundary and the smoothness of the blood vessel running. The tissue structure prior model is independent of the device-specific parameter and integrates the universal human tissue characteristics, which can be adapted to the data collected by different ultrasound devices (such as probes of different manufacturers). Even if the device model is changed to cause the signal characteristics to change, the tissue structure prior model can still correct the structural deviation based on the unified clinical standard, ensure that the reconstructed images output by different devices remain consistent in clinical interpretation, and enhance the cross-device applicability of the method.

[0123] Step 109 introduces a pre-trained tissue structure prior model to perform clinical rationality verification and correction on the predicted Gaussian lattice distribution. The correction corrects reasonable but unreasonable deviations caused by signal sparsity, for example, if the amplitude of a Gaussian point in the optimized lattice in a certain region deviates significantly from the prior statistical range of the normal tissue in that region (e.g., bone-level high reflectivity appears in the liver region), the amplitude is adjusted to a reasonable range by the prior model; if the structure morphology is broken (e.g., the blood vessel suddenly stops), the Gaussian point position is completed based on the anatomical morphology prior, and finally the decoded lattice with signal matching and clinical rationality is output, which is used for image rendering. By introducing clinical prior knowledge, the limitations of pure data-driven optimization in sparse scenarios are compensated, and the reconstructed results are further clinically reliable based on physical fidelity.

[0124] Embodiment ten: The characteristics (such as probe element arrangement, signal sampling rate) of different ultrasound devices differ significantly. In order to improve the stability of cross-device reconstruction accuracy, this embodiment proposes to further add an adaptation method when the ultrasound device is changed, which specifically includes the following steps: Step 110, obtaining the device parameters of the current ultrasound device.

[0125] When the ultrasound acquisition device is changed, the device parameters related to the reconstruction of the ultrasonic image sampled in the extremely sparse space are accurately collected, and the collected device parameters include two types: one is device-specific parameters, such as transducer geometry (number of elements, element spacing, element arrangement method such as linear / convex array / phased array), transducer inherent characteristics (focal length, bandwidth, sensitivity curve, point spread function), hardware limitation parameters (maximum imaging depth, transverse / longitudinal resolution limit); the second is the core acquisition configuration parameters, such as sampling rate, sound speed estimation preset value, signal acquisition channel number, etc.

[0126] Step 111, based on the device parameters, adjusting the parameter extraction dimension and encoding mapping relationship in the pre-trained system response encoding model to obtain an updated encoding model; the system response encoding model is used to generate an encoding vector corresponding to the system parameters.

[0127] The system response encoding model is a model used to convert the system parameters (especially the device-specific parameters) of the ultrasound device into an encoding vector. The core function is to extract the key features of the device hardware characteristics (such as the beam direction determined by the transducer geometry, the signal resolution determined by the sampling rate), and convert them into an encoding vector recognizable by the forward acoustic propagation model through an encoding mapping relationship. In this embodiment, the specific model type and model structure of the system response encoding model are not limited, and can be set according to the data processing needs in the actual application scenario.

[0128] The system response coding model has the ability to extract device parameter characteristics and encode after pre-training, but this ability is based on the device types covered in the multi-scene ultrasound data. When a new device is encountered, whose parameter characteristics are different from the devices in the training data, the parameter extraction dimension and encoding mapping relationship of the original model may not be suitable. To this end, the pre-trained model is fine-tuned based on the actual parameters of the new device (such as element spacing, special sampling rate, etc.) in this embodiment. On the one hand, the parameter extraction dimension is adjusted (new or deleted device parameter characteristic dimensions are extracted), to ensure that the core difference characteristics of the new device can be captured. On the other hand, the encoding mapping relationship is adjusted (the conversion rule from features to vectors is optimized), to ensure that the encoding vector can accurately reflect the hardware characteristics of the new device. When the subsequent forward acoustic propagation model adjusts the parameters based on the encoding vector, it can accurately reproduce the physical laws of sound wave propagation of the new device, avoid the physical modeling deviation caused by the distortion of the encoding vector, and provide protection for the stability of the cross-device reconstruction accuracy.

[0129] Step 112, calling the updated coding model to encode the system parameters to generate a new encoding vector that adapts to the new ultrasound device.

[0130] The updated coding model is a special encoder customized and optimized for the characteristics of the new ultrasound device (the parameter extraction dimension and mapping relationship have been adjusted), and the system parameters refer to the core hardware parameters of the new device (such as transducer element arrangement, sampling rate, sound speed estimate, etc.). This step calls this customized model to perform a series of feature extraction-quantization mapping-dimension compression processes on the system parameters of the new device, and finally outputs a low-dimensional, structured new encoding vector.

[0131] Step 113, adjusting the model parameters of the forward acoustic propagation model according to the new encoding vector.

[0132] The core function of the forward acoustic propagation model is to simulate the physical process of sound waves being emitted from the probe, interacting with the tissue, and being received by the probe. Its simulation accuracy completely depends on the matching degree of the internal model parameters (such as beam direction, propagation time delay calculation coefficient, energy attenuation factor, etc.) and the actual device characteristics. The new encoding vector is a mathematical condensation of the hardware characteristics of the new device (such as probe element arrangement, sampling rate). This step analyzes the feature information in the vector and adjusts the key parameters of the model in reverse, so that the physical simulation logic of the model is consistent with the actual working rules of the new device. The adjusted forward acoustic propagation model can generate a synthetic observation signal that is highly matched in physical laws with the extremely sparse spatial observation sequence collected by the new device based on the hardware characteristics of the new device, ensuring that the reconstructed ultrasound image after replacing the new device still maintains high fidelity and clinical usability.

[0133] For example, if the new encoding vector reflects that the new device array element spacing is smaller (0.2mm) and the beam focusing is more accurate, the beam spread angle related parameter in the forward acoustic propagation model is adjusted, and the simulated beam coverage range is reduced to match the focusing characteristics of the new device.

[0134] Based on the above introduction, the method provided by the embodiment adjusts the parameter extraction dimension and mapping relationship of the encoding model after replacing the new ultrasonic device, so that the encoding vector can accurately capture the hardware characteristics (such as the element difference between the convex array probe and the linear probe) of the new device, and then the physical modeling of the forward acoustic propagation model is adapted to the new device, avoiding the increase of signal domain loss caused by device replacement, ensuring that the reconstruction accuracy remains consistent on different devices; At the same time, the method integrates the tissue feature rules under different devices, which can not only correct the deviation caused by the signal sparsity of the new device, but also ensure the uniformity of the clinical interpretation standard of the output images of different devices (such as the consistency of the reflectivity gray scale range of the same tissue), solving the clinical pain point that doctors need to adapt to the image features after replacing the device.

[0135] Embodiment eleven: To further clarify the technical logic of the foregoing method embodiment, the embodiment combines the core representation advantages of sparse differentiable Gaussian splatting (3D Gaussian Splatting) to construct a complete technical process covering data acquisition-model training-image reconstruction. The specific implementation steps are as follows: 1. Constructing a full-space sampled ultrasound data set, such as Figure 5 As shown in a full-space sampled ultrasound data set construction scheme diagram.

[0136] This step realizes high-quality ultrasound data acquisition, real reference (Ground Truth) image generation, and computationally efficient three-dimensional reconstruction by building a translation-rotation scanning ultrasonic imaging system, combining Gaussian splatting modeling and physical optimization. Through adaptive optimization and CUDA acceleration, the data set availability is improved, laying the foundation for clinical real-time application, which is divided into two parts: (1) Obtaining original ultrasound data.

[0137] An ultrasound imaging platform with translation-rotation scanning capability is built. First, experimental data are obtained by combining a phantom. Then, a clinical focused ultrasound system is built. The system is configured with a translation stage and a rotation stage to give the probe translation and rotation scanning capabilities. An outer imaging cavity is added to ensure acoustic coupling and not to interfere with scanning. The timing of the control module is adjusted to synchronize scanning and imaging. After completing small-scale clinical tests and optimizing imaging parameters to meet clinical standards, large-scale clinical sample imaging experiments are carried out based on the system to obtain diversified clinical data. The core of the ultrasound imaging system includes a signal excitation module, an ultrasound signal detection module, and a control module. When working, the control module coordinates the timing of each module, controls the signal excitation module to output ultrasound signals (acting on the imaging target), and controls the ultrasound signal detection module to record the ultrasound signals reflected / scattered by the target to generate raw ultrasound data (such as RF signals).

[0138] (2) Calculate the true reference image corresponding to the original data.

[0139] Considering that representing a three-dimensional image with dense voxels will result in storage and memory consumption beyond the current technical level, the present process uses a 3D Gaussian lattice (3D GS-Based) to parameterize a three-dimensional ultrasound image, converts a three-dimensional biological tissue structure into a set of Gaussian distributions with attributes, and simulates its morphological structure characteristics. The Gaussian lattice representation matching the original ultrasound data is obtained through an optimization method as the ground truth of the data set. In the verification stage, CT and MRI images of a small number of samples are also obtained to cross-verify the reliability of the reconstructed images. The core of this process is to physically model the mapping relationship between the original ultrasound data and the high-resolution image, simulate the propagation process of ultrasound waves in biological tissues, and specifically include: Forward propagation modeling: a forward acoustic propagation model based on Gaussian ellipsoid is constructed to simulate the propagation process of ultrasound waves and calculate the initial parameters of the Gaussian lattice corresponding to the ultrasound signals; Backward gradient optimization: a gradient backpropagation optimization strategy is used to adjust the position, covariance, density, and other parameters of the Gaussian ellipsoid to ensure accurate mapping of the Gaussian lattice to the original ultrasound signals.

[0140] At the same time, a dynamic Gaussian ellipsoid generation mechanism is designed: for key areas, Gaussian points are duplicated and split (to realize local structure refinement and improve resolution), and for redundant areas, pruning and optimization are performed (to reduce the consumption of computing resources), and by controlling the growth curve of the number of Gaussian lattices, fine modeling is realized under the premise of controllable calculation amount. In addition, a rendering module is designed to convert the Gaussian lattice to a voxel grid (Voxel) to ensure compatibility with traditional image processing methods and optimize the rendering calculation process to improve real-time display capability in clinical applications.

[0141] 2, extremely sparse spatial sampling ultrasound reconstruction, such as Figure 6A flowchart of an extremely sparse spatial sampling ultrasound reconstruction method is shown.

[0142] After the dataset construction is completed, the generative model is pre-trained in an unconditional scenario (to capture the joint distribution of a large number of Gaussian point arrays in a self-supervised manner), then conditional sampling is performed based on unobserved sparse original ultrasound data constraints, and the gold standard Gaussian point array is used to fine-tune the generative model. The core innovation of this link is to model the discrete and unstructured Gaussian point array (3DGS) as a continuous probability field, to build a new reconstruction paradigm based on decoupled Gaussian function representation and generative modeling, and to realize high-quality reconstruction under extremely sparse measurement through an encoder-generator-decoder three-module, as follows: (1) Construct a Gaussian point array continuous representation method An uncoupled continuous function space representation scheme is used to implicitly structure the discrete Gaussian point array: a Gaussian probability field (GPF) and a Gaussian attribute field (GAF) are constructed to probabilistically model the geometric characteristics of any position in three-dimensional space. The Gaussian attribute field not only predicts the tissue density characteristics, but also includes the covariance parameters related to geometric transformation. The mathematical advantage of this method is that it converts the discrete Gaussian point array into a differentiable implicit representation through a continuous field function, providing a structured probability space for subsequent generative modeling.

[0143] (2) Construct a Gaussian continuous field diffusion generative model A hierarchical generative model is constructed in the continuous function space to realize data-driven probability field synthesis. The generation process includes two steps of variational autoencoder (VAE) compression representation and latent space diffusion modeling: Through the VAE architecture, the Gaussian continuous field is mapped to the latent space by the encoder, and then the Gaussian field is reconstructed by the decoder; The latent space diffusion model (LDM) is trained in the latent space, and through the step-by-step injection of noise-reverse denoising mechanism, it learns the data distribution rules to generate high-quality reconstruction data.

[0144] This model can adapt to different types of sparse original ultrasound data input (such as spatial spectrum loss, under-sampling, etc.) by adjusting the conditional / unconditional generation strategy.

[0145] (3) Realize the mapping of Gaussian continuous field to Gaussian point array A geometric perception discretization algorithm is designed to efficiently convert the continuous field to an explicit Gaussian point array: adaptive spatial subdivision is performed based on the Gaussian probability field (GPF), and an octree algorithm is used for geometric sampling, while the reconstruction accuracy is further improved through optimization of attribute prediction and covariance constraints. Compared with classic methods such as MarchingCubes isosurface extraction, this algorithm can provide more fine-grained geometric adaptability and geometric- appearance coupled attributes, significantly improving the accuracy and usability of three-dimensional reconstruction.

[0146] (4) Two-stage reconstruction procedure Pre-training stage: The VAE-LDM architecture is pre-trained unsupervised on a large-scale Gaussian lattice representation of 3D ultrasound image dataset, learning the Gaussian field intrinsic distribution corresponding to the ultrasound image; at the same time, the mapping relationship between the original ultrasound data and the Gaussian continuous field latent variable is established through contrastive learning, providing a basis for the inference stage.

[0147] Inference stage: Based on the spatially sparse original ultrasound data, the conditional probability field is generated by adjusting the diffusion process; according to the actual application scenario, different types of spatial spectrum sparse sampling are simulated using complete original ultrasound data to fine-tune the generative model; finally, the probability field generated by the decoder is converted into an explicit Gaussian lattice, which is input into the differentiable renderer to perform real-time image rendering and reconstruction, and the final ultrasound image is output.

[0148] Embodiment twelve: The embodiment relates to an ultrasound image reconstruction device for extremely sparse spatial sampling, and a schematic diagram of the ultrasound image reconstruction device for extremely sparse spatial sampling in the embodiment can be as shown in the figure, which comprises a data acquisition module 201, a Gaussian lattice generation module 202 and an image rendering module 203. Figure 7

[0149] The data acquisition module 201 is used for acquiring an extremely sparse spatial observation sequence collected under extremely sparse sampling conditions; The Gaussian lattice generation module 202 is used for inputting the extremely sparse spatial observation sequence into a pre-trained generative model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence; the generative model has a mapping relationship between full spatial sampling ultrasound data and full spatial Gaussian lattice distribution after pre-training; The image rendering module 203 is used for performing image rendering according to the predicted Gaussian lattice distribution to obtain an ultrasound image reconstructed based on the extremely sparse spatial observation sequence.

[0150] It should be noted that the content in the ultrasound image reconstruction device for extremely sparse spatial sampling provided in the embodiment can be mutually referred to the ultrasound image reconstruction method for extremely sparse spatial sampling provided in the above-mentioned embodiments, and repeated parts will not be described herein.

[0151] ​The device for ultrasonic image reconstruction provided by the embodiment is used for extremely sparse spatial sampling, and the data acquisition module can be compatible with an ultrasonic imaging platform with a translation-rotation scanning capability (without relying on an expensive unfocused transducer array), can directly acquire an extremely sparse spatial observation sequence, and is suitable for a low-cost hardware application scenario such as a primary medical institution; the Gaussian point array generation module can use a full-space structure prior to complete missing information of the extremely sparse observation based on a pre-training generation model, generate a precise predicted Gaussian point array distribution, and avoid problems such as structure ambiguity and many artifacts caused by sparse data in a traditional device; and the image rendering module can quickly output an ultrasonic image meeting clinical requirements based on the predicted Gaussian point array, and the parameterized representation of the Gaussian point array can be compatible with a traditional voxel grid processing method.

[0152] In addition, it should be noted that each module involved in the embodiment is a logical module, and in actual application, one logical unit can be one physical unit, or a part of one physical unit, or realized by combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units not closely related to solving the technical problems proposed in the present application are not introduced in the embodiment, but this does not mean that there are no other units in the embodiment.

[0153] Embodiment thirteen Another embodiment of the present application relates to an electronic device, such as Figure 8 As shown in the figure, the electronic device comprises at least one processor 301 and a memory 302 connected with the at least one processor 301; the memory 302 stores instructions executable by the at least one processor 301, and the instructions are executed by the at least one processor 301 to enable the at least one processor 301 to perform the steps of the ultrasonic image reconstruction method for extremely sparse spatial sampling in each of the above embodiments.

[0154] The memory and the processor are connected in a bus mode, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits together, which are well known in the art, and therefore, they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0155] The processor is responsible for managing the bus and general processing, and can also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory can be used to store data used by the processor when executing its operations.

[0156] It is to be understood that the above-described embodiments are merely illustrative of the principles of the application, and that numerous and various modifications can be made by those skilled in the art without departing from the spirit and scope of the application.

Claims

1. A method for ultrasound image reconstruction using extremely sparse spatial sampling, characterized in that, include: Acquire extremely sparse spatial observation sequences obtained under extremely sparse sampling conditions; The extremely sparse spatial observation sequence is input into a pre-trained generative model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence; the generative model is pre-trained to have a mapping relationship between full-space sampled ultrasound data and full-space Gaussian lattice distribution; Image rendering is performed based on the predicted Gaussian lattice distribution to obtain an ultrasound image reconstructed based on the extremely sparse spatial observation sequence.

2. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 1, characterized in that, The generative model obtains the mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution, including: Using an ultrasound imaging platform with translation-rotation scanning capabilities, full-space sampling ultrasound data corresponding to the phantom were acquired; The full-space sampled ultrasound data is parametrically represented and transformed using a trained physical mapping model to obtain the full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data. Determine the mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution.

3. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 2, characterized in that, Training the physical mapping model includes: A forward acoustic propagation model based on a Gaussian ellipsoid is constructed, and the full-space sampled ultrasound data is input into the forward acoustic propagation model to determine the initial full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data; An optimization strategy based on gradient backpropagation is adopted. Based on the biological tissue morphology characteristics corresponding to the full-space sampled ultrasound data, the forward acoustic propagation model is adaptively optimized to determine the full-space Gaussian lattice distribution corresponding to the full-space sampled ultrasound data. The parameters include at least one of the position, covariance, and density of the Gaussian ellipsoid.

4. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 2, characterized in that, The generative model includes a latent space encoder, a latent space diffusion model, and a latent space decoder. Determining the mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution includes: The latent space encoder is used to map the full-space Gaussian lattice distribution to the latent space to obtain a low-dimensional latent vector; The low-dimensional latent vectors in the latent space are trained using the latent space diffusion model to obtain generative model parameters that conform to the statistical law of the distribution of Gaussian points in the whole space. Based on the trained latent space diffusion model, the low-dimensional latent vector is denoised in reverse to obtain the completed latent vector; The latent space decoder is used to reconstruct the full-space Gaussian lattice distribution based on the completed latent vector; Based on the reconstructed full-space Gaussian lattice distribution, a mapping relationship is generated between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution.

5. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 4, characterized in that, After reconstructing the full-space Gaussian lattice distribution using the latent space decoder based on the completed latent vector, the method further includes: The reconstructed full-space Gaussian point distribution is input into the latent space encoder in the pre-trained latent space generation and completion model. The latent space encoder is used to perform low-dimensional latent vector mapping of the three-dimensional structural information to obtain the mapping vector. Extract the observation mask corresponding to the extremely sparse spatial observation sequence, and mark the sparse sampling position and missing region in the observation mask; The labeled observation mask and encoding vector are input into the latent space diffusion model as constraints for the inverse diffusion process. The latent space diffusion model calculates the inverse diffusion process of the mapping vector based on the constraints to obtain the completed mapping vector. The completed mapping vector is input into the latent space decoder to reconstruct the three-dimensional structural information and generate a completed Gaussian matrix. The step of generating a mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution based on the reconstructed full-space Gaussian lattice distribution specifically involves generating a mapping relationship between the full-space sampled ultrasound data and the full-space Gaussian lattice distribution based on the completed Gaussian lattice.

6. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 4, characterized in that, Before mapping the full-space Gaussian lattice distribution to the latent space using the latent space encoder, the process includes: Based on the Gaussian probability field and Gaussian attribute field, implicit structured modeling is performed on the full-space sampled ultrasound data to obtain the full-space Gaussian lattice probability representation corresponding to the full-space sampled ultrasound data; Based on the full-space Gaussian lattice probability representation, a full-space Gaussian continuous probability field corresponding to the full-space sampled ultrasound data is generated. The full-space Gaussian lattice distribution is obtained based on the full-space Gaussian continuous probability field.

7. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 1, characterized in that, The step of inputting the extremely sparse spatial observation sequence into a pre-trained generative model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence includes: The extremely sparse spatial observation sequence is input into a pre-trained generative model to generate a predicted Gaussian continuous field corresponding to the extremely sparse spatial observation sequence through a pre-trained latent space diffusion model in the generative model. The predicted Gaussian continuous field is converted into an initial predicted Gaussian lattice distribution using the pre-trained latent space decoder in the generative model. The initial predicted Gaussian point matrix distribution is fine-tuned using the mapping relationship to obtain the predicted Gaussian point matrix distribution.

8. The ultrasound image reconstruction method for extremely sparse spatial sampling according to claim 1, characterized in that, The step of rendering the image based on the predicted Gaussian lattice distribution to obtain the reconstructed ultrasound image based on the extremely sparse spatial observation sequence includes: From the predicted Gaussian point distribution, select high-weight Gaussian points whose learnable weights reach a preset threshold, and low-weight Gaussian points whose learnable weights do not reach the preset threshold; Gaussian adaptive filtering is performed on the high-weight Gaussian points, and Gaussian smoothing filtering is performed on the low-weight Gaussian points; The filtered high-weighted image corresponding to the fine scale and the filtered low-weighted image corresponding to the coarse scale are then fused at the pixel level to obtain the reconstructed ultrasound image.

9. An ultrasound image reconstruction device for extremely sparse spatial sampling, characterized in that, include: The data acquisition module is used to acquire extremely sparse spatial observation sequences obtained under extremely sparse sampling conditions; A Gaussian lattice generation module is used to input the extremely sparse spatial observation sequence into a pre-trained generative model to generate a predicted Gaussian lattice distribution corresponding to the extremely sparse spatial observation sequence; the generative model is pre-trained to have a mapping relationship between full-space sampled ultrasound data and full-space Gaussian lattice distribution; The image rendering module is used to render the image based on the predicted Gaussian dot matrix distribution to obtain the reconstructed ultrasound image based on the extremely sparse spatial observation sequence.

10. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the ultrasound image reconstruction method for extremely sparse spatial sampling as described in any one of claims 1 to 8.

Images