An ultrasonic three-dimensional surface reconstruction method based on point cloud registration

Abstract

The application discloses a kind of based on point cloud registration ultrasonic three-dimensional surface reconstruction method, by multimodal image acquisition system, with sensor-free free scanning mode to obtain two-dimensional ultrasonic image sequence and high signal-to-noise ratio magnetic resonance image, after pretreatment, segmentation, coordinate conversion and denoising, generate initial point cloud dataset;Construct the deep learning model of fusion three-level point cloud registration framework, spatial point cloud generation module, neural implicit surface reconstruction module and joint loss function, realize multimodal point cloud high-precision registration and three-dimensional surface reconstruction;The ultrasonic sequence of patient diagnosis and treatment area scanning is acquired, after input model, eliminate point cloud misplacement by three-level registration, output high-precision three-dimensional point cloud by point cloud generation module, finally by neural implicit surface reconstruction module generates smooth complete three-dimensional ultrasonic model, provides accurate three-dimensional anatomical information for clinical diagnosis.The method can significantly improve the precision, smoothness and robustness of sensor-free free scanning ultrasonic three-dimensional reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an ultrasonic three-dimensional surface reconstruction method based on point cloud registration. Background Technology

[0002] Three-dimensional ultrasound imaging, with its unique advantages of being non-invasive, intuitive, and providing three-dimensional anatomical information, has been widely used in various clinical scenarios such as prostate targeted biopsy, vascular assessment, and cardiac and abdominal examinations, becoming an important supporting technology in the field of precision medicine. Among them, sensorless free-scanning three-dimensional ultrasound technology has broken free from the constraints of dedicated sensors and fixed scanning methods, possessing the characteristics of flexible operation, high portability, and wide clinical applicability, and has become a research hotspot in the current field of ultrasound imaging. However, in actual clinical scanning, the free movement of the ultrasound probe is prone to motion disturbances, and two-dimensional ultrasound images themselves have inherent defects such as strong noise, sparse texture, and blurred tissue structure edges. These factors directly lead to problems such as point cloud spatial misalignment, rough reconstructed surface, loss of anatomical details, and topological distortion in the three-dimensional reconstruction process, seriously restricting the accuracy of three-dimensional reconstruction and its clinical application value.

[0003] Traditional ultrasound 3D surface reconstruction methods often employ algorithms such as Marching Cubes to directly extract surface structures from noisy voxels. These methods are susceptible to ultrasound speckle noise and reconstruction voids, resulting in severely jagged surfaces in the generated 3D models. This necessitates additional post-processing steps such as smoothing and interpolation, and the quality of the reconstructed surface is significantly limited by the original voxel resolution. In existing technologies, point cloud registration and surface reconstruction are often independent sequential processes, lacking effective collaborative optimization mechanisms. Accuracy deviations generated during registration cannot be effectively transferred to the reconstruction stage for targeted correction, leading to reconstruction distortions in complex anatomical regions. Furthermore, most reconstruction methods do not fully utilize the morphological prior information from high signal-to-noise ratio modes such as magnetic resonance (MR), making it difficult to effectively compensate for the inherent defects of low signal-to-noise ratio and blurred boundaries in ultrasound images. When dealing with complex clinical scenarios such as large soft tissue deformations and nonlinear probe movements, cross-modal registration accuracy is insufficient, and inter-frame structural registration errors are difficult to control effectively.

[0004] Furthermore, existing sensorless free-scan reconstruction methods have significant limitations: some methods focus solely on probe trajectory estimation while neglecting the optimization of reconstructed surface quality; others rely excessively on complex models, resulting in excessive computational overhead and failing to meet the real-time requirements of clinical end-devices. Simultaneously, traditional registration methods use only geometric distance as a constraint, ignoring the consistency of intensity distribution of anatomical structures and lacking sufficient detail alignment capabilities. The resulting 3D models fail to meet the core requirements of high reconstruction accuracy, good surface smoothness, and complete anatomical structures for clinical lesion localization, preoperative planning, and intraoperative navigation. Therefore, there is an urgent need to provide a tracking-free free-scan 3D ultrasound image reconstruction method that integrates cross-modal prior information and achieves collaborative optimization of point cloud registration and surface reconstruction to overcome the technical bottlenecks of low reconstruction accuracy, rough surfaces, missing details, and poor robustness in existing technologies. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an ultrasound three-dimensional surface reconstruction method based on point cloud registration. This method can significantly improve the accuracy, smoothness, and robustness of sensorless free scanning ultrasound three-dimensional reconstruction, completely eliminating the dependence on external motion sensors, and providing a high-fidelity, easily deployable three-dimensional image analysis tool for precise clinical diagnosis and treatment.

[0006] To achieve the above objectives, this invention provides an ultrasonic three-dimensional surface reconstruction method based on point cloud registration, comprising the following steps: Step 1: Acquire multimodal image dataset; Use a multimodal image acquisition system to acquire two-dimensional ultrasound image sequences and high signal-to-noise ratio magnetic resonance images through sensorless free scanning. After preprocessing, a standardized multimodal image dataset is formed, and then an initial point cloud dataset is generated through segmentation, coordinate transformation, and denoising. Step 2: Construct a 3D ultrasound image reconstruction model; a deep learning model is constructed using a three-level point cloud registration framework, a spatial point cloud generation module, a neural implicit surface reconstruction module, and a joint loss function to achieve high-precision registration of multimodal point clouds and 3D surface reconstruction. Step 3: 3D ultrasound image surface reconstruction; using an image acquisition system, the patient's treatment area is scanned using a sensorless free-scanning trajectory to obtain 2D ultrasound image sequence data; the 2D ultrasound image sequence data is input into the 3D ultrasound image reconstruction model. First, cross-modal precise alignment is completed through a three-level point cloud registration framework to eliminate point cloud misalignment caused by probe motion disturbance and soft tissue deformation; then, a high-precision 3D point cloud is output through the spatial point cloud generation module; finally, a smooth, continuous, and anatomically complete 3D ultrasound model is generated through the neural implicit surface reconstruction module, providing accurate 3D anatomical information for clinical diagnosis.

[0007] As a preferred embodiment, the process of obtaining the multimodal image dataset in step 1 is as follows: S11: Build a multimodal image acquisition system; Build a multimodal image acquisition system using a computer, an ultrasound scanning device, and a magnetic resonance scanning device. The computer is connected to the ultrasound scanning device and the magnetic resonance scanning device respectively for data acquisition and synchronous storage. S12: Multimodal data acquisition; Employing a sensorless free-scanning method, the patient's soft tissue area is scanned using an ultrasound probe. The ultrasound scanning device acquires two-dimensional ultrasound image sequences in real time and sends them to a computer; Simultaneously, high signal-to-noise ratio magnetic resonance images are acquired through a magnetic resonance scanning device as morphological prior data; No speckle noise suppression is applied during the scanning process to preserve the original features of the ultrasound images; S13: Data preprocessing and standardization; The computer performs preprocessing operations such as resampling, intensity normalization, and ROI cropping on the collected multimodal data; Data is integrated according to patient dimensions to form a multimodal image dataset; Based on the μ-RegPro public dataset standard, fully labeled paired data are selected to finally form a standardized dataset, which is divided into training set, test set and validation set according to clinical standard proportions; S14: Tissue segmentation; The pre-trained nnUNet segmentation model is used to automatically process the two-dimensional ultrasound sequence, identify and extract the pixel-level mask of the target tissue, remove background redundant noise interference, and obtain a clean region of interest. S15: Coordinate transformation to generate point cloud; Based on the calibration parameters and timestamp information of the ultrasonic probe, the coordinates of the two-dimensional mask pixels are transformed. Convert to 3D world coordinates Generate a spatial point cloud sequence; S16: Point cloud preprocessing; Denoising and downsampling are performed on the generated spatial point cloud to remove outliers and form an initial point cloud dataset containing spatial structural features.

[0008] As a preferred embodiment, the process of constructing the three-dimensional ultrasound image reconstruction model in step 2 is as follows: S21: Construct a three-level point cloud registration framework; S21-1: Construct the image segmentation submodule; The pre-trained nnUNet segmentation model is used to automatically process two-dimensional ultrasound sequences, identify and extract pixel-level masks of target tissues, remove background redundant noise interference, and obtain a pure region of interest. Then, the 26-neighborhood voxel detection algorithm is used to extract the three-dimensional voxel coordinates corresponding to the mask boundary, remove redundant interference such as background tissue and acoustic noise, and retain only the pure boundary voxel set of the target anatomical structure, providing high-quality geometric input for subsequent point cloud registration. S21-2: Construct a rigid alignment submodule; Using the ANTS toolkit or spatial transformation matrix solving algorithm, the rigid transformation parameters between point clouds of different scanning trajectories are calculated to achieve preliminary alignment in the global coordinate system, resulting in a rigid alignment submodule for eliminating gross displacement deviations. S21-3: Construct a multi-scale self-attention fine-tuning submodule; By extracting point cloud features under different receptive fields through a multi-branch parallel structure, and using self-attention operation to capture the long-range spatial dependencies between features, the precise correction of nonlinear tissue deformation is achieved. At the same time, a comprehensive anatomical region matching loss is introduced to constrain the registration process from the dual dimensions of intensity distribution and anatomical structure, thereby reducing the structural registration error between corresponding point clouds in ultrasound image sequences and forming a multi-scale self-attention fine registration submodule. S22: Construct a spatial point cloud generation module; Based on the boundary voxels of the segmentation mask, establish an affine mapping relationship from voxel coordinates to three-dimensional physical coordinates, and transform the boundary voxels of the two-dimensional ultrasonic segmentation mask into a three-dimensional spatial point cloud to form a spatial point cloud generation module. S23: Construct a neural implicit surface reconstruction module; construct an implicit function expression network based on a multilayer perceptron, take spatial coordinates as input, and predict the corresponding symbolic distance function value; at the same time, introduce a discriminator network to construct adversarial surface constraints, and enhance the smoothness and detail fidelity of the surface by judging the difference in normal vector distribution between the predicted surface and the real anatomical surface; through self-supervised learning combined with symbolic consistency constraints, correct the SDF prediction bias and enhance the clarity of surface boundaries, transforming the discrete 3D point cloud into a clear, smooth, low-noise, and topologically accurate 3D model, forming the neural implicit surface module; S24: Construct a joint loss function; construct a joint loss function by combining the weighted form of the sum of the comprehensive anatomical region matching loss and the surface reconstruction loss; S25: Model Training and Validation; Integrate the three-level point cloud registration framework, spatial point cloud generation module and neural implicit surface reconstruction module to construct an end-to-end deep learning model, use the training set for iterative training, minimize the joint loss through backpropagation, use the validation set to optimize parameters, and use the test set to validate the performance to obtain the three-dimensional ultrasound image reconstruction model.

[0009] As a preferred option, in S21-2, the feature extraction process using the multi-scale self-attention fine registration submodule is as follows: S21-21: Input the point cloud features into three parallel convolutional branches, and extract multi-scale local spatial features using 3×3, 5×5, and 7×7 convolutional kernels respectively; S21-22: Concatenate and fuse multi-scale features, and generate query vectors using linear mapping. Key vector Sum value vector As shown in (1), (2), and (3) respectively, the self-attention response map is calculated according to formula (4). This enables the enhancement of key anatomical structures in point cloud features; (1); (2); (3); (4).

[0010] As a preferred option, in S23, the feature extraction process using the neural implicit surface reconstruction module is as follows: S23-1: Generate dense 3D query points; represent the registered ROI point cloud as... ,in, The total number of point clouds; for each point According to Gaussian distribution 25 3D query points in a three-dimensional coordinate system were randomly sampled, among which... Defined as The Euclidean distance to its 50th nearest neighbor, and all query points form a set. ; S23-2: SDF Definition and Network Prediction; SDF values ​​are defined as positive for points outside the surface, zero for points on the surface, and negative for points inside the surface; This is demonstrated through an MLP network. Predict the SDF value of the query point The target surface is defined by the zero level set of the SDF. ; S23-3: Projection and Self-Supervised Learning; MLP Networks During Training Simultaneously predict query points SDF value With gradient The query point is then projected along the gradient direction to its nearest neighbor in the ROI point cloud. The projected query point is obtained according to formula (5). ; (5); S23-4: Geometric constraint loss; Three loss functions are used for optimization: self-supervised projection loss, symbolic consistency loss, and adversarial surface constraint loss. This corrects the SDF prediction bias from the dual dimensions of symbolic stability and surface authenticity, thus addressing the inherent defects of self-supervised learning. The generator and discriminator adopt an alternating optimization strategy until the network converges. S23-5: Surface Extraction; The Marching Cubes algorithm is used to extract a 3D surface mesh model from the learned SDF.

[0011] As a preferred option, the process of constructing the joint loss function in S24 is as follows: S24-1: Comprehensive anatomical region matching loss; calculate ROI mutual information loss according to formulas (6), (7) and (8) respectively. ROI-weighted multi-class Dice loss and deformation field regularization loss Then, according to formula (9), the weighted fusion is used to obtain the comprehensive anatomical region matching loss. ; (6); (7); (8); = + +λ (9); In the formula, Pref,r represents the ROI region of the point cloud to be registered after fine registration; Pref,r represents the ROI region of the reference point cloud. Mutual information is defined as follows: As an indicator to measure the degree of information sharing between two random variables, , They are respectively , marginal entropy, Let the joint entropy of the two be denoted as . Number of structure categories; and The source point cloud and the target point cloud are respectively the first... Point sets of class structures; Category weights; For the generated deformation field; For deformation field in voxel The gradient at the point; Ω represents the entire three-dimensional voxel space; These are the weighting coefficients for the regularization loss; S24-2: Surface reconstruction loss; self-supervised loss is calculated using formulas (10), (11), (12) and (13), respectively. Symbol consistency loss Generator adversarial loss Adversarial loss against discriminator ; (10); (11); (12); (13); In the formula, For projection query points; For neighboring points; This refers to the batch size during the training process; The gradient of the signed distance function; Cosine distance; ; ; ; The SDF value predicted by the generator; The ideal surface SDF value; S24-3: Joint loss function; the joint loss function is obtained by fusing the comprehensive anatomical region matching loss and surface reconstruction loss according to formula (14). ; (14).

[0012] As a preferred option, in step 3, the process of achieving accurate cross-modal alignment using a three-level point cloud registration framework is as follows: S31-1: Input a two-dimensional ultrasound image sequence, generate a segmentation mask frame by frame through the nnUNet segmentation network, and extract the boundary voxels of the target anatomical structure; S31-2: Solve the translation, rotation, and scaling transformation matrices through the rigid alignment module to unify the point cloud to be registered to the global coordinates of the reference point cloud and eliminate global spatial misalignment; S31-3: The dual-stream encoder of the fine registration network extracts dual-modal features, enhances cross-modal feature interaction through a multi-scale self-attention module, and optimizes local detail alignment by combining comprehensive anatomical region matching loss, outputting a precisely registered 3D point cloud.

[0013] As a preferred option, in step 3, the process of outputting a high-precision 3D point cloud through the spatial point cloud generation module is as follows: S32-1: Extract the boundary voxel index of the segmentation mask ; S32-2: Through affine transformation matrix Mapping voxel coordinates to three-dimensional physical coordinates As shown in formula (15); (15); In the formula, , , These represent the voxel spacing of the ultrasound image along each axis, used to compensate for resolution differences in imaging devices; direction cosine matrix. The rotational orientation of the image coordinate axes relative to the physical spatial anatomical axes is defined; This represents the offset of the image origin in the physical coordinate system; S32-3: Apply the fine registration transformation operator The final three-dimensional surface point cloud is obtained. .

[0014] As a preferred embodiment, in step 3, the process of generating a smooth, continuous, and anatomically complete three-dimensional ultrasound model via the neural implicit surface reconstruction module is as follows: S33-1: Construct an implicit function network based on a multilayer perceptron, taking spatial coordinates as input and outputting the symbolic distance function value; S33-2: Sample 3D query points around the registered point cloud according to a Gaussian distribution, covering the internal and external spatial regions of the target tissue; S33-3: Learning the signed distance function and gradient of the query point in a self-supervised manner using a multilayer perceptron; S33-4: Introduce a discriminator network to construct an adversarial surface constraint (OSC-ADL), which enhances surface smoothness and detail fidelity by discriminating the difference in normal vector distribution between the predicted surface and the real anatomical surface; S33-5: Introducing symbol consistency constraints to stabilize symbol prediction, and then optimizing it together with adversarial surface constraints to enhance the clarity of surface boundaries; S33-6: The Marching Cubes algorithm is used to extract the zero level set of the signed distance function to generate a smooth and continuous three-dimensional surface mesh model.

[0015] This invention addresses the technical challenges of sensorless free-scanning ultrasound 3D reconstruction, such as low registration accuracy, rough surfaces, and topological misalignment caused by strong image noise, weak texture, and lack of motion tracking information. It proposes a tracking-free free-scanning 3D ultrasound image reconstruction method based on point cloud registration and neural implicit representation. Through the synergistic optimization of cross-modal morphological prior guidance, a three-level point cloud registration framework, and neural implicit surface reconstruction, the method significantly improves reconstruction accuracy, surface smoothness, and anatomical integrity. It also possesses outstanding advantages such as no hardware dependency, lightweight deployment, and strong generalization ability. Compared with existing technologies, this invention has the following advantages: 1. Significantly improved registration accuracy; a cross-modal three-level point cloud registration framework is adopted: First, the ultrasound region of interest is extracted through the nnUNet segmentation network and noise interference is removed by 26-neighbor voxel detection to obtain pure boundary voxels; Second, the global coordinate system is unified by the ANTS rigid alignment module to eliminate spatial offset and angular deviation caused by probe free movement; Finally, a multi-scale self-attention (MSAM) fine registration network is introduced, which extracts cross-modal features through a dual-stream encoder, captures long-range spatial dependence using the self-attention mechanism, and combines comprehensive anatomical region matching loss (ROI mutual information, weighted multi-class Dice, deformation field regularization) to constrain registration from three dimensions: intensity distribution, structural morphology, and deformation smoothness. It can adapt to complex nonlinear scanning trajectories such as L-shaped, C-shaped, and S-shaped, effectively solving the point cloud misalignment problem under soft tissue deformation and complex nonlinear scanning trajectories, significantly reducing inter-frame registration error and boundary alignment error, and significantly improving registration robustness.

[0016] 2. Comprehensive optimization of surface reconstruction quality; a surface reconstruction module is constructed based on neural implicit representation: the MLP network is trained through self-supervised learning to predict the symbolic distance function (SDF) and its gradient, and innovatively, symbolic consistency constraint (SCC) and adversarial surface constraint (OSC-ADL) are introduced. SCC avoids surface distortion caused by symbol confusion by penalizing the deviation between the gradient direction and the projection direction; OSC-ADL adopts a generator-discriminator adversarial mechanism to force the network to output zero SDF value at the anatomical surface, enhancing the accuracy of surface boundary fitting. The dual geometric constraints work together to effectively overcome the problems of rough reconstructed surface, missing details, hole breaks, and topological distortion caused by ultrasound speckle noise and image edge blurring, generating a smooth, continuous, anatomically complete, and topologically accurate 3D surface model with excellent clinical visualization effects.

[0017] 3. Sensorless and lightweight deployment: The entire reconstruction process requires no external motion sensors, optical positioning equipment, or other dedicated hardware, relying entirely on the image's own features, significantly reducing clinical equipment costs and operational complexity. The model employs a collaborative optimization design, resulting in low computational overhead and compatibility with portable ultrasound and other end-device devices. It improves reconstruction accuracy while meeting real-time clinical needs, making it suitable for practical clinical scenarios such as preoperative planning and intraoperative navigation.

[0018] 4. Strong generalization and adaptability; it integrates the morphological priors of high signal-to-noise ratio and clear anatomical boundaries from magnetic resonance imaging, enhancing the model's adaptability to different ultrasound equipment, different anatomical locations (prostate, blood vessels, abdominal organs), and ultrasound images of varying quality. The three-level point cloud registration framework can adapt to various complex nonlinear scanning image data, maintaining stable reconstruction performance in complex clinical scenarios.

[0019] 5. Coordinated optimization of registration and reconstruction, enabling efficient end-to-end inference: A joint loss function was designed, integrating ROI mutual information loss, weighted multi-class Dice loss, deformation field regularization loss (for fine registration), self-supervised projection loss, sign consistency loss, and adversarial loss (for surface reconstruction). Simultaneously, local registration errors and global anatomical structure consistency are constrained, achieving coordinated optimization of registration and reconstruction. The model is lightweight and has a fast inference speed, capable of generating high-quality 3D ultrasound images in real-time on portable ultrasound devices without any additional hardware support. This assists doctors in intuitively locating lesions and accurately assessing the extent of lesions, providing more accurate and objective 3D diagnostic information for clinical scenarios such as prostate targeted biopsy, vascular assessment, and abdominal organ examination, effectively improving the efficiency and standardization of ultrasound diagnosis and treatment.

[0020] This method significantly improves the accuracy, smoothness, and robustness of sensorless free-scan ultrasound 3D reconstruction, completely eliminating the dependence on external motion sensors. It provides a high-fidelity, easily deployable 3D image analysis tool for precise clinical diagnosis and treatment, and has extremely high engineering application value and clinical application prospects. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention; Figure 2 This is a structural diagram of the nnUNet segmentation network in this invention; Figure 3 This is a schematic diagram of the three-level point cloud registration framework in this invention; Figure 4 This is a schematic diagram of the multi-scale self-attention module in this invention; Figure 5 This is a structural diagram of the spatial point cloud generation module in this invention; Figure 6 This is a schematic diagram of the resistant surface constraint in this invention. Detailed Implementation

[0022] like Figures 1 to 6 As shown, this invention provides a method for three-dimensional ultrasound surface reconstruction based on point cloud registration. The following description, using the reconstruction of three-dimensional ultrasound of prostate tissue as an example and in conjunction with the accompanying drawings, further illustrates the invention, specifically including the following steps: Step 1: Acquire multimodal image dataset; Using a multimodal image acquisition system (computer, ultrasound scanning equipment, magnetic resonance scanning equipment), acquire two-dimensional ultrasound image sequences and high signal-to-noise ratio magnetic resonance images through sensorless free scanning. After preprocessing, a standardized multimodal image dataset is formed, and then an initial point cloud dataset is generated through segmentation, coordinate transformation, denoising and other processing. As a preferred option, the process of obtaining a multimodal image dataset is as follows: S11: Establish a multimodal image acquisition system; A multimodal image acquisition system is established using a computer, ultrasound scanning equipment, and magnetic resonance scanning equipment. The computer is connected to both the ultrasound scanning equipment and the magnetic resonance scanning equipment for data acquisition and synchronous storage. Preferably, a 3.0T superconducting magnetic resonance scanner and a Philips iU22 ultrasound system are used, which can simultaneously acquire high signal-to-noise ratio MR images and clinical standard ultrasound images, ensuring the accuracy of multimodal data pairing. S12: Multimodal data acquisition; employing a sensorless free-scanning method, the ultrasound probe scans the patient's prostate, blood vessels, and other soft tissue areas. The ultrasound scanning device acquires two-dimensional ultrasound image sequences in real time and sends them to a computer; simultaneously, high signal-to-noise ratio magnetic resonance (MR) images are acquired through a magnetic resonance scanning device as morphological prior data. Preferably, the MR images are Z-score normalized to eliminate equipment difference interference; no speckle noise suppression is applied during the scanning process to preserve the original features of the ultrasound images; S13: Data Preprocessing and Standardization; The computer performs preprocessing operations on the collected multimodal data, including resampling (unifying voxel spacing), intensity normalization (mapping grayscale to the [0,1] interval), and ROI (region of interest) cropping; Data is integrated according to patient dimensions to form a multimodal image dataset; Based on the μ-RegPro public dataset standard, fully labeled (MR-TRUS) paired data are selected to finally form a μ-RegPro standardized dataset. The data is stored in NIfTI format to facilitate batch reading and training of the model, and is divided into training set, test set and validation set according to clinical standard proportions; S14: Tissue segmentation; A pre-trained nnUNet segmentation model is used to automatically process two-dimensional ultrasound sequences, identifying and extracting pixel-level masks of target tissues, and removing background redundant noise interference, such as... Figure 2 As shown, a clean region of interest (ROI) is obtained. S15: Coordinate transformation to generate point cloud; Based on the calibration parameters and timestamp information of the ultrasonic probe, the coordinates of the two-dimensional mask pixels are transformed. Convert to 3D world coordinates Generate a spatial point cloud sequence; S16: Point cloud preprocessing; Denoising and downsampling are performed on the generated spatial point cloud to remove outliers and form an initial point cloud dataset containing spatial structural features.

[0023] In this technical solution, a simultaneous ultrasound and magnetic resonance imaging (MRI) acquisition system is built. Two-dimensional ultrasound sequences and high signal-to-noise ratio (SNR) MR images are acquired using a sensorless free-scanning method, effectively preserving the original ultrasound features. After preprocessing such as resampling, normalization, and ROI cropping, a standardized dataset is formed. Paired data are then selected based on the μ-RegPro standard to ensure the consistency and comparability of multimodal data. Furthermore, the nnUNet segmentation network is used to accurately extract target masks and remove background noise. Combined with coordinate transformation and point cloud preprocessing, a high-quality initial point cloud is generated. This process effectively integrates cross-modal morphological priors, providing a clean, accurate, and standardized data foundation for subsequent three-level point cloud registration and 3D surface reconstruction, significantly improving the stability of model training and reconstruction accuracy.

[0024] Step 2: Construct a 3D ultrasound image reconstruction model; adopt a three-level point cloud registration framework (e.g., Figure 3 As shown, it includes an image segmentation module, a rigid alignment sub-module, a multi-scale self-attention fine registration sub-module, a spatial point cloud generation module, a neural implicit surface reconstruction module, and a joint loss function to construct a deep learning model, thereby achieving high-precision registration of multimodal point clouds and 3D surface reconstruction. As a preferred option, the process of constructing a three-dimensional ultrasound image reconstruction model is as follows: S21: Construct a three-level point cloud registration framework; S21-1: Ultrasonic image segmentation and boundary voxel extraction; The input consists of a sequence of two-dimensional ultrasound images acquired through sensorless free scanning. Each frame of the sequence is then fed into the trained nnUNet segmentation network. This network employs a symmetric encoder-decoder architecture and, addressing the characteristics of high noise and blurred tissue structure edges in ultrasound images, automatically and accurately segments target anatomical regions such as the prostate, generating a binary segmentation mask. A 26-neighborhood voxel detection algorithm is then used to extract the three-dimensional voxel coordinates corresponding to the mask boundaries, removing redundant interference such as background tissue and acoustic noise, retaining only the pure boundary voxel set of the target anatomical structure, providing high-quality geometric input for subsequent point cloud registration. S21-2: Construct a rigid alignment submodule; Using the ANTS toolkit or spatial transformation matrix solving algorithm, the rigid transformation parameters (including rotation, translation and scaling) between point clouds of different scanning trajectories are calculated to achieve preliminary alignment in the global coordinate system, resulting in a rigid alignment submodule for eliminating gross displacement deviations. Through this rigid alignment submodule, the rigid transformation matrix can be solved to unify the global coordinate system of multi-trajectory point clouds and complete cross-modal space initialization. S21-3: Construct a multi-scale self-attention (MSAM) fine-tuning submodule; A fine registration network is constructed using depthwise separable convolutions at different scales combined with a self-attention mechanism. Specifically, in the MSAM module, point cloud features under different receptive fields are extracted through a multi-branch parallel structure, and self-attention operations are used to capture long-range spatial dependencies between features to achieve accurate correction of tissue nonlinear deformation. Simultaneously, a comprehensive anatomical region matching loss is introduced to constrain the registration process from both intensity distribution and anatomical structure dimensions, reducing structural registration errors between corresponding point clouds in ultrasound image sequences, forming a multi-scale self-attention fine registration submodule. This multi-scale self-attention module can capture long-distance feature dependencies across modalities, and the comprehensive anatomical region matching loss optimizes registration from both intensity and structure dimensions, allowing the framework to perfectly adapt to scenarios with large soft tissue deformations, significantly reducing inter-frame registration errors. Figure 4 As shown; As a preferred option, the feature extraction process using the multi-scale self-attention fine registration submodule is as follows: S21-21: Input the point cloud features into three parallel convolutional branches, and extract multi-scale local spatial features using 3×3, 5×5, and 7×7 convolutional kernels respectively; S21-22: Concatenate and fuse multi-scale features, and generate query vectors using linear mapping. Key vector Sum value vector As shown in (1), (2), and (3) respectively, the self-attention response map is calculated according to formula (4). This enables the enhancement of key anatomical structures in point cloud features; (1); (2); (3); (4).

[0025] S22: Construct a spatial point cloud generation module; based on the boundary voxels of the segmentation mask, establish an affine mapping relationship from voxel coordinates to three-dimensional physical coordinates, transforming the boundary voxels of the two-dimensional ultrasonic segmentation mask into a three-dimensional spatial point cloud, thus forming the spatial point cloud generation module; such as Figure 5 As shown, this module fully preserves the surface topological features of the anatomical structure through precise conversion between voxel index and physical coordinates, providing misaligned and highly pure 3D point cloud data for subsequent surface reconstruction, thus solving the problem of spatial distortion in traditional point cloud generation. S23: Constructing a neural implicit surface reconstruction module; an implicit function representation network is built based on a multilayer perceptron (MLP) to predict the corresponding signed distance function (SDF) value using spatial coordinates as input; simultaneously, a discriminator network is introduced to construct adversarial surface constraints (OSC-ADL), which enhances surface smoothness and detail fidelity by discriminating the difference in normal vector distribution between the predicted surface and the real anatomical surface; through self-supervised learning combined with signed consistency constraints (SCC), SDF prediction bias is corrected and surface boundary clarity is enhanced, transforming discrete 3D point clouds into a clear, smooth, low-noise, and topologically accurate 3D model, forming the neural implicit surface module, such as... Figure 6 As shown; In this technical solution, a three-level point cloud registration framework (nnUNet accurately segments tissue contours in 2D ultrasound images, rigid alignment eliminates global misalignment, and multi-scale self-attention fine registration corrects nonlinear deformation of soft tissue) achieves high-precision progressive alignment of cross-modal point clouds, effectively solving the point cloud misalignment problem under free scanning. The spatial point cloud generation module uses affine mapping to accurately convert the segmentation mask into 3D physical coordinates, ensuring the misalignment-free and pure topology of the point cloud. The neural implicit surface reconstruction module learns SDF based on MLP and introduces symbolic consistency constraints and adversarial surface constraints, significantly improving surface smoothness, detail fidelity, and topological accuracy. Overall model co-optimization greatly improves the registration accuracy, surface quality, and anatomical structure integrity of 3D reconstruction, and requires no external sensors, possessing advantages such as lightweight and strong generalization.

[0026] As a preferred approach, the feature extraction process using the neural implicit surface reconstruction module is as follows: S23-1: Generate dense 3D query points; generate dense 3D query points around the registered ROI point cloud to ensure the network can capture distance distribution information inside and outside the surface; represent the registered ROI point cloud as... ,in, The total number of point clouds; for each point According to Gaussian distribution 25 3D query points in a three-dimensional coordinate system were randomly sampled, among which... Defined as The Euclidean distance to its 50th nearest neighbor, and all query points form a set. This setting ensures that the query points can cover the key areas around the point cloud without introducing irrelevant information due to an excessively large sampling range. The sampling range of the query points is locally concentrated around the ROI point cloud, which not only ensures the sample density of SDF learning but also reduces computational complexity, providing an efficient and effective input sample set for subsequent self-supervised training. S23-2: SDF Definition and Network Prediction; The Signed Distance Function (SDF) is an implicit continuous representation that defines a 3D shape by describing the signed distance from any point in space to the target surface: points outside the surface have a positive SDF value, points on the surface have a zero SDF value, and points inside the surface have a negative SDF value; This is achieved through an MLP network. Predict the SDF value of the query point No additional real SDF labels or surface normal vector information are required; the target surface is defined by the zero level set of the SDF. This provides a foundation for subsequent surface extraction; S23-3: Projection and Self-Supervised Learning; MLP Networks During Training Simultaneously predict query points SDF value With gradient The query point is then projected along the gradient direction to its nearest neighbor in the ROI point cloud. The projected query point is obtained according to formula (5). The gradient is calculated through backpropagation of the network. Through this projection process, the network can achieve self-supervised learning by leveraging the spatial relationships within the point cloud, without relying on external labels. (5); S23-4: Geometric Constraint Loss; Three loss functions are used for optimization: self-supervised projection loss, symbolic consistency loss (SCC), and adversarial surface constraint (OSC-ADL). This corrects the SDF prediction bias from the dual dimensions of symbolic stability and surface authenticity, addressing the inherent defects of self-supervised learning and optimizing the accuracy and stability of SDF prediction. The generator and discriminator adopt an alternating optimization strategy until the network converges. By minimizing the self-supervised projection loss, the network can be forced to learn accurate SDF values ​​and gradient directions. S23-5: Surface Extraction; The Marching Cubes algorithm is used to extract a 3D surface mesh model from the learned SDF. This algorithm traverses the 3D voxel mesh, determines the SDF symbol of the voxel vertices, and then interpolates to calculate the intersection points of the SDF zero level set and the voxel edges. Finally, the intersection points are connected to form a refined 3D volume data model, which can be directly exported as standard format files such as OBJ and PLY.

[0027] In this technical solution, a dense 3D query point (based on Gaussian distribution and nearest neighbor distance) is adaptively sampled around the registered ROI point cloud, providing a reasonably distributed and uniformly dense efficient sample set for learning the symbolic distance function. An MLP network is used for self-supervised learning of the SDF value and its gradient, eliminating the need for external labels. A projection mechanism forces the network to learn using the spatial relationships of the point cloud, effectively overcoming the difficulty of lacking ground truth labels in ultrasonic data. A triple loss mechanism—self-supervised projection loss, symbolic consistency loss, and adversarial surface constraint—is introduced to correct SDF prediction bias from both symbolic stability and surface realism dimensions, significantly improving the smoothness, topological accuracy, and detail fidelity of surface reconstruction. Finally, the Marching Cubes algorithm is used to extract a high-quality 3D mesh model. This module achieves high-precision implicit surface reconstruction without external sensors, solving the problems of surface roughness, holes, and distortion inherent in traditional methods.

[0028] S24: Construct a joint loss function; construct a joint loss function by combining the weighted sum of the Comprehensive Anatomical Region Matching Loss (CARML) and the surface reconstruction loss (including self-supervised projection loss, sign consistency loss and adversarial loss); where CARML integrates intensity similarity, structural correlation and deformation field smoothing terms; As a preferred option, the process of constructing the joint loss function is as follows: S24-1: Comprehensive anatomical region matching loss; calculate ROI mutual information loss according to formulas (6), (7) and (8) respectively. ROI-weighted multi-class Dice loss and deformation field regularization loss Then, according to formula (9), the weighted fusion is used to obtain the comprehensive anatomical region matching loss. ; (6); (7); (8); = + +λ (9); In the formula, Pref,r represents the ROI region of the point cloud to be registered after fine registration; Pref,r represents the ROI region of the reference point cloud. Mutual information is defined as follows: As an indicator to measure the degree of information sharing between two random variables, , They are respectively , marginal entropy, The joint entropy of the two is the loss, which can effectively enhance the correlation between the ultrasonic boundary point cloud to be registered and the MR reference point cloud in terms of local spatial topology and feature distribution. This refers to the number of structural categories (such as prostate gland, lesions, vascular branches, etc.). and The source point cloud and the target point cloud are respectively the first... Point sets of class structures; As a category weight, the weight of basic structures such as the prostate gland is set to 0.1 and the weight of key structures such as lesions is set to 0.3 according to the importance of the structure, so as to ensure the accurate alignment of key anatomical structures. For the generated deformation field; For deformation field in voxel The gradient at the point; Ω represents the entire three-dimensional voxel space; The weight coefficients for regularization loss are determined through cross-validation. These weights effectively balance the requirements of deformation field smoothness and registration accuracy, ensuring that the network avoids unreasonable deformation while optimizing intensity distribution and structural morphology similarity. S24-2: Surface reconstruction loss; self-supervised loss is calculated using formulas (10), (11), (12) and (13), respectively. Symbol consistency loss Generator adversarial loss Adversarial loss against discriminator ; (10); (11); (12); (13); In the formula, For projection query points; For neighboring points; This refers to the batch size during the training process; The gradient of the signed distance function; The cosine distance is used, and the penalty range is [0,2]. The loss is minimized when the gradient direction is consistent with the projection direction. ; ; ; The SDF value predicted by the generator; The ideal surface SDF value; S24-3: Joint loss function; the joint loss function is obtained by fusing the comprehensive anatomical region matching loss and surface reconstruction loss according to formula (14). ; (14).

[0029] This technical solution achieves synergistic optimization of registration and reconstruction by integrating comprehensive anatomical region matching loss and surface reconstruction loss. The anatomical region matching loss utilizes ROI mutual information, weighted multi-class Dice, and deformation field regularization to constrain cross-modal point cloud registration from three dimensions: intensity distribution, structural morphology, and deformation smoothness, effectively enhancing the alignment of local spatial topology with key anatomical structures. The surface reconstruction loss introduces self-supervised projection, sign consistency, and adversarial constraints, forcing the network to learn accurate SDF values ​​and gradient directions, while stabilizing sign prediction and enhancing surface realism. The joint loss balances the local accuracy of registration with the global anatomical consistency of reconstruction, significantly improving the inter-frame alignment quality, surface smoothness, and topological integrity of the 3D model, providing a reliable optimization target for high-fidelity 3D ultrasound reconstruction under sensorless free scanning.

[0030] S25: Model Training and Validation; Integrate the three-level point cloud registration framework, spatial point cloud generation module and neural implicit surface reconstruction module to construct an end-to-end deep learning model, use the training set for iterative training, minimize the joint loss through backpropagation, use the validation set to optimize parameters, and use the test set to validate the performance to obtain the three-dimensional ultrasound image reconstruction model.

[0031] Step 3: 3D ultrasound image surface reconstruction; using an image acquisition system, the patient's treatment area is scanned using a sensorless free-scanning trajectory to obtain 2D ultrasound image sequence data; the 2D ultrasound image sequence data is input into the 3D ultrasound image reconstruction model. First, cross-modal precise alignment is completed through a three-level point cloud registration framework to eliminate point cloud misalignment caused by probe motion disturbance and soft tissue deformation; then, a high-precision 3D point cloud is output through the spatial point cloud generation module; finally, a smooth, continuous, and anatomically complete 3D ultrasound model is generated through the neural implicit surface reconstruction module, providing accurate 3D anatomical information for clinical diagnosis.

[0032] As a preferred option, the process of achieving accurate cross-modal alignment using a three-level point cloud registration framework is as follows: S31-1: Ultrasound Image Segmentation and Boundary Voxel Extraction; A sequence of two-dimensional ultrasound images acquired through sensorless free scanning is input, and the sequence is fed frame-by-frame into a trained nnUNet segmentation network. This network employs a symmetric encoder-decoder architecture, automatically and accurately segmenting target anatomical regions such as the prostate, taking advantage of the high noise and blurred edges of tissue structures in ultrasound images, generating a binary segmentation mask. Then, a 26-neighborhood voxel detection algorithm is used to extract the three-dimensional voxel coordinates corresponding to the mask boundaries, removing redundant interference such as background tissue and acoustic noise, retaining only the pure boundary voxel set of the target anatomical structure, providing high-quality geometric input for subsequent point cloud registration. S31-2: Rigid alignment eliminates global spatial misalignment. The extracted clean boundary voxels are converted into an initial 3D point cloud, and a rigid alignment module is built based on the ANTS advanced registration toolkit. Using the reference point cloud as a benchmark, the optimal rigid transformation matrix, including translation, rotation, and scaling, is solved by minimizing the mean square error loss between point clouds. The point clouds to be registered under different scan frames and nonlinear trajectories are uniformly mapped to the global 3D coordinate system of the reference point cloud, completely eliminating rigid errors such as global spatial offset and angular deflection caused by the probe's free movement. This provides stable spatial initialization for subsequent fine registration and avoids getting trapped in local optima. S31-3: Nonlinear Fine Registration and Local Deformation Correction; The rigidly aligned point cloud is input into a fine registration network. The network employs a parameter-free dual-stream encoder structure to independently extract multi-scale features from both the ultrasound point cloud and the MRI reference point cloud. Subsequently, a multi-scale self-attention module is connected, which captures the long-distance spatial dependence of cross-modal features through linear transformations of the query, key, and value matrices, enhancing feature interactions in key anatomical regions such as glands and lesions. Simultaneously, using comprehensive anatomical region matching loss as the optimization objective, it combines ROI mutual information loss, ROI weighted multi-class Dice loss, and deformation field regularization loss to constrain registration from three dimensions: intensity distribution, structural morphology, and deformation smoothness. This precisely corrects subtle local misalignments caused by soft tissue deformation, ultimately outputting a spatially perfectly aligned 3D point cloud with accurate anatomical structure matching.

[0033] In this technical solution, pure boundary voxels are extracted through nnUNet segmentation and 26-neighborhood detection to provide high-quality input for registration; rigid alignment uses the ANTS toolkit to eliminate global spatial misalignment and avoid local optima; fine registration adopts a dual-stream encoder and a multi-scale self-attention module, combined with comprehensive anatomical region matching loss (mutual information, Dice, regularization), to accurately correct soft tissue deformation, achieving high-precision and robust alignment of cross-modal point clouds, significantly reducing inter-frame registration errors, and adapting to complex free scanning scenarios.

[0034] As a preferred method, the process of outputting a high-precision 3D point cloud through the spatial point cloud generation module is as follows: S32-1: Extract the boundary voxel index of the segmentation mask ; S32-2: Through affine transformation matrix Mapping voxel coordinates to three-dimensional physical coordinates As shown in formula (15); (15); In the formula, , , These represent the voxel spacing of the ultrasound image along each axis, used to compensate for resolution differences in imaging devices; direction cosine matrix. The rotational orientation of the image coordinate axes relative to the physical spatial anatomical axes is defined; This represents the offset of the image origin in the physical coordinate system; through this mapping mechanism, discrete voxel indices are restored to length measures with actual physical meaning, ensuring the accuracy of the 3D reconstruction results in a clinical anatomical sense. S32-3: Apply the fine registration transformation operator The final three-dimensional surface point cloud is obtained. .

[0035] In this technical solution, the resolution differences and spatial orientation deviations of imaging devices are compensated by the precise mapping of voxel indexes and affine transformation matrices, and the two-dimensional segmentation mask is efficiently converted into a three-dimensional coordinate point cloud with actual physical meaning. Combined with the fine registration transformation operator, residual spatial misalignment is further eliminated, and a clean, misalignment-free, and high-precision three-dimensional point cloud is finally output. This provides a data foundation with complete anatomical structure and accurate spatial position for subsequent neural implicit surface reconstruction, effectively solving the problems of spatial distortion and topological misalignment in traditional point cloud generation.

[0036] As a preferred method, the process of generating a smooth, continuous, and anatomically complete three-dimensional ultrasound model via the neural implicit surface reconstruction module is as follows: S33-1: Constructing an implicit function network; Constructing an implicit function network based on a multilayer perceptron (MLP), taking spatial coordinates as input and outputting the symbolic distance function (SDF) value; S33-2: Generate 3D query points; For the 3D point cloud of the ROI after spatial alignment following three-level registration, use each target point in the point cloud as the center and follow a Gaussian distribution. Randomly sample three-dimensional query points, where the standard deviation of the Gaussian distribution is... Defined as the Euclidean distance between the center point and its 50th nearest neighbor, with 25 query points sampled for each center point. The query points uniformly cover the internal, surface, and external three-dimensional spatial regions of the target anatomical tissue, providing a reasonably distributed and uniformly dense sample set for learning the signed distance function.

[0037] S33-3: Self-supervised learning of the symbolic distance function and gradient; the sampled 3D query points are input into a multilayer perceptron (MLP) network, which is trained using self-supervised learning to learn the symbolic distance function (SDF) and gradient information of any point in the point cloud. The symbolic distance function is defined as the signed distance from a point in space to the anatomical surface: the SDF value is positive for points outside the surface, zero for points on the surface, and negative for points inside the surface. During training, the network projects the query points along the gradient direction to the nearest neighbor points in the point cloud, achieving unlabeled self-supervised learning based on the spatial relationships of the point cloud.

[0038] S33-4: Introduction of Adversarial Surface Constraints (OSC-ADL); A discriminator network is introduced to construct adversarial surface constraints, enhancing surface smoothness and detail fidelity by discriminating the difference in normal vector distribution between the predicted surface and the real anatomical surface. Specifically, a generator-discriminator adversarial learning mechanism is adopted: the generator fits the SDF distribution, and the discriminator distinguishes between the predicted SDF and the ground truth SDF of the real surface, forcing the network to output SDF values ​​close to zero at the anatomical surface, thereby enhancing the fitting accuracy of the surface boundary.

[0039] S33-5: Introducing Sign Consistency Constraint (SCC); By penalizing the deviation between the gradient direction and the projection direction of the query point, the SDF sign prediction of the interior, surface, and exterior of the point cloud is forced to stabilize, avoiding surface distortion caused by sign chaos. Through joint optimization of sign consistency constraint and adversarial surface constraint, the clarity of surface boundaries is enhanced. S33-6: Extracting the 3D surface mesh model; employing the Marching Cubes algorithm to traverse the 3D voxel mesh, determining the SDF sign of each mesh vertex, and calculating the intersection points of the SDF zero level set and voxel edges through linear interpolation. Connecting these intersection points sequentially generates a smooth and continuous 3D surface mesh model. This allows for the generation of a topologically accurate, smooth, and complete 3D ultrasound model, which can be exported as standard format files such as .obj and .ply. High-quality 3D reconstruction can be achieved without external sensor assistance, providing precise anatomical information for clinical targeted biopsies and preoperative planning.

[0040] In this technical solution, uniformly distributed 3D query points are generated through adaptive Gaussian sampling, and the SDF and its gradient are learned in a self-supervised manner without the need for external labels. Adversarial surface constraints (OSC-ADL) and sign consistency constraints (SCC) are introduced to enhance the surface boundary fitting accuracy and sign prediction stability, respectively, effectively overcoming the surface roughness and lack of detail caused by ultrasonic noise. Finally, the Marching Cubes algorithm is used to extract a smooth, continuous, and topologically accurate 3D mesh model. The entire process generates a high-fidelity anatomical model without the need for external sensors, providing reliable 3D image support for clinical targeted biopsies and preoperative planning.

[0041] This invention addresses the technical challenges of sensorless free-scanning ultrasound 3D reconstruction, such as low registration accuracy, rough surfaces, and topological misalignment caused by strong image noise, weak texture, and lack of motion tracking information. It proposes a tracking-free free-scanning 3D ultrasound image reconstruction method based on point cloud registration and neural implicit representation. Through the synergistic optimization of cross-modal morphological prior guidance, a three-level point cloud registration framework, and neural implicit surface reconstruction, the method significantly improves reconstruction accuracy, surface smoothness, and anatomical integrity. It also possesses outstanding advantages such as no hardware dependency, lightweight deployment, and strong generalization ability.

[0042] This method significantly improves the accuracy, smoothness, and robustness of sensorless free-scan ultrasound 3D reconstruction, completely eliminating the dependence on external motion sensors. It provides a high-fidelity, easily deployable 3D image analysis tool for precise clinical diagnosis and treatment, and has extremely high engineering application value and clinical application prospects.

Claims

1. A method for ultrasonic three-dimensional surface reconstruction based on point cloud registration, characterized in that, Includes the following steps: Step 1: Acquire multimodal image dataset; Use a multimodal image acquisition system to acquire two-dimensional ultrasound image sequences and high signal-to-noise ratio magnetic resonance images through sensorless free scanning. After preprocessing, a standardized multimodal image dataset is formed, and then an initial point cloud dataset is generated through segmentation, coordinate transformation, and denoising. Step 2: Construct a 3D ultrasound image reconstruction model; a deep learning model is constructed using a three-level point cloud registration framework, a spatial point cloud generation module, a neural implicit surface reconstruction module, and a joint loss function to achieve high-precision registration of multimodal point clouds and 3D surface reconstruction. Step 3: 3D ultrasound image surface reconstruction; using an image acquisition system, the patient's treatment area is scanned using a sensorless free-scanning trajectory to obtain 2D ultrasound image sequence data; the 2D ultrasound image sequence data is input into the 3D ultrasound image reconstruction model. First, cross-modal precise alignment is completed through a three-level point cloud registration framework to eliminate point cloud misalignment caused by probe motion disturbance and soft tissue deformation; then, a high-precision 3D point cloud is output through the spatial point cloud generation module; finally, a smooth, continuous, and anatomically complete 3D ultrasound model is generated through the neural implicit surface reconstruction module, providing accurate 3D anatomical information for clinical diagnosis.

2. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 1, characterized in that, In step 1, the process of obtaining the multimodal image dataset is as follows: S11: Build a multimodal image acquisition system; Build a multimodal image acquisition system using a computer, an ultrasound scanning device, and a magnetic resonance scanning device. The computer is connected to the ultrasound scanning device and the magnetic resonance scanning device respectively for data acquisition and synchronous storage. S12: Multimodal data acquisition; Employing a sensorless free-scanning method, the patient's soft tissue area is scanned using an ultrasound probe. The ultrasound scanning device acquires two-dimensional ultrasound image sequences in real time and sends them to a computer; Simultaneously, high signal-to-noise ratio magnetic resonance images are acquired through a magnetic resonance scanning device as morphological prior data; No speckle noise suppression is applied during the scanning process to preserve the original features of the ultrasound images; S13: Data preprocessing and standardization; The computer performs preprocessing operations such as resampling, intensity normalization, and ROI cropping on the collected multimodal data; Data is integrated according to patient dimensions to form a multimodal image dataset; Based on the μ-RegPro public dataset standard, fully labeled paired data are selected to finally form a standardized dataset, which is divided into training set, test set and validation set according to clinical standard proportions; S14: Tissue segmentation; The pre-trained nnUNet segmentation model is used to automatically process the two-dimensional ultrasound sequence, identify and extract the pixel-level mask of the target tissue, remove background redundant noise interference, and obtain a clean region of interest. S15: Coordinate transformation to generate point cloud; Based on the calibration parameters and timestamp information of the ultrasonic probe, the coordinates of the two-dimensional mask pixels are transformed. Convert to 3D world coordinates Generate a spatial point cloud sequence; S16: Point cloud preprocessing; Denoising and downsampling are performed on the generated spatial point cloud to remove outliers and form an initial point cloud dataset containing spatial structural features.

3. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 1 or 2, characterized in that, In step 2, the process of constructing the three-dimensional ultrasound image reconstruction model is as follows: S21: Construct a three-level point cloud registration framework; S21-1: Construct the image segmentation submodule; The pre-trained nnUNet segmentation model is used to automatically process two-dimensional ultrasound sequences, identify and extract pixel-level masks of target tissues, remove background redundant noise interference, and obtain a pure region of interest. Then, the 26-neighborhood voxel detection algorithm is used to extract the three-dimensional voxel coordinates corresponding to the mask boundary, remove redundant interference such as background tissue and acoustic noise, and retain only the pure boundary voxel set of the target anatomical structure, providing high-quality geometric input for subsequent point cloud registration. S21-2: Construct a rigid alignment submodule; Construct a rigid alignment submodule; use the ANTS toolkit or spatial transformation matrix solving algorithm to calculate the rigid transformation parameters between point clouds of different scanning trajectories, achieve preliminary alignment in the global coordinate system, and obtain a rigid alignment submodule for eliminating gross displacement deviations; S21-3: Construct a multi-scale self-attention fine-tuning submodule; By extracting point cloud features under different receptive fields through a multi-branch parallel structure, and using self-attention operation to capture the long-range spatial dependencies between features, the precise correction of nonlinear tissue deformation is achieved. At the same time, a comprehensive anatomical region matching loss is introduced to constrain the registration process from the dual dimensions of intensity distribution and anatomical structure, thereby reducing the structural registration error between corresponding point clouds in ultrasound image sequences and forming a multi-scale self-attention fine registration submodule. S22: Construct a spatial point cloud generation module; Based on the boundary voxels of the segmentation mask, establish an affine mapping relationship from voxel coordinates to three-dimensional physical coordinates, and transform the boundary voxels of the two-dimensional ultrasonic segmentation mask into a three-dimensional spatial point cloud to form a spatial point cloud generation module. S23: Construct a neural implicit surface reconstruction module; construct an implicit function expression network based on a multilayer perceptron, take spatial coordinates as input, and predict the corresponding symbolic distance function value; at the same time, introduce a discriminator network to construct adversarial surface constraints, and enhance the smoothness and detail fidelity of the surface by judging the difference in normal vector distribution between the predicted surface and the real anatomical surface; through self-supervised learning combined with symbolic consistency constraints, correct the SDF prediction bias and enhance the clarity of surface boundaries, transforming the discrete 3D point cloud into a clear, smooth, low-noise, and topologically accurate 3D model, forming the neural implicit surface module; S24: Construct a joint loss function; construct a joint loss function by combining the weighted form of the sum of the comprehensive anatomical region matching loss and the surface reconstruction loss; S25: Model Training and Validation; Integrate the three-level point cloud registration framework, spatial point cloud generation module and neural implicit surface reconstruction module to construct an end-to-end deep learning model, use the training set for iterative training, minimize the joint loss through backpropagation, use the validation set to optimize parameters, and use the test set to validate the performance to obtain the three-dimensional ultrasound image reconstruction model.

4. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 3, characterized in that, In S21-2, the feature extraction process using the multi-scale self-attention fine registration submodule is as follows: S21-21: Input the point cloud features into three parallel convolutional branches, and extract multi-scale local spatial features using 3×3, 5×5, and 7×7 convolutional kernels respectively; S21-22: Concatenate and fuse multi-scale features, and generate query vectors using linear mapping. Key vector Sum value vector As shown in (1), (2), and (3) respectively, the self-attention response map is calculated according to formula (4). This enables the enhancement of key anatomical structures in point cloud features; （1）； （2）； （3）； （4）。 5. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 4, characterized in that, In S23, the feature extraction process using the neural implicit surface reconstruction module is as follows: S23-1: Generate dense 3D query points; represent the registered ROI point cloud as... ,in, The total number of point clouds; for each point According to Gaussian distribution 25 3D query points in a three-dimensional coordinate system were randomly sampled, among which... Defined as The Euclidean distance to its 50th nearest neighbor, and all query points form a set. ; S23-2: SDF Definition and Network Prediction; SDF values ​​are defined as positive for points outside the surface, zero for points on the surface, and negative for points inside the surface; This is demonstrated through an MLP network. Predict the SDF value of the query point The target surface is defined by the zero level set of the SDF. ; S23-3: Projection and Self-Supervised Learning; MLP Networks During Training Simultaneously predict query points SDF value With gradient The query point is then projected along the gradient direction to its nearest neighbor in the ROI point cloud. The projected query point is obtained according to formula (5). ; （5）； S23-4: Geometric constraint loss; Three loss functions are used for optimization: self-supervised projection loss, symbolic consistency loss, and adversarial surface constraint loss. This corrects the SDF prediction bias from the dual dimensions of symbolic stability and surface authenticity, thus addressing the inherent defects of self-supervised learning. The generator and discriminator adopt an alternating optimization strategy until the network converges. S23-5: Surface Extraction; The Marching Cubes algorithm is used to extract a 3D surface mesh model from the learned SDF.

6. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 5, characterized in that, In S24, the process of constructing the joint loss function is as follows: S24-1: Comprehensive anatomical region matching loss; calculate ROI mutual information loss according to formulas (6), (7) and (8) respectively. ROI-weighted multi-class Dice loss and deformation field regularization loss Then, according to formula (9), the weighted fusion is used to obtain the comprehensive anatomical region matching loss. ; （6）； （7）； （8）； = + +λ (9); In the formula, Pref,r represents the ROI region of the point cloud to be registered after fine registration; Pref,r represents the ROI region of the reference point cloud. Mutual information is defined as follows: As an indicator to measure the degree of information sharing between two random variables, , They are respectively , marginal entropy, Let be the joint entropy of the two. Number of structure categories; and The source point cloud and the target point cloud are respectively the first... Point sets of class structures; Category weights; For the generated deformation field; For deformation field in voxel The gradient at the point; Ω represents the entire three-dimensional voxel space; These are the weighting coefficients for the regularization loss; S24-2: Surface reconstruction loss; self-supervised loss is calculated using formulas (10), (11), (12) and (13), respectively. Symbol consistency loss Generator adversarial loss Adversarial loss against discriminator ; （10）； （11）； （12）； （13）； In the formula, For projection query points; For neighboring points; This refers to the batch size during the training process; The gradient of the signed distance function; Cosine distance; ; ; ; The SDF value predicted by the generator; The ideal surface SDF value; S24-3: Joint loss function; the joint loss function is obtained by fusing the comprehensive anatomical region matching loss and surface reconstruction loss according to formula (14). ; （14）。 7. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 6, characterized in that, In step 3, the process of achieving accurate cross-modal alignment using the three-level point cloud registration framework is as follows: S31-1: Input a two-dimensional ultrasound image sequence, generate a segmentation mask frame by frame through the nnUNet segmentation network, and extract the boundary voxels of the target anatomical structure; S31-2: Solve the translation, rotation, and scaling transformation matrices through the rigid alignment module to unify the point cloud to be registered to the global coordinates of the reference point cloud and eliminate global spatial misalignment; S31-3: The dual-stream encoder of the fine registration network extracts dual-modal features, enhances cross-modal feature interaction through a multi-scale self-attention module, and optimizes local detail alignment by combining comprehensive anatomical region matching loss, outputting a precisely registered 3D point cloud.

8. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 7, characterized in that, In step 3, the process of outputting a high-precision 3D point cloud through the spatial point cloud generation module is as follows: S32-1: Extract the boundary voxel index of the segmentation mask ; S32-2: Through affine transformation matrix Mapping voxel coordinates to three-dimensional physical coordinates As shown in formula (15); （15）； In the formula, , , These represent the voxel spacing of the ultrasound image along each axis, used to compensate for resolution differences in imaging devices; direction cosine matrix. The rotational orientation of the image coordinate axes relative to the physical spatial anatomical axes is defined; This represents the offset of the image origin in the physical coordinate system; S32-3: Apply the fine registration transformation operator The final three-dimensional surface point cloud is obtained. .

9. The ultrasonic three-dimensional surface reconstruction method based on point cloud registration according to claim 8, characterized in that, In step 3, the process of generating a smooth, continuous, and anatomically complete three-dimensional ultrasound model via the neural implicit surface reconstruction module is as follows: S33-1: Construct an implicit function network based on a multilayer perceptron, taking spatial coordinates as input and outputting the symbolic distance function value; S33-2: Sample 3D query points around the registered point cloud according to a Gaussian distribution, covering the internal and external spatial regions of the target tissue; S33-3: Learning the signed distance function and gradient of the query point in a self-supervised manner using a multilayer perceptron; S33-4: Introduce a discriminator network to construct an adversarial surface constraint (OSC-ADL), which enhances surface smoothness and detail fidelity by discriminating the difference in normal vector distribution between the predicted surface and the real anatomical surface; S33-5: Introducing symbol consistency constraints to stabilize symbol prediction, and then optimizing it together with adversarial surface constraints to enhance the clarity of surface boundaries; S33-6: The Marching Cubes algorithm is used to extract the zero level set of the signed distance function to generate a smooth and continuous three-dimensional surface mesh model.

Images