Methods, devices, media, and software products for 3D point cloud data processing based on propulsion front surface reconstruction.

Abstract

The embodiment of the specification provides a three-dimensional point cloud data processing method, device, medium and program product based on a pushing front surface reconstruction, and relates to the field of intelligent medical treatment. The method comprises the following steps: acquiring three-dimensional point cloud data of a target object; constructing a spatial index of the three-dimensional point cloud data; searching for neighboring points of each point in the point cloud data according to the spatial index, locating the neighboring points of each point in the point cloud data, and calculating an average neighboring distance between each point and the neighboring points; processing the point cloud data by grid downsampling according to the average neighboring distance to obtain processed point cloud data; selecting at least one initial point from the processed point cloud data as a starting point of surface reconstruction, creating a local surface sheet of a geometric shape based on the initial point; taking the local surface sheet as a starting front for a reconstruction process; and continuously expanding outward on the basis of the starting front, adding new local surface sheets to construct an entire surface of a cardiac internal cavity, and obtaining a reconstructed three-dimensional model.

Description

Technical Field

[0001] This invention relates to the field of intelligent healthcare, and more specifically, to a method, apparatus, medium, and program product for processing three-dimensional point cloud data based on the reconstruction of the propellant surface. Background Technology

[0002] Existing surface reconstruction techniques are mainly divided into those based on traditional algorithms and those based on deep learning. Traditional algorithm-based surface reconstruction techniques include Poisson reconstruction and Alpha Shape reconstruction. Poisson reconstruction generates smooth, continuous surfaces by solving the Poisson equation, making it suitable for processing noisy point cloud data and widely used in medical imaging and 3D scanning. Alpha Shape reconstruction, on the other hand, constructs geometric shapes by controlling the alpha parameter, effectively handling complex topological structures and often used for local surface construction and geometric analysis. Through these traditional geometric and topological algorithms, surface reconstruction can generate high-precision triangular meshes and 3D models.

[0003] Deep learning-based surface reconstruction methods automatically generate 3D models by learning from large amounts of 3D data using neural networks. Using techniques such as Convolutional Neural Networks (CNNs) or Generative Adversarial Networks (GANs), deep learning methods can extract complex features from 2D images or point cloud data to directly generate 3D surface structures. These methods are commonly used in fields such as organ reconstruction in medical imaging and structural modeling of 3D objects. Leveraging the powerful data processing capabilities of deep learning models, they can achieve automated surface reconstruction in complex scenarios.

[0004] The performance of traditional surface reconstruction algorithms is often significantly affected by parameter settings, especially when dealing with complex geometries or noisy data, making it difficult to maintain consistent reconstruction accuracy across different scenarios. While deep learning-based surface reconstruction algorithms can automate the processing of large-scale data, they are highly dependent on high-quality training data, and the interpretability of model results is low. When faced with noisy or unknown data, it is difficult to guarantee high-precision reconstruction results.

[0005] Furthermore, when processing complex geometric structures or noisy 3D model data, due to limitations in technological development or the original data, the constructed 3D models often contain interfering information or unnecessary tissue parts, resulting in noise or artifacts. This leads to the 3D model presented in clinical practice having insufficient accuracy to meet the operational needs of clinicians, affecting the accuracy of doctors' judgment and operation during surgery. Moreover, non-real-time erasing methods cannot eliminate interfering information in a timely manner, delaying the making of key decisions, thereby increasing the risk and complexity of the surgery.

[0006] Especially when dealing with complex structures such as the cardiac chambers, the construction of their 3D models faces extremely severe technical challenges. The cardiac chambers have very unique anatomical characteristics: First, the inner walls of the chambers are not smooth curved surfaces, but are densely covered with crisscrossing trabecular structures. This fine and irregular geometry is easily confused with noise in image data. Second, there are complex connections between the various chambers of the heart, with dramatic changes in the topology of the valve region and the apex, often accompanied by interference from small suspended structures such as chordae tendineae and papillary muscles. While existing point cloud-based mesh refinement or denoising algorithms exist, most are designed for regular geometries or rigid objects, and their direct application to cardiac chamber model processing has significant limitations. 1. Lack of adaptive ability to perceive anatomy: Conventional AI modeling processing methods are often based on global curvature or high-frequency signal filtering, which makes it difficult to distinguish between normal physiological complex textures and pathological / interference artifacts in the heart chambers. While removing noise, the algorithm can easily erase important microtubule anatomical features in the heart chambers, or leave interference information due to excessive preservation of details, resulting in a distorted model after processing.

[0007] 2. Inability to meet the dual requirements of real-time performance and precision during surgery: During cardiac interventional procedures, surgeons need to observe the internal structures of the heart chambers in real time. Existing non-real-time erasure methods cannot cope with the large amount of data and complex topological structures in cardiac models, and cannot promptly eliminate floating interference point clouds or erroneous meshes. When surgeons attempt to identify critical areas such as the ventricular septum, residual artifacts can severely obstruct the field of vision, making it difficult for them to accurately determine the contact relationship between the catheter tip and the heart wall, increasing the risk of perforation or misoperation. Summary of the Invention

[0008] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention provides a method, apparatus, medium, and program product for processing three-dimensional point cloud data based on propellant surface reconstruction; the method of this invention can adaptively adjust parameter settings according to the characteristics of different organs, thereby achieving more stable and accurate surface reconstruction.

[0009] The first aspect of this application discloses a method for processing three-dimensional point cloud data based on surface reconstruction before propulsion, the method comprising: S101, acquire the 3D point cloud data of the target object; S102, construct a spatial index for the 3D point cloud data; search for the nearest neighbor of each point according to the spatial index, locate the nearest neighbor of each point in the point cloud data, and calculate the average nearest neighbor distance between each point and its nearest neighbor. S103, based on the average neighbor spacing, the point cloud data is processed by grid downsampling to obtain the processed point cloud data; S104, Select at least one initial point from the processed point cloud data as the starting point for surface reconstruction, and create a local surface patch with a geometric shape based on the initial point; use the local surface patch as the starting front of the reconstruction process; S105, based on the initial frontal line, continuously expands outward, adding new local surface patches to construct the entire surface of the cardiac cavity, resulting in a reconstructed three-dimensional model.

[0010] In some embodiments, the target includes any one or more of the cardiac chambers, such as the left atrium, right atrium, left ventricle, and right ventricle.

[0011] The second aspect of this application discloses a model processing method for adaptive thinning and interactive erasing of 3D meshes based on point clouds, the method comprising: Obtain the reconstructed three-dimensional model obtained according to the method described in the first aspect of this application; Calculate and obtain the number of local surface patches in the 3D model; calculate the maximum subdivision level based on the number of local surface patches; Obtain the spatial boundary parameters of the 3D model, and calculate the size range of the 3D model based on the spatial boundary parameters; set the maximum allowable side length of a single newly generated local surface patch during the subdivision process according to the size range. The subdivision filter is configured according to the maximum number of subdivisions and the maximum allowed edge length. The subdivision filter is used to control the division granularity of newly generated local surface patches in the 3D model. The 3D model is adaptively subdivided according to the configured subdivision filter to obtain the adaptively subdivided 3D model.

[0012] A third aspect of this application discloses an interactive erasing method for use in the second aspect of this application, the method comprising: Create a model selector for selecting specific regions of the 3D model, a point selector for selecting point clouds in the 3D model, and a locator for spatial search; In response to the operation command of the model selector for at least one region object in the 3D model, the region object is picked up to obtain the picked region object; In response to the operation command of the point selector for the point cloud data of the picked region object, the point cloud data is picked up to obtain the point cloud location information; In response to the locator's operation command for the target point, the system searches for all surrounding vertices within the set radius of the target point to obtain a set of indices for all vertices. The facet to be deleted is identified as the deletion unit. In response to the operation command for the deletion unit, an erasure operation is performed, and the 3D model of the deleted unit is updated and obtained.

[0013] The fourth aspect of this application discloses a computer device, the device comprising: a memory and a processor; the memory for storing a computer program; and the processor for executing the computer program to implement the steps of the above-described method.

[0014] The fifth aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0015] The sixth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0016] This application has the following advantages: 1. This application addresses the issues of high noise and sparse, uneven distribution of point clouds obtained from multi-view cardiac ultrasound image registration. It constructs a spatial index, locates each point based on the index, and calculates the average neighbor distance between each point and its neighbors to achieve a quantitative perception of the local density characteristics of the point cloud. Furthermore, it processes the data using a grid downsampling method based on the average neighbor distance to smooth noise and balance the point cloud distribution, providing stable data for subsequent surface reconstruction and improving the stability and robustness of data from different cases during the reconstruction process.

[0017] 2. This application employs a forward surface reconstruction method, which gradually expands outward from the initial point to generate a curved surface. This method is particularly suitable for anatomical organs such as cardiac chambers with clear internal and external boundaries and complex geometric structures. Combined with preprocessing steps for different data, this application can more accurately generate continuous and smooth three-dimensional surface models. It can effectively avoid the problems of noise or artifacts caused by the inclusion of interfering information or unnecessary tissue in traditional reconstruction methods. It can also effectively avoid surface discontinuities, breaks, or voids that may occur in traditional methods, and is more consistent with the actual anatomical morphology of the complex geometric structure of the heart.

[0018] 3. Unlike traditional algorithms that indiscriminately refine or smooth the entire model, this application fully considers the structural heterogeneity of the cardiac chambers. Through adaptive refinement based on point cloud curvature changes and local density, it accurately identifies the geometric differences between the normal physiological structures of the cardiac chamber walls and image noise. This method selectively preserves the high curvature features of key cardiac regions, avoiding the loss of cardiac microanatomical structures caused by the "one-size-fits-all" denoising approach of conventional AI algorithms, thus ensuring the anatomical realism of the model in clinical applications.

[0019] 4. This application addresses the challenges of limited visibility and severe structural obstruction during cardiac surgery. By considering the topological connectivity of the cardiac chambers, it proposes an interactive erasure method based on point cloud location information (not simple geometric clipping). In practice, surgeons can identify and erase floating artifacts or non-target areas obstructing critical views in real time, as needed. This real-time feedback mechanism allows surgeons to quickly obtain a clear view of the "surgical passage," significantly improving the accuracy of judgment and operational safety when catheters navigate through complex cardiac chambers. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the method flow provided in the first aspect of the present invention; Figure 2 This is a schematic diagram of a three-dimensional point cloud data processing system based on the reconstruction of the propellant surface provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the architecture of an exemplary computing device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the storage medium provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the cardiac chamber acquisition system provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a two-dimensional image of the cardiac chambers provided in an embodiment of the present invention; Figure 8 This is a Canny edge segmentation effect diagram provided in an embodiment of the present invention; Figure 9 This is a three-dimensional point cloud effect of the cardiac chamber after registration, provided in an embodiment of the present invention. Figure 10 This is a diagram illustrating the effect of the surface reconstruction algorithm before propulsion provided in an embodiment of the present invention. Figure 11 These are before-and-after effect diagrams of adaptive subdivision provided in an embodiment of the present invention; wherein, Figure 11 Image 'a' shows the effect before adaptive subdivision. Figure 11 b is the effect diagram after adaptive subdivision; Figure 12 This is a real-time erasure model effect diagram provided in the embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0023] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Figure 1 This is a schematic flowchart of a three-dimensional point cloud data processing method based on surface reconstruction before propulsion provided by an embodiment of the present invention. Specifically, the method includes the following steps: S101, acquire the 3D point cloud data of the target object; In some embodiments, the target includes any one or more of the cardiac chambers, such as the left atrium, right atrium, left ventricle, and right ventricle.

[0026] In some embodiments, the target is any animal (e.g., a mammal), including but not limited to humans, non-human primates, rodents, etc., who will become the recipient of a specific treatment.

[0027] In some embodiments, the 3D point cloud data is registered, and the registration method includes: a. Use an ultrasound probe with an IMU to acquire multi-view two-dimensional images of the heart chambers. The IMU is used to record the acceleration and angular velocity information when the ultrasound probe acquires each two-dimensional image. It also includes: using the Canny algorithm to segment the edges of a 2D image. The specific steps include: applying Gaussian filtering to the 2D image to smooth it and reduce noise, thus reducing false detections in subsequent steps; using the Sobel operator to calculate the gradients of the image in the horizontal and vertical directions to find intensity changes in the image; comparing the gradient intensity of the current pixel with the gradient intensity of neighboring pixels along the positive and negative gradient directions; retaining the pixels with the largest gradient intensity as edge points; and filtering out strong edge pixels based on predefined high and low thresholds to obtain chamber edge information. b. Convert each two-dimensional image into three-dimensional point cloud data according to the coordinate system transformation; specifically, establish a coordinate system with the ultrasound probe as the origin and the z-axis perpendicular to the ultrasound image, and transform all two-dimensional points on the ultrasound image into three-dimensional points. For example, a point (x, y) on the two-dimensional image is transformed into a three-dimensional coordinate point (x, y, 0). c. Based on the acceleration and angular velocity information acquired by the IMU, the various 3D point cloud data are transformed into a unified coordinate system to obtain 3D point cloud data. Specifically, a handheld ultrasound probe is used to acquire 3D point cloud data from different perspectives within the cardiac chambers. Simultaneously, the IMU records the acceleration and angular velocity changes of the ultrasound probe in real time, thereby calculating the real-time position and pose information of the ultrasound probe. Finally, based on the acquired position and pose information, the cardiac chamber point cloud data from different perspectives are transformed into a unified coordinate system.

[0028] S102, construct a spatial index for the 3D point cloud data; search for the nearest neighbor of each point according to the spatial index, locate the nearest neighbor of each point in the point cloud data, and calculate the average nearest neighbor distance between each point and its nearest neighbor. In some embodiments, proximity search includes nearest neighbor search and radius search; the KdTreeFLANN class in the PCL (Point CloudLibrary) library is used to construct a K-dimensional tree of the point cloud data to optimize the subsequent proximity search process and improve the performance of proximity search (such as nearest neighbor search and radius search).

[0029] S103, based on the average neighbor spacing, the point cloud data is processed by grid downsampling to obtain the processed point cloud data; S104, Select at least one initial point from the processed point cloud data as the starting point for surface reconstruction, and create a local surface patch with a geometric shape based on the initial point; use the local surface patch as the starting front of the reconstruction process; Optional geometric shapes include triangles and quadrilaterals.

[0030] S105, based on the initial frontal line, continuously expands outward by combining the local density and distribution of surrounding points to ensure the continuity and smoothness of the entire surface, and adds new local surface patches to construct the entire surface of the cardiac cavity, thus obtaining a reconstructed three-dimensional model.

[0031] In some embodiments, after S105, the method further includes: detecting whether there are voids on the entire surface of the reconstructed three-dimensional model; if voids are found, filling the voids to obtain a three-dimensional reconstructed model of the cardiac chamber, ensuring that the generated model is closed and complete.

[0032] This embodiment also discloses a model processing method for adaptive thinning and interactive erasing of 3D meshes based on point clouds, the method including: Obtain the reconstructed three-dimensional model obtained according to the method described in the first aspect of this application; Calculate and obtain the number of local surface patches in the 3D model; calculate the maximum number of subdivisions based on the number of local surface patches; the maximum number of subdivisions is calculated by multiplying the logarithm of the number of local surface patches by a preset factor; the preset factor is set according to time budget, memory budget, device computing power, region of interest, and task mode; Obtain the spatial boundary parameters of the 3D model, and calculate the size range of the 3D model based on the spatial boundary parameters; set the maximum allowable side length of a single newly generated local surface patch during the subdivision process according to the size range; in some specific embodiments, the maximum side length of a single local surface patch is one-hundredth of the model size; while maintaining the original shape characteristics, the subdivided mesh model adds necessary levels of detail. The subdivision filter is configured according to the maximum number of subdivisions and the maximum allowed edge length. The subdivision filter is used to control the division granularity of newly generated local surface patches in the 3D model. The 3D model is adaptively subdivided according to the configured subdivision filter to obtain the 3D model after adaptive subdivision, which not only enhances the operability of the model, but also significantly improves the accuracy of erasing and editing operations.

[0033] A third aspect of this application discloses an interactive erasing method for use in the second aspect of this application, the method comprising: Create a model selector for selecting specific regions of the 3D model, a point selector for selecting point clouds in the 3D model, and a locator for spatial search; In response to the operation command of the model selector for at least one region object in the 3D model, the region object is picked up to obtain the picked region object; In response to the operation command of the point selector for the point cloud data of the picked region object, the point cloud data is picked up to obtain the point cloud location information; In response to the locator's operation command for the target point, the system searches for all surrounding vertices within the set radius of the target point to obtain a set of indices for all vertices. The facet to be deleted is identified as the deletion unit. In response to the operation command for the deletion unit, an erasure operation is performed, and the 3D model of the deleted unit is updated and obtained.

[0034] In some embodiments, a “deletion unit” should be understood as a set of indices of faces to be deleted; vertices / edges themselves are not direct “deletion units”, but rather are cleaned up as an adjunct (e.g., if a vertex loses all its adjacent faces after face deletion, they are removed together). Model selector (S202): First selects the region object (coarse-grained ROI), such as a blood vessel / a piece of the atrium; obtains the "picked region object"; Point selector (S203): Precisely picks the point cloud / vertices within the region to obtain the point cloud position information (seed point); Locator (S204): With the seed point as the center, finds all surrounding vertices within a set radius to obtain the vertex index set (Kd tree / spatial index acceleration); The purpose of recording the point cloud location information and index set is to control the range and accuracy, and to avoid "wiping too much / wiping too little".

[0035] In some embodiments, after obtaining the index set of all vertices, the method further includes: traversing each vertex index to obtain the actual point coordinates from the backed-up original data; and for each vertex, finding the local surface patch associated with each vertex to obtain the index set of the associated local surface patch.

[0036] In some embodiments, before performing the erase / delete unit operation, the method further includes: backing up the data of the 3D model to ensure that it can be restored to the state before the operation in case of errors or user needs.

[0037] Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention, such as... Figure 3 As shown, the device 2000 may include: one or more processors 2010 and one or more memories 2020; wherein the memories store computer-readable code that, when run by the one or more processors, can perform the methods described above.

[0038] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, operations, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor, and can be based on an x86 or ARM architecture.

[0039] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0040] For example, the method or apparatus according to embodiments of this disclosure can also be used by means of Figure 4 The architecture of the computing device 3000 shown is used for implementation. For example... Figure 4 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 4 One or more components in the computing device shown.

[0041] This invention also includes a computer-readable storage medium, such as... Figure 5The diagram illustrates a storage medium 4000 provided in an embodiment of the present invention. The computer storage medium 4020 stores computer-readable instructions 4010. When the computer-readable instructions 4010 are executed by a processor, the method described above according to embodiments of the present disclosure can be performed. The computer-readable storage medium in the embodiments of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), and Direct Memory Bus Random Access Memory (DRRAM). It should be noted that the memory used in the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0042] This disclosure also provides a computer program product or system, including a computer program that, when executed by a processor, implements the steps of the above-described method. The computer program product or computer program includes: Point cloud data acquisition module 201 is used or configured to acquire three-dimensional point cloud data of a target object; Distance calculation module 202 is used or configured to construct a spatial index of 3D point cloud data; search for each point based on the spatial index to locate the neighboring points of each point in the point cloud data; and calculate the average neighbor distance between each point and its neighboring points. Point cloud data preprocessing module 203 is used or configured to process point cloud data by grid downsampling based on average neighbor spacing to obtain processed point cloud data; The surface patch creation module 204 is used or configured to select at least one initial point from the processed point cloud data as the starting point for surface reconstruction, create a local surface patch with a geometric shape based on the initial point, and use the local surface patch as the starting front of the reconstruction process; The 3D model reconstruction module 205 is used or configured to continuously expand outward from the starting front to add new local surface patches to construct the entire surface of the cardiac cavity, thereby obtaining a reconstructed 3D model. Specific implementation examples: This invention constructs an accurate three-dimensional model of the target by combining surface reconstruction technology, which solves the problem of traditional catheter positioning methods relying too much on doctors' experience, greatly assists doctors in accurate navigation, and thus improves the efficiency and success rate of surgery. This measuring device mainly solves the problem of three-dimensional modeling of the cardiac cavity from multiple perspectives. The principle of the three-dimensional modeling and real-time erasure algorithm of the cardiac cavity: (1) Use an ultrasound probe with an inertial measurement unit (IMU) to acquire multi-view two-dimensional images of the cardiac cavity; (2) Convert each two-dimensional image into three-dimensional point cloud data according to the coordinate system transformation; (3) Based on the acceleration and angular velocity information acquired by the IMU, convert each three-dimensional point cloud data into a unified coordinate system to form complete three-dimensional point cloud data; (4) Use the Advancing Front Surface Reconstruction (AFSR) algorithm to perform surface modeling of the three-dimensional point cloud; (5) Use an adaptive subdivision algorithm to realize the automatic subdivision of the mesh model; (6) Real-time erasure function of the three-dimensional model is realized based on the visualization toolkit (VTK). The cardiac cavity acquisition system is as follows Figure 6 As shown, 1 is the display: used to display a three-dimensional model of the heart chambers. 2 is the connector: a device that connects the display and the catheter. 3 is the catheter: a device used for medical diagnosis, treatment, and monitoring. 4 is the ultrasound probe: used to acquire two-dimensional ultrasound data. 5 is the IMU: used to acquire the position and pose information of the ultrasound probe at different time points.

[0044] The three-dimensional modeling and real-time erasure method based on intracardiac ultrasound catheters mainly includes: two-dimensional ultrasound image acquisition, three-dimensional point cloud acquisition, IMU registration, point cloud surface reconstruction, and real-time model erasure.

[0045] 1. Acquisition of two-dimensional ultrasound images This invention provides a two-dimensional image of the cardiac chambers acquired using an ultrasound probe. The specific acquisition steps are as follows: First, an ultrasound probe is inserted into the heart chambers via a vein using a catheter. The probe has a sound wave transmitter and receiver, capable of sending and receiving ultrasound signals. Second, the probe emits high-frequency sound waves that pass through the heart tissue. When the sound waves encounter different tissue or organ boundaries, some of the sound waves are reflected back. The probe receives these reflected sound waves and converts them into electrical signals. Then, the computer processes the received electrical signals to generate a two-dimensional image of the heart chambers. The two-dimensional image of the cardiac chambers acquired using the ultrasound probe is shown below. Figure 7 As shown.

[0046] 2. Canny edge segmentation This invention employs the Canny algorithm for edge segmentation of two-dimensional ultrasound images. The specific steps are as follows: First, the input image is subjected to Gaussian filtering to smooth the image and reduce noise, thereby reducing false detections in subsequent steps. The Gaussian filter discretizes the Gaussian function, substituting the corresponding horizontal and vertical coordinate indices into the Gaussian function to obtain the corresponding values. The two-dimensional Gaussian function is shown in the following formula: where (x, y) are coordinates, and σ is the standard deviation.

[0047]

[0048] Different filter sizes will produce different values. Below is the formula for calculating the value of a (2k+1)×(2k+1) filter:

[0049] Next, the gradient of the image is calculated. The Sobel operator is used to calculate the gradient of the image in the horizontal and vertical directions to find the intensity changes in the image. The Sobel operator consists of two 3×3 matrices, respectively. and The former is used to calculate the gradient matrix of x pixels in an image. The latter is used to calculate the gradient matrix of the y-pixels of the image. The specific form is shown in the following formula: =

[0050] =

[0051] in, I Let be a grayscale image matrix, and here... This represents a cross-correlation operation (convolution can be viewed as a cross-correlation operation after rotating the convolution kernel by 180°). It should be noted that the origin of the image matrix coordinate system is at the top left corner, and the x-direction is from left to right, while the y-direction is from top to bottom. Then, from... Calculate the gradient intensity matrix .

[0052] Next, the gradient strength of the current pixel is compared with the gradient strengths of its neighboring pixels along the positive and negative gradient directions. The pixel with the largest gradient strength in the region is retained as an edge point. Finally, a high threshold and a low threshold are defined. Pixels with gradient strength below the low threshold will be suppressed and not considered edge points; pixels with gradient strength above the high threshold are identified as strong edges and will be retained as edge points. The effect of Canny edge detection is as follows: Figure 8 As shown.

[0053] 3. IMU point cloud registration This invention uses an IMU sensor to achieve point cloud registration. The specific steps are as follows: First, a coordinate system is established with the ultrasound probe as the origin and the z-axis perpendicular to the ultrasound image, transforming two-dimensional points on the ultrasound image into three-dimensional points. For example, a point (x, y) on the two-dimensional image is converted into a three-dimensional coordinate point (x, y, 0). Second, the handheld ultrasound probe is used to acquire three-dimensional point cloud data from different perspectives within the heart chambers. Simultaneously, the IMU records the acceleration and angular velocity changes of the ultrasound probe in real time, thereby calculating the real-time position and pose information of the ultrasound probe. Finally, based on the acquired position and pose information, the heart chamber point cloud data from different perspectives are transformed into a unified coordinate system. The registered three-dimensional point cloud effect of the heart chambers is shown below. Figure 9 As shown.

[0054] 4. Advance surface reconstruction The present invention uses an adaptive parameter-based surface reconstruction algorithm to realize the surface reconstruction of three-dimensional point clouds. The specific implementation steps are as follows: (1) The KdTreeFLANN class in the PCL (Point Cloud Library) is used to construct a K-tree of point cloud data to optimize the subsequent proximity search process. By performing K-nearest neighbor search on each point, the neighboring points of each point in the point cloud are quickly located, and the average neighbor distance between points is effectively calculated; (2) Based on the average neighbor distance of the point cloud, the point cloud data is simplified by grid downsampling; (3) Select one or more initial points from the processed point cloud as the starting point for surface reconstruction, and based on the selected initial points, create a small local surface patch (usually a triangle or quadrilateral), which serves as the starting front edge for the reconstruction process; (4) Front edge expansion: Based on the current front edge, continuously expand outward, gradually construct the entire surface by adding new patches, and consider the local density and distribution of surrounding points to ensure the continuity and smoothness of the surface; (5) Detect and fill any holes that may be generated during the reconstruction process to ensure that the generated model is closed and complete, and smooth the reconstructed surface to improve the visual quality and physical properties of the model, and maintain consistency with the original point cloud data as much as possible. The model effect after reconstruction by the adaptive parameter advance surface reconstruction algorithm is as follows Figure 10 As shown.

[0055] 5. Adaptive subdivision of 3D mesh models This study proposes a bounding box-based adaptive subdivision algorithm for 3D meshes, aiming to solve the problem of not being able to accurately erase specific areas due to the excessive size of individual triangular faces in the mesh model. This algorithm effectively improves the accuracy and flexibility of the mesh model in detail processing by automating the subdivision process. The specific implementation steps are as follows: (1) Obtaining the number of points and faces: Calculate and obtain the number of points and faces in the 3D model; (2) Obtain the spatial boundary of the model and calculate the overall size range of the model based on these boundaries; (3) Determining the maximum number of subdivisions: Calculate the maximum number of subdivisions based on the number of faces and multiply the logarithm of the number of faces by a preset factor; (4) Set the maximum side length to one percent of the model size based on the size range of the model. This parameter is used to control the maximum allowable side length of each newly generated face during the subdivision process, ensuring that the subdivided mesh model increases the necessary level of detail while maintaining the original shape features; (5) Configuring the subdivision filter: The subdivision filter is correctly configured according to the above parameter settings, especially the setting of the maximum side length parameter, to ensure that the generated new face can meet the needs of fine operation; (6) Executing adaptive subdivision: After the configuration is completed, the subdivision filter is triggered to perform adaptive subdivision on the entire 3D model. This process not only enhances the model's operability but also significantly improves the accuracy of erasing and editing operations. The effects before and after adaptive subdivision are as follows: Figure 11 a, Figure 11 As shown in b. The subdivision filter is an algorithm module used to control the granularity of the triangular facets in the 3D mesh. It allows for finer mesh details, providing conditions for subsequent local erasing or editing. Its function is: (1) Cut the oversized triangular facet into more smaller triangles according to certain rules (e.g., maximum allowed side length); (2) Preserve the original curved surface shape, but make the mesh distribution more uniform and the facet smaller, so as to facilitate subsequent local editing (e.g., erasing a small area); (3) Improve the resolution and editability of the model.

[0056] 6. Eraser Real-time Erasing Model This invention describes a real-time erasing algorithm for three-dimensional models. The algorithm achieves accurate model editing through efficient graphics processing technology and user interaction design. The following are the detailed implementation steps of the algorithm: (1) Initialization and configuration.

[0057] Instantiate a selector for selecting specific parts of the 3D model, enabling users to specify and select specific areas of the model; instantiate a selector for precisely selecting points in the model, allowing users to accurately select specific points to perform editing operations; initialize a locator based on a Kd-tree spatial search structure to accelerate access and location of points in large-scale point cloud data. (2) Interactive event response. When the erase mode is activated, the system tracks the current position of the mouse in real time and picks up the 3D object under the mouse position through the model selector. Use the point selector to accurately pick up the point cloud data at that position, and locate the point through the picking position information obtained by the model selector. When the user presses the left mouse button, the system marks the start of the erase mode and records the current state of the model to prepare for the erase operation to be performed. When the user releases the left mouse button, the system ends the erase mode, solidifies the changes and updates the model display to ensure the immediacy and accuracy of the user's operation. Use the locator to find all surrounding vertices within the set radius, and the method returns the set of indices of these vertices. Traverse each vertex index and obtain the actual point coordinates from the backed-up original data to prepare data for subsequent erase operations.

[0058] For each found vertex, query all triangles associated with that vertex and collect their indices into a set to avoid duplicate processing. Then, call the erase method one by one to delete these triangles. (3) Data update and reconstruction. Use the delete cell method to remove all cells marked for deletion from the displayed data to maintain the integrity of the data structure. Update the displayed model data to reflect the latest state and re-render the scene so that the erasure effect is immediately visible. (4) Post-processing. Back up the model data before the erasure operation to ensure that it can be restored to the state before the operation in case of errors or user needs. After the operation is completed, restore the original data or continue to use the modified data as needed. The erased model is as follows Figure 12 As shown.

[0059] The model selector refers to the feature that allows the user to select a region at once, such as a portion of an atrium, a valve, or a segment of a blood vessel. It is generally used to select several faces or subsets of a mesh.

[0060] A vertex selector refers to the ability to precisely select a single mesh vertex by clicking the mouse on a 3D model. This is used for very fine-grained editing, such as erasing several faces or individual points.

[0061] A locator is a tool that, given coordinates, quickly finds the nearest point or surface in a mesh or point cloud. It can even find all points within a certain radius.

[0062] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0063] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0064] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0065] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0066] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0067] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0068] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

Claims

1. A method for processing three-dimensional point cloud data based on the reconstruction of the propellant surface, characterized in that, The method includes: S101, acquire the 3D point cloud data of the target object; S102, construct a spatial index for the three-dimensional point cloud data; search for the nearest neighbor of each point according to the spatial index, locate the nearest neighbor of each point in the point cloud data, and calculate the average nearest neighbor distance between each point and the nearest neighbor. S103, based on the average neighbor spacing, the point cloud data is processed by grid downsampling to obtain the processed point cloud data; S104, Select at least one initial point from the processed point cloud data as the starting point for surface reconstruction, and create a local surface patch with a geometric shape based on the initial point; use the local surface patch as the starting front edge of the reconstruction process; S105, based on the aforementioned starting front, and combined with the local density and distribution of surrounding points, continuously expand outward, adding new local surface patches to construct the entire surface of the cardiac cavity, thus obtaining a reconstructed three-dimensional model.

2. The three-dimensional point cloud data processing method based on pre-propulsion surface reconstruction according to claim 1, characterized in that, The target objects include any one or more of the cardiac chambers, namely the left atrium, right atrium, left ventricle, and right ventricle; Optionally, the proximity search includes nearest neighbor search and radius search; the KdTreeFLANN class in the PCL library is used to construct a K-dimensional tree of the point cloud data to optimize the proximity search.

3. The three-dimensional point cloud data processing method based on the reconstruction of the propellant surface according to claim 1, characterized in that, Following S105, the method further includes: detecting whether there are voids on the entire surface of the reconstructed three-dimensional model; if voids are found, filling the voids to obtain a three-dimensional reconstructed model of the heart chambers. Optionally, the 3D point cloud data is registered, and the registration method includes: A two-dimensional image of the cardiac chamber from multiple perspectives is acquired using an ultrasound probe equipped with an IMU, wherein the IMU is used to record acceleration and angular velocity information when the ultrasound probe acquires each of the two-dimensional images; Each of the two-dimensional images is converted into three-dimensional point cloud data according to the coordinate system transformation; Based on the acceleration and angular velocity information obtained by the IMU, the three-dimensional point cloud data are transformed into a unified coordinate system to obtain three-dimensional point cloud data.

4. A model processing method for adaptive refinement and interactive erasure of 3D meshes based on point clouds, characterized in that, The method includes: Obtain the reconstructed three-dimensional model obtained by the method according to any one of claims 1-3; Calculate and obtain the number of local surface patches in the 3D model; calculate the maximum subdivision level based on the number of local surface patches; Obtain the spatial boundary parameters of the 3D model, and calculate the size range of the 3D model based on the spatial boundary parameters; set the maximum allowable side length of a single newly generated local surface patch during the subdivision process according to the size range. The subdivision filter is configured according to the maximum number of subdivisions and the maximum allowed edge length. The subdivision filter is used to control the division granularity of newly generated local surface patches in the 3D model. The 3D model is adaptively subdivided according to the configured subdivision filter to obtain the adaptively subdivided 3D model. Optionally, the maximum number of subdivisions is calculated by multiplying the logarithm of the number of local surface patches by a preset factor; the preset factor is set according to time budget, memory budget, device computing power, region of interest, and task mode. Optionally, the maximum allowable side length of the single local surface patch is one-Nth of the model size, where N is a natural number greater than 1.

5. An interactive erasing method for claim 4, characterized in that, The method includes: Create a model selector for selecting specific regions of the 3D model, a point selector for selecting point clouds in the 3D model, and a locator for spatial search; In response to the operation command of the model selector for at least one region object in the 3D model, the region object is picked up to obtain the picked region object; In response to the operation command of the point selector for the point cloud data of the picked region object, the point cloud data is picked up to obtain the point cloud location information; In response to the locator's operation command for the target point, the system searches for all surrounding vertices within the set radius of the target point to obtain a set of indices for all vertices. The facet to be deleted is identified as the deletion unit. In response to the operation command for the deletion unit, an erasure operation is performed, and the 3D model of the deleted unit is updated and obtained.

6. The interactive erasure method according to claim 5, characterized in that, After obtaining the set of indices for all vertices, the method further includes: traversing each vertex index to obtain the actual point coordinates from the backed-up original data; and for each vertex, finding the local surface patch associated with each vertex to obtain the set of indices for the associated local surface patch.

7. The interactive erasing method according to claim 5, characterized in that, Before performing the erase / delete unit operation, the method further includes: backing up the data of the 3D model to ensure that it can be restored to the state before the operation in case of errors or user needs.

8. A computer device, characterized in that, The device includes: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

Images