A Planning and Navigation Method and System for a Puncturing Robot

Abstract

The present invention discloses a planning and navigation method and system for a puncture robot. The method comprises: registering preoperative high-resolution images with ultrasound simulation data through a dual-channel convolutional neural network, performing high-precision spatial alignment, and generating multimodal fusion image data. fuse ; The multimodal fusion image data I fuse The system performs topological structure analysis and constructs a semantically enhanced three-dimensional model. Based on the semantically enhanced three-dimensional model, global path planning is performed through obstacle spatial encoding and reinforcement learning pre-planning. Based on the results of global path planning, navigation parameters and a lightweight model that can be imported into the navigation system are generated, and preoperative navigation data is output. The system includes a dual-channel convolutional neural network registration module, a semantically enhanced three-dimensional model construction module, a global path planning module, and a navigation data encapsulation module. This invention improves the accuracy and stability of registration, can effectively solve the problem of multimodal imaging data fusion, and achieves accurate preoperative navigation data output.

Description

Technical Field

[0001] The present invention relates to the technical field of puncture robot navigation, and in particular to a planning and navigation method and system for a puncture robot. Background Art

[0002] With the rapid development of medical imaging technology and computer science, vision-guided puncture robots have been widely used in minimally invasive surgery. Precisely planning and positioning the puncture path, especially during the preoperative phase, is a key issue in the current field of medical robotics. Traditional puncture robot navigation methods often rely on a single imaging data source, such as CT or MRI images. However, these images struggle to meet the requirements for high-precision and robust navigation due to resolution limitations, imaging modality differences, and tissue distortion during acquisition.

[0003] In recent years, robot calibration methods based on binocular vision have become a research hotspot due to their high efficiency and cost-effectiveness. Binocular vision can provide richer depth information by leveraging the spatial relationship between cameras, thereby supporting higher-precision 3D reconstruction and positioning. However, existing binocular vision calibration methods still face certain technical challenges in practical applications, including insufficient image registration accuracy, error accumulation during 3D reconstruction, high image noise interference, and problems with the fusion of images from different modalities.

[0004] Currently, there is a growing market demand for precision preoperative positioning, particularly for puncture robots. Accurate preoperative image registration and path planning can significantly improve surgical safety and efficiency, while reducing operative time and postoperative recovery. Therefore, developing a high-precision binocular vision-based calibration method that can accurately register and effectively integrate preoperative image data is a pressing issue in the field of medical robotics.

[0005] The defects of existing technical solutions are mainly reflected in the following aspects: (1) Insufficient accuracy. Traditional image registration methods often have problems of low registration accuracy and poor algorithm stability. Especially when the image resolution is low or there is noise, it is difficult to achieve high-precision registration effects; (2) Image fusion is difficult. The differences in contrast, texture, and resolution between images of different modalities make it difficult for existing technologies to effectively eliminate information loss or errors when fusing multimodal data; (3) The calibration process is cumbersome. The current calibration method usually requires the cooperation of multiple sensor data, and the operation process is complicated and easily affected by the external environment, resulting in unstable calibration results; (4) Poor real-time performance. The existing vision-based puncture robot calibration method usually has a large amount of computation and poor real-time performance, making it difficult to provide real-time feedback during the operation. Summary of the Invention

[0006] The present invention aims to provide a planning and navigation method and system for a puncture robot, addressing at least one of the aforementioned technical issues. By fusing high-resolution images with ultrasound simulation data and employing a dual-channel convolutional neural network for precise image registration and contrast alignment, the system improves registration accuracy and stability. By constructing a semantically enhanced 3D model and implementing global path planning, the system effectively addresses the fusion of multimodal image data and enables accurate preoperative navigation data output.

[0007] The embodiment of the present invention is achieved as follows:

[0008] A planning and navigation method for a puncture robot, comprising:

[0009] The preoperative high-resolution images and ultrasound simulation data are registered through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multimodal fusion image data. .

[0010] The multimodal fusion image data Perform topological structure analysis and construct semantically enhanced three-dimensional models.

[0011] Based on the semantically enhanced three-dimensional model, global path planning is performed through obstacle space encoding and reinforcement learning pre-planning.

[0012] Based on the results of global path planning, navigation parameters and a lightweight model that can be imported into the navigation system are generated, and preoperative navigation data is output.

[0013] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the process of registering the preoperative high-resolution image with the ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multimodal fusion image data includes:

[0014] Convert preoperative high-resolution imaging data into three-dimensional , converting ultrasound simulation data into three-dimensional .

[0015] Isotropic resampling of preoperative high-resolution image 3D data and ultrasound simulation 3D data was performed to unify the resolution and obtain preoperative high-resolution image space standardized data. Unified data with ultrasound simulation space .

[0016] Design a dual-channel convolutional neural network to unify the ultrasound simulation data The grayscale distribution is mapped to the preoperative high-resolution image space normalized data , perform inter-modal contrast alignment to obtain ultrasound simulation space standardized data .

[0017] The preoperative high-resolution image spatially normalized data Perform elastic transformation to generate preoperative high-resolution registration images , where tri is the trilinear interpolation kernel, for The neighborhood voxels of .

[0018] The preoperative high-resolution registered image Spatially normalized data with the ultrasound simulation Weighted fusion to generate multimodal fusion image data ,in, and is the weight, and the weight value is adaptively adjusted according to the local signal-to-noise ratio.

[0019] Its technical effect is: using a dual-channel convolutional neural network for registration, fully exploring the deep features between the two different modal images, and effectively improving the accuracy of spatial alignment through the nonlinear mapping ability of the neural network. Using a deep learning-based registration algorithm, it can automatically complete the spatial alignment of images and inter-modal contrast alignment. The preoperative high-resolution images and ultrasound simulation data are fused into multimodal fusion image data to maximize the advantages of different modal data and provide richer anatomical structure information. High-precision spatial alignment and modality contrast alignment provide high-quality input data for subsequent topological structure analysis, three-dimensional model construction and path planning.

[0020] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the dual-channel convolutional neural network includes:

[0021] Preoperative image encoder branch and ultrasound image encoder branch, the preoperative high-resolution image space normalization data Input the preoperative image encoder branch to extract preoperative image features , the ultrasound simulation spatial normalization data Input the ultrasound image encoder branch to extract the ultrasound image features .

[0022] Cross-modal attention fusion module, for each level of preoperative image features and ultrasound imaging features , calculate the preoperative image channel weights respectively and ultrasound image channel weights , where GAP is global average pooling, σ is Sigmoid function, W is a learnable weight matrix, and the preoperative image features are and the ultrasound imaging features Perform weighted feature splicing to obtain dynamic fusion multimodal features .

[0023] Deformation field prediction decoder, which dynamically fuses multimodal features Upsample and output a three-dimensional dense deformation field , the preoperative high-resolution imaging spatial normalization data In the three-dimensional dense deformation field Perform elastic transformation.

[0024] Its technical effect is as follows: The cross-modal attention fusion module calculates channel weights for preoperative and ultrasound image features at each level and weights different channels using global average pooling (GAP) and sigmoid functions. This dynamically adjusts the importance of each modal feature to ensure optimal expression of the fused features. The deformation field prediction decoder upsamples the dynamically fused multimodal features to generate a dense three-dimensional deformation field. In this way, the network can predict the deformation patterns of the image in space and accurately achieve spatial registration between the preoperative image and the ultrasound simulation image.

[0025] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the multimodal fusion image data Topological structure analysis and semantically enhanced 3D model construction include:

[0026] Input multimodal fusion image data , perform voxel-level semantic segmentation and output voxel-level semantic segmentation label map .

[0027] The semantic segmentation label map As input, construct a topological feature map ,in, is a set of nodes, each node Represents an anatomical structure, is an edge set, representing the spatial adjacency relationship of the anatomical structure, is the adjacency matrix, element is the edge weight element.

[0028] The topological feature map is integrated with the segmentation result to construct a graph convolutional network, and a semantic attribute vector is assigned to each voxel x. , design node The eigenvector of , graph convolutional layer ,in, For the anatomical category, is the distance to the nearest blood vessel, is the estimated value of tissue elastic modulus, is the geometric feature, is the feature matrix of the l-th layer node, is the degree matrix, is the learnable parameter matrix, is the ReLU activation function, and the output of the graph convolutional network is the enhanced node features .

[0029] The node features and the semantic segmentation label map As input, for each voxel x, the semantic attributes are fused with the graph features of the nearest node ,Isosurfaces are extracted based on the marching cubes algorithm, and enhanced 3D models with semantic labels are generated.

[0030] The technical benefits of voxel-level semantic segmentation are: By classifying the anatomical structure of each voxel, voxel-level semantic segmentation can accurately identify and distinguish different types of tissue or anatomical regions, such as tumors, vascular branches, and bones. Using a weighted cross-entropy loss function and smoothing weights, it effectively balances segmentation errors across different categories, reduces unclear boundaries and ambiguous categories, and improves segmentation accuracy. By constructing a topological feature graph based on the semantic segmentation results, each anatomical structure (such as a tumor or vascular branch) is represented as a node, and the adjacency relationships between nodes (i.e., the spatial connections between anatomical structures) are described as a set of edges. Edge weights are set to control the attenuation of spatial correlation, allowing the topological feature graph to accurately capture the spatial distances and connectivity between anatomical structures. This is particularly true in complex anatomical regions, such as the vascular network near a tumor or bone structure, where it can meticulously reflect the relationships between individual components. When constructing the graph convolutional network, each node (anatomical structure) is assigned a semantic attribute vector. This attribute vector not only contains anatomical category information but also structural information such as tissue elastic modulus, geometric features (such as volume and surface area), and distance to the nearest vessel. The introduction of this information greatly enhances the model's anatomical perception, allowing the generated 3D model to more accurately reflect the physical properties and relationships of different tissues. The fusion of node features and semantic segmentation label maps further enhances the model's semantic understanding of anatomical regions, ensuring that each voxel not only has information about its spatial location but also possesses rich semantic properties, such as whether it is a tumor, blood vessel, or other important anatomical structure. Based on these fused features, the semantically labeled enhanced 3D model generated by extracting isosurfaces using the marching cubes algorithm can meticulously and accurately represent each anatomical structure and its spatial relationships.

[0031] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for the puncture robot, the global path planning is performed based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning, including:

[0032] Taking the semantically enhanced 3D model as input, the risk value of each voxel x is calculated according to the anatomical category and adjacent structures to which each voxel x belongs. ,

[0033] in, is the Euclidean distance from voxel x to the nearest vessel, is the Euclidean distance from voxel x to the nearest bone.

[0034] Divide the 3D space into a voxel grid , each voxel stores the risk value , the probability of generating occupied maps , completing the obstacle space encoding.

[0035] Occupy the map based on the probability , set the starting point and target point , at the starting point and the target point Generate and output multiple candidate rough paths ,Each path consists of a sequence of key waypoints.

[0036] Set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path through reinforcement learning .

[0037] The optimal puncture path Path smoothing is performed using B-spline basis functions, and continuous B-spline curves are output. ,in, is the B-spline basis function, As the control point, complete the global path planning.

[0038] The technical benefit lies in calculating a risk value based on the anatomical class and adjacent structures of each voxel, enabling precise risk assessment. This path planning not only considers spatial geometry but also incorporates functional properties of tissues and organs, such as risk information for blood vessels and bones, enabling safer puncture procedures by avoiding critical areas. By dividing the three-dimensional space into a voxel grid, each voxel stores its risk value and generates a probabilistic occupancy map. Obstacle areas (such as blood vessels and bones) are clearly marked in space. Based on the layout of the probabilistic occupancy map, hazardous areas are identified and avoided, ensuring that the path planning system fully considers various risks, optimizes path selection, and avoids collisions with critical structures during puncture. Global path planning based on a semantically enhanced 3D model not only considers spatial geometry (such as the specific location of blood vessels and bones) but also dynamically optimizes based on anatomical risk information. This allows the puncture robot to automatically and intelligently adjust its path during the task, avoiding potential hazardous areas and maximizing the safety and success rate of the puncture.

[0039] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for the puncture robot, the state space and action space of the local adjustment layer are set, the reward function is set, and the optimal puncture path is generated through reinforcement learning. include:

[0040] Constructing a state space including storing risk values for the voxels in the obstacle space encoding , tissue elastic modulus with real-time ultrasound feedback and the global rough path reference curvature .

[0041] Define the action space, including the robot pitch angle adjustment parameters , yaw angle adjustment parameters , Needle insertion step length , if the current position risk value , then limit the needle step length If it is near bones , then the pitch angle adjustment parameter and the yaw angle adjustment parameter The adjustment is halved.

[0042] Set a multi-objective reward function, take the state space and the action space as inputs of the multi-objective reward function, and the reward item includes a target proximity reward , safety rewards , smoothing penalty and tissue damage penalty , output state vector, generate optimal puncture path .

[0043] The technical benefits of this approach are as follows: By introducing real-time ultrasound feedback of the tissue elastic modulus and the global rough path reference curvature, the state space construction can reflect the soft and hard properties of the tissue (such as the elastic modulus) and the geometric properties of the path (such as the curvature) in real time. This makes local path adjustment more flexible and precise, and can dynamically adapt to the characteristics of different tissues and the adjustment requirements of the puncture path. Through a multi-objective reward function, multiple objectives are comprehensively considered, including a target proximity reward, a safety reward, a smoothness penalty, and a tissue damage penalty. This ensures that during the path optimization process, the robot not only considers the proximity of the path to the target point, but also ensures the smoothness of the path and the safety of the puncture process. The target proximity reward guides the robot to approach the target position more quickly and accurately; the safety reward encourages the robot to avoid high-risk areas such as blood vessels and bones; the smoothness penalty reduces sharp turns in the path to ensure a smooth path; and the tissue damage penalty ensures that the path does not damage sensitive tissues, thus balancing various requirements to generate a safe and efficient optimal path.

[0044] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, generating navigation parameters and a lightweight model that can be imported into a navigation system based on the results of global path planning, and outputting preoperative navigation data include:

[0045] Design trapezoidal velocity profile , maximize the needle velocity while satisfying the acceleration constraint, ,in, is the arc length of the continuous B-spline curve, is the maximum permissible speed, is the arc length of the acceleration segment, is the maximum acceleration, is the total arc length, is the starting arc length of the deceleration segment.

[0046] Calculate total puncture time .

[0047] The trapezoidal velocity profile and the total puncture time The code is then converted into a time-position lookup table and fed into a real-time controller.

[0048] Convert the semantically enhanced 3D model into a sparse octree structure, define node merging rules, , get the octree model ,in, is the spatial area corresponding to the octree node, is the mean value of the attribute within the node, Merge threshold for the attribute.

[0049] The trapezoidal velocity profile The octree model is taken as input and encapsulated as a MATLAB navigation parameter package.

[0050] Its technical effect is that the design of the trapezoidal velocity profile maximizes the needle insertion speed while meeting the constraints of acceleration and deceleration, ensuring the smoothness and safety of path execution. By rationally planning the ratio of acceleration, uniform speed and deceleration segments, the stability of the puncture process is ensured, and errors caused by drastic speed changes are avoided. By encoding the trapezoidal velocity profile and puncture time through a time position lookup table, this information can be quickly imported into the real-time controller, ensuring the smoothness and efficiency of the puncture operation. The introduction of the octree structure transforms the originally complex three-dimensional spatial model into a lightweight structure, simplifies data processing and storage, and effectively compresses the data volume. While reducing the computational and storage burden, it can still retain key spatial information, allowing the spatial regions of different anatomical structures to be efficiently decomposed and processed. The attribute mean and merging threshold within each node control the balance between data accuracy and efficiency, ensuring that the model has higher efficiency in the calculation process while maintaining sufficient accuracy.

[0051] A planning and navigation system for a puncture robot, comprising:

[0052] The dual-channel convolutional neural network registration module is used to register preoperative high-resolution images with ultrasound simulation data through a dual-channel convolutional neural network, perform high-precision spatial alignment, and generate multimodal fusion image data. .

[0053] Semantic enhancement 3D model building module, used to transform the multimodal fusion image data Perform topological structure analysis and construct semantically enhanced three-dimensional models.

[0054] A global path planning module is used to perform global path planning based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning.

[0055] The navigation data encapsulation module is used to generate navigation parameters and a lightweight model that can be imported into the navigation system based on the results of global path planning, and output preoperative navigation data.

[0056] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the planning and navigation method for a puncture robot as described above is implemented.

[0057] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the aforementioned planning and navigation method for a puncture robot.

[0058] The beneficial effects of the embodiments of the present invention are:

[0059] This invention achieves precise spatial alignment by aligning preoperative high-resolution images with simulated ultrasound data using a dual-channel convolutional neural network. Combining the advantages of preoperative imaging and real-time ultrasound data, it provides accurate anatomical information to the puncture robot, significantly improving the precision and safety of puncture procedures. This is particularly effective in preventing damage during punctures around critical anatomical structures such as tumors and blood vessels.

[0060] This invention constructs a semantically enhanced three-dimensional model by topologically analyzing multimodal fusion image data, accurately reflecting the body's anatomical structures and their spatial relationships. This not only increases the semantic depth of the model but also effectively improves the positioning accuracy of the puncture robot, providing more detailed anatomical guidance. When constructing the topological feature map, the graph convolutional network is used to assign enhanced feature information to nodes, further improving the model's accuracy and adaptability. This helps the puncture robot more accurately identify and avoid potential obstacles in complex environments, thereby enhancing the safety of the puncture process.

[0061] This invention utilizes obstacle spatial encoding and reinforcement learning pre-planning to intelligently generate the optimal puncture path based on actual conditions. The introduction of reinforcement learning enables the system to automatically adjust the path in complex environments, avoid dangerous areas, and optimize the operation path during the puncture process, ensuring efficient and safe punctures. By setting up a state space, action space, and multi-objective reward function, the path is dynamically adjusted based on real-time feedback, and local path optimization is combined with actual feedback, further improving the accuracy and stability of the puncture process.

[0062] By designing a trapezoidal velocity profile, the present invention maximizes needle insertion speed while ensuring that acceleration and deceleration conform to physical constraints, thereby improving puncture efficiency and smoothness, effectively reducing errors caused by excessive or insufficient speed, and enhancing the stability of the puncture operation. By calculating the total puncture time and combining it with the trapezoidal velocity profile data input by the real-time controller, the puncture time is precisely controlled, avoiding excessive or short puncture times that may affect surgical outcomes.

[0063] By converting the semantically enhanced 3D model into a sparse octree structure, this invention significantly reduces data processing complexity, enabling rapid response and execution of path planning tasks during real-time control. The introduction of the octree not only saves storage space but also significantly improves computational efficiency, particularly in dynamic environments, helping to reduce latency and increase the responsiveness of the puncture robot. Dynamic adjustments to the path and speed based on real-time ultrasound feedback ensure that the robot can readily adapt to new situations during surgery, providing efficient and safe puncture operations.

[0064] The present invention fuses ultrasonic simulation data with preoperative high-resolution images, utilizes complementary information from different image sources, enhances the ability to identify different tissues, and improves adaptability in complex environments. The robot can perform precise puncture operations on different individual patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 The figure is a flow chart of the planning and navigation method for the puncture robot according to the present invention. DETAILED DESCRIPTION

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0068] Please refer to Figure 1 The first embodiment of the present invention provides a planning and navigation method for a puncture robot, which includes: registering preoperative high-resolution images and ultrasound simulation data through a dual-channel convolutional neural network, performing high-precision spatial alignment, and generating multimodal fusion image data. ; The multimodal fusion image data Perform topological structure analysis and construct a semantically enhanced three-dimensional model; based on the semantically enhanced three-dimensional model, perform global path planning through obstacle space encoding and reinforcement learning pre-planning; based on the results of global path planning, generate navigation parameters and a lightweight model that can be imported into the navigation system, and output preoperative navigation data.

[0069] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the method of registering the preoperative high-resolution image with the ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multimodal fusion image data includes: converting the preoperative high-resolution image data into three-dimensional , converting ultrasound simulation data into three-dimensional ; Isotropically resample the preoperative high-resolution image 3D data and ultrasound simulation 3D data to unify the resolution and obtain preoperative high-resolution image space standardized data Unified data with ultrasound simulation space ; Among them, the steps of isotropic resampling are ,in, is the target voxel spacing, and the sinc function is used for anti-aliasing; a two-channel convolutional neural network is designed to unify the ultrasound simulation data The grayscale distribution is mapped to the preoperative high-resolution image space normalized data , perform inter-modal contrast alignment to obtain ultrasound simulation space standardized data ; Standardize the preoperative high-resolution image space data Perform elastic transformation to generate preoperative high-resolution registration images , where tri is the trilinear interpolation kernel, for The neighborhood voxels of the preoperative high-resolution registration image Spatially normalized data with the ultrasound simulation Weighted fusion to generate multimodal fusion image data ,in, and is the weight, and the weight value is adaptively adjusted according to the local signal-to-noise ratio.

[0070] Its technical effect is: using a dual-channel convolutional neural network for registration, fully exploring the deep features between the two different modal images, and effectively improving the accuracy of spatial alignment through the nonlinear mapping ability of the neural network. Using a deep learning-based registration algorithm, it can automatically complete the spatial alignment of images and inter-modal contrast alignment. The preoperative high-resolution images and ultrasound simulation data are fused into multimodal fusion image data to maximize the advantages of different modal data and provide richer anatomical structure information. High-precision spatial alignment and modality contrast alignment provide high-quality input data for subsequent topological structure analysis, three-dimensional model construction and path planning.

[0071] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the dual-channel convolutional neural network includes: a preoperative image encoder branch and an ultrasound image encoder branch, the preoperative high-resolution image spatial normalization data Input the preoperative image encoder branch to extract preoperative image features , the ultrasound simulation spatial normalization data Input the ultrasound image encoder branch to extract the ultrasound image features ; Cross-modal attention fusion module, for each level of preoperative image features and ultrasound imaging features , calculate the preoperative image channel weights respectively and ultrasound image channel weights , where GAP is global average pooling, σ is Sigmoid function, W is a learnable weight matrix, and the preoperative image features are and the ultrasound imaging features Perform weighted feature splicing to obtain dynamic fusion multimodal features ; Deformation field prediction decoder, which dynamically fuses multimodal features Upsample and output a three-dimensional dense deformation field , the preoperative high-resolution imaging spatial normalization data In the three-dimensional dense deformation field Perform elastic transformation.

[0072] Its technical effect is as follows: The cross-modal attention fusion module calculates channel weights for preoperative and ultrasound image features at each level and weights different channels using global average pooling (GAP) and sigmoid functions. This dynamically adjusts the importance of each modal feature to ensure optimal expression of the fused features. The deformation field prediction decoder upsamples the dynamically fused multimodal features to generate a dense three-dimensional deformation field. In this way, the network can predict the deformation patterns of the image in space and accurately achieve spatial registration between the preoperative image and the ultrasound simulation image.

[0073] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the multimodal fusion image data Perform topological structure analysis and semantic enhancement 3D model construction, including: input multimodal fusion image data , perform voxel-level semantic segmentation and output voxel-level semantic segmentation label map , where the segmentation loss function is , C is the number of categories, is the one-hot encoding of the true category c of voxel x, is the category probability predicted by the network, is the category weight, is the 6-neighborhood of voxel x, is the smoothing term weight, represents the weighted cross entropy, Represents boundary smoothing constraints; the semantic segmentation label map As input, construct a topological feature map ,in, is a set of nodes, each node represents an anatomical structure (e.g., a tumor, a blood vessel branch), is an edge set, representing the spatial adjacency relationship of the anatomical structure, is the adjacency matrix, element For edge weights (specifically, by extracting the centroid coordinates for each anatomical category, such as the tumor centroid , extract key points, set up edge connection rules, if there is a connected path between two key points and the distance ,Pick =20mm, then establish the edge , edge weight ,σ=5mm, control the spatial correlation attenuation, and establish the topological feature map ); The topological feature map is integrated with the segmentation result to construct a graph convolutional network, and a semantic attribute vector is assigned to each voxel x. , design node The eigenvector of , graph convolutional layer ,in, For the anatomical category, is the distance to the nearest blood vessel, is the estimated value of tissue elastic modulus, are geometric characteristics (volume, surface area), is the feature matrix of the l-th layer node, is the degree matrix, is the learnable parameter matrix, is the ReLU activation function, and the output of the graph convolutional network is the enhanced node features ; The node features and the semantic segmentation label map As input, for each voxel x, the semantic attributes are fused with the graph features of the nearest node ,Isosurfaces are extracted based on the marching cubes algorithm, and enhanced 3D models with semantic labels are generated.

[0074] The technical benefits of voxel-level semantic segmentation are: By classifying the anatomical structure of each voxel, voxel-level semantic segmentation can accurately identify and distinguish different types of tissue or anatomical regions, such as tumors, vascular branches, and bones. Using a weighted cross-entropy loss function and smoothing weights, it effectively balances segmentation errors across different categories, reduces unclear boundaries and ambiguous categories, and improves segmentation accuracy. By constructing a topological feature graph based on the semantic segmentation results, each anatomical structure (such as a tumor or vascular branch) is represented as a node, and the adjacency relationships between nodes (i.e., the spatial connections between anatomical structures) are described as a set of edges. Edge weights are set to control the attenuation of spatial correlation, allowing the topological feature graph to accurately capture the spatial distances and connectivity between anatomical structures. This is particularly true in complex anatomical regions, such as the vascular network near a tumor or bone structure, where it can meticulously reflect the relationships between individual components. When constructing the graph convolutional network, each node (anatomical structure) is assigned a semantic attribute vector. This attribute vector not only contains anatomical category information but also structural information such as tissue elastic modulus, geometric features (such as volume and surface area), and distance to the nearest vessel. The introduction of this information greatly enhances the model's anatomical perception, allowing the generated 3D model to more accurately reflect the physical properties and relationships of different tissues. The fusion of node features and semantic segmentation label maps further enhances the model's semantic understanding of anatomical regions, ensuring that each voxel not only has information about its spatial location but also possesses rich semantic properties, such as whether it is a tumor, blood vessel, or other important anatomical structure. Based on these fused features, the semantically labeled enhanced 3D model generated by extracting isosurfaces using the marching cubes algorithm can meticulously and accurately represent each anatomical structure and its spatial relationships.

[0075] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for the puncture robot, the global path planning is performed based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning, including: using the semantically enhanced three-dimensional model as input, calculating the risk value of each voxel x according to the anatomical category and adjacent structures to which each voxel x belongs ,in, is the Euclidean distance from voxel x to the nearest vessel, is the Euclidean distance from voxel x to the nearest bone; divide the three-dimensional space into a voxel grid , each voxel stores the risk value , the probability of generating occupied maps , complete the obstacle space encoding; based on the probability of occupying the map , set the starting point and target point , at the starting point and the target point Generate and output multiple candidate rough paths , each path consists of a sequence of key waypoints; set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path through reinforcement learning ; For the optimal puncture path Path smoothing is performed using B-spline basis functions, and continuous B-spline curves are output. ,in, is the B-spline basis function, As the control point, complete the global path planning.

[0076] The technical benefit lies in calculating a risk value based on the anatomical class and adjacent structures of each voxel, enabling precise risk assessment. This path planning not only considers spatial geometry but also incorporates functional properties of tissues and organs, such as risk information for blood vessels and bones, enabling safer puncture procedures by avoiding critical areas. By dividing the three-dimensional space into a voxel grid, each voxel stores its risk value and generates a probabilistic occupancy map. Obstacle areas (such as blood vessels and bones) are clearly marked in space. Based on the layout of the probabilistic occupancy map, hazardous areas are identified and avoided, ensuring that the path planning system fully considers various risks, optimizes path selection, and avoids collisions with critical structures during puncture. Global path planning based on a semantically enhanced 3D model not only considers spatial geometry (such as the specific location of blood vessels and bones) but also dynamically optimizes based on anatomical risk information. This allows the puncture robot to automatically and intelligently adjust its path during the task, avoiding potential hazardous areas and maximizing the safety and success rate of the puncture.

[0077] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for the puncture robot, the state space and action space of the local adjustment layer are set, the reward function is set, and the optimal puncture path is generated through reinforcement learning. Comprising: constructing a state space including storing risk values of the voxels in the obstacle space encoding , tissue elastic modulus with real-time ultrasound feedback and the global rough path reference curvature ; Define the action space, including the robot pitch angle adjustment parameters , yaw angle adjustment parameters , Needle insertion step length , if the current position risk value , then limit the needle step length If it is near bones , then the pitch angle adjustment parameter and the yaw angle adjustment parameter The adjustment range is halved; a multi-objective reward function is set, and the state space and the action space are used as inputs of the multi-objective reward function. The reward items include the target proximity reward , safety rewards , smoothing penalty and tissue damage penalty , output state vector, generate optimal puncture path .

[0078] The technical benefits of this approach are as follows: By introducing real-time ultrasound feedback of the tissue elastic modulus and the global rough path reference curvature, the state space construction can reflect the soft and hard properties of the tissue (such as the elastic modulus) and the geometric properties of the path (such as the curvature) in real time. This makes local path adjustment more flexible and precise, and can dynamically adapt to the characteristics of different tissues and the adjustment requirements of the puncture path. Through a multi-objective reward function, multiple objectives are comprehensively considered, including a target proximity reward, a safety reward, a smoothness penalty, and a tissue damage penalty. This ensures that during the path optimization process, the robot not only considers the proximity of the path to the target point, but also ensures the smoothness of the path and the safety of the puncture process. The target proximity reward guides the robot to approach the target position more quickly and accurately; the safety reward encourages the robot to avoid high-risk areas such as blood vessels and bones; the smoothness penalty reduces sharp turns in the path to ensure a smooth path; and the tissue damage penalty ensures that the path does not damage sensitive tissues, thus balancing various requirements to generate a safe and efficient optimal path.

[0079] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the above-mentioned method generates navigation parameters and a lightweight model that can be imported into a navigation system based on the results of global path planning, and outputs preoperative navigation data including: designing a trapezoidal velocity profile , maximize the needle velocity while satisfying the acceleration constraint, ,in, is the arc length of the continuous B-spline curve, is the maximum permissible speed, is the arc length of the acceleration segment, is the maximum acceleration, is the total arc length, is the arc length at the start of the deceleration segment; calculate the total puncture time ; The trapezoidal velocity profile and the total puncture time The code is converted into a time position lookup table and input into a real-time controller; the semantically enhanced 3D model is converted into a sparse octree structure, and node merging rules are defined. , get the octree model ,in, is the spatial area corresponding to the octree node, is the mean value of the attribute within the node, is the attribute merging threshold; the trapezoidal velocity profile The octree model is taken as input and encapsulated as a MATLAB navigation parameter package.

[0080] Its technical effect is that the design of the trapezoidal velocity profile maximizes the needle insertion speed while meeting the constraints of acceleration and deceleration, ensuring the smoothness and safety of path execution. By rationally planning the ratio of acceleration, uniform speed and deceleration segments, the stability of the puncture process is ensured, and errors caused by drastic speed changes are avoided. By encoding the trapezoidal velocity profile and puncture time through a time position lookup table, this information can be quickly imported into the real-time controller, ensuring the smoothness and efficiency of the puncture operation. The introduction of the octree structure transforms the originally complex three-dimensional spatial model into a lightweight structure, simplifies data processing and storage, and effectively compresses the data volume. While reducing the computational and storage burden, it can still retain key spatial information, allowing the spatial regions of different anatomical structures to be efficiently decomposed and processed. The attribute mean and merging threshold within each node control the balance between data accuracy and efficiency, ensuring that the model has higher efficiency in the calculation process while maintaining sufficient accuracy.

[0081] The second embodiment of the present invention provides a planning and navigation system for a puncture robot, which includes: a dual-channel convolutional neural network registration module for registering preoperative high-resolution images and ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multimodal fusion image data. ; Semantic enhancement 3D model building module, used to transform the multimodal fusion image data Perform topological structure analysis and construct a semantically enhanced three-dimensional model; a global path planning module is used to perform global path planning based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning; a navigation data encapsulation module is used to generate navigation parameters and a lightweight model that can be imported into the navigation system based on the results of global path planning, and output preoperative navigation data.

[0082] A third embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the planning and navigation method for the puncture robot as described above is implemented.

[0083] A fourth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the planning and navigation method for the puncture robot as described above is implemented.

[0084] The computer program product for the planning and navigation method and device for a puncture robot provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0085] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, it can execute the above-mentioned planning and navigation method for the puncture robot, thereby improving the accuracy and stability of the alignment, effectively solving the problem of multimodal image data fusion, and realizing accurate preoperative navigation data output.

[0086] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A planning and navigation method for a puncture robot, characterized in that: include: The preoperative high-resolution images and ultrasound simulation data are registered through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multimodal fusion image data. Specifically, it includes converting preoperative high-resolution imaging data into three-dimensional , converting ultrasound simulation data into three-dimensional , respectively, perform isotropic resampling on the preoperative high-resolution image 3D data and ultrasound simulation 3D data to unify the resolution and obtain the preoperative high-resolution image space standardized data Unified data with ultrasound simulation space , design a dual-channel convolutional neural network to unify the ultrasound simulation data The grayscale distribution is mapped to the preoperative high-resolution image space normalized data , perform inter-modal contrast alignment to obtain ultrasound simulation space standardized data , the preoperative high-resolution image space normalized data Perform elastic transformation to generate preoperative high-resolution registration images , where tri is the trilinear interpolation kernel, for The neighborhood voxels of , where is the target voxel spacing, is a three-dimensional dense deformation field, and the preoperative high-resolution registration image Spatially normalized data with the ultrasound simulation Weighted fusion to generate multimodal fusion image data ,in, and is the weight, and the weight value is adaptively adjusted according to the local signal-to-noise ratio; The multimodal fusion image data Perform topological structure analysis and construct semantically enhanced 3D models, including inputting multimodal fusion image data , perform voxel-level semantic segmentation and output voxel-level semantic segmentation label map , the semantic segmentation label map As input, a topological feature map is constructed, the topological feature map is fused with the segmentation result, and a graph convolutional network is constructed to assign a semantic attribute vector to each voxel x. , design node The eigenvector of , graph convolutional layer ,in, For the anatomical category, is the distance to the nearest blood vessel, is the estimated value of tissue elastic modulus, is the geometric feature, is the feature matrix of the l-th layer node, is the degree matrix, is the learnable parameter matrix, is the ReLU activation function, and the output of the graph convolutional network is the enhanced node features , the node features and the semantic segmentation label map As input, for each voxel x, the semantic attributes are fused with the graph features of the nearest node , based on the marching cubes algorithm, isosurfaces are extracted and enhanced 3D models with semantic labels are generated; Based on the semantically enhanced 3D model, global path planning is performed through obstacle space encoding and reinforcement learning pre-planning; Based on the results of global path planning, navigation parameters and a lightweight model that can be imported into the navigation system are generated, and preoperative navigation data is output.

2. The planning and navigation method for a puncture robot according to claim 1, characterized in that: The dual-channel convolutional neural network includes: Preoperative image encoder branch and ultrasound image encoder branch, the preoperative high-resolution image space normalization data Input the preoperative image encoder branch to extract preoperative image features , the ultrasound simulation spatial normalization data Input the ultrasound image encoder branch to extract the ultrasound image features ; Cross-modal attention fusion module, for each level of preoperative image features and ultrasound imaging features , calculate the preoperative image channel weights respectively and ultrasound image channel weights , where GAP is global average pooling, σ is Sigmoid function, W is a learnable weight matrix, and the preoperative image features are and the ultrasound imaging features Perform weighted feature splicing to obtain dynamic fusion multimodal features ; Deformation field prediction decoder, which dynamically fuses multimodal features Upsample and output a three-dimensional dense deformation field , the preoperative high-resolution imaging spatial normalization data In the three-dimensional dense deformation field Perform elastic transformation.

3. The planning and navigation method for a puncture robot according to claim 1, characterized in that: The global path planning based on the semantically enhanced three-dimensional model is performed through obstacle space encoding and reinforcement learning pre-planning, including: Taking the semantically enhanced 3D model as input, the risk value of each voxel x is calculated according to the anatomical category and adjacent structures to which each voxel x belongs. , in, is the Euclidean distance from voxel x to the nearest vessel, is the Euclidean distance from voxel x to the nearest bone; Divide the 3D space into a voxel grid , each voxel stores the risk value , the probability of generating occupied maps , complete the obstacle space coding; Occupy the map based on the probability , set the starting point and target point , at the starting point and the target point Generate and output multiple candidate rough paths ,Each path consists of a sequence of key waypoints; Set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path through reinforcement learning ; The optimal puncture path Path smoothing is performed using B-spline basis functions, and continuous B-spline curves are output. ,in, is the B-spline basis function, As the control point, complete the global path planning.

4. The planning and navigation method for a puncture robot according to claim 3, characterized in that: The settings are used to locally adjust the state space and action space of the layer, set the reward function, and generate the optimal puncture path through reinforcement learning. include: Constructing a state space including storing risk values for the voxels in the obstacle space encoding , tissue elastic modulus with real-time ultrasound feedback and the global rough path reference curvature ; Define the action space, including the robot pitch angle adjustment parameters , yaw angle adjustment parameters , Needle insertion step length , if the current position risk value , then limit the needle step length If it is near bones , then the pitch angle adjustment parameter and the yaw angle adjustment parameter The adjustment range is halved; Set a multi-objective reward function, take the state space and the action space as inputs of the multi-objective reward function, and the reward item includes a target proximity reward , safety rewards , smoothing penalty and tissue damage penalty , output state vector, generate optimal puncture path .

5. The planning and navigation method for a puncture robot according to claim 4, characterized in that: The generation of navigation parameters and a lightweight model that can be imported into the navigation system based on the results of the global path planning, and the output of preoperative navigation data include: Design trapezoidal velocity profile , maximize the needle velocity while satisfying the acceleration constraint, ,in, is the arc length of the continuous B-spline curve, is the maximum permissible speed, is the arc length of the acceleration segment, is the maximum acceleration, is the total arc length, is the starting arc length of the deceleration segment; Calculate total puncture time ; The trapezoidal velocity profile and the total puncture time Encoded as a time-position lookup table and input to the real-time controller; Convert the semantically enhanced 3D model into a sparse octree structure, define node merging rules, , get the octree model ,in, is the spatial area corresponding to the octree node, is the mean value of the attribute within the node, is the attribute merging threshold; The trapezoidal velocity profile The octree model is taken as input and encapsulated as a MATLAB navigation parameter package.

6. A planning and navigation system for a puncture robot, characterized in that: include: The dual-channel convolutional neural network registration module is used to register preoperative high-resolution images with ultrasound simulation data through a dual-channel convolutional neural network, perform high-precision spatial alignment, and generate multimodal fusion image data. The operations performed by the dual-channel convolutional neural network registration module include converting the preoperative high-resolution image data into three-dimensional , converting ultrasound simulation data into three-dimensional , respectively, perform isotropic resampling on the preoperative high-resolution image 3D data and ultrasound simulation 3D data to unify the resolution and obtain the preoperative high-resolution image space standardized data Unified data with ultrasound simulation space , design a dual-channel convolutional neural network to unify the ultrasound simulation data The grayscale distribution is mapped to the preoperative high-resolution image space normalized data , perform inter-modal contrast alignment to obtain ultrasound simulation space standardized data , the preoperative high-resolution image space normalized data Perform elastic transformation to generate preoperative high-resolution registration images , where tri is the trilinear interpolation kernel, for The neighborhood voxels of , where is the target voxel spacing, is a three-dimensional dense deformation field, and the preoperative high-resolution registration image Spatially normalized data with the ultrasound simulation Weighted fusion to generate multimodal fusion image data ,in, and is the weight, and the weight value is adaptively adjusted according to the local signal-to-noise ratio; Semantic enhancement 3D model building module, used to transform the multimodal fusion image data Perform topological structure analysis and construct semantically enhanced 3D models. The operations performed by the semantically enhanced 3D model construction module include inputting multimodal fusion image data , perform voxel-level semantic segmentation and output voxel-level semantic segmentation label map , the semantic segmentation label map As input, a topological feature map is constructed, the topological feature map is fused with the segmentation result, and a graph convolutional network is constructed to assign a semantic attribute vector to each voxel x. , design node The eigenvector of , graph convolutional layer ,in, For the anatomical category, is the distance to the nearest blood vessel, is the estimated value of tissue elastic modulus, is the geometric feature, is the feature matrix of the l-th layer node, is the degree matrix, is the learnable parameter matrix, is the ReLU activation function, and the output of the graph convolutional network is the enhanced node features , the node features and the semantic segmentation label map As input, for each voxel x, the semantic attributes are fused with the graph features of the nearest node , based on the marching cubes algorithm, isosurfaces are extracted and enhanced 3D models with semantic labels are generated; A global path planning module, configured to perform global path planning based on the semantically enhanced 3D model through obstacle spatial encoding and reinforcement learning pre-planning; The navigation data encapsulation module is used to generate navigation parameters and a lightweight model that can be imported into the navigation system based on the results of global path planning, and output preoperative navigation data.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the planning and navigation method for a puncture robot according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the planning and navigation method for a puncture robot according to any one of claims 1 to 5 is implemented.

Images