Planning and navigation method and system for puncture robot

Abstract

The invention discloses a planning and navigation method and system for a puncture robot. The method comprises the steps that a preoperative high-resolution image and ultrasonic simulation data are registered through a two-channel convolutional neural network, high-precision space alignment is carried out, and multi-modal fusion body image data Ifuse are generated; carrying out topological structure analysis on the multi-modal fusion body image data Ifuse, and carrying out semantic enhancement three-dimensional model construction; based on the semantic enhancement three-dimensional model, performing global path planning through obstacle space coding and reinforcement learning pre-planning; according to the global path planning result, navigation parameters and a lightweight model capable of being imported into a navigation system are generated, and preoperative navigation data are output. The system comprises a two-channel convolutional neural network registration module, a semantic enhancement three-dimensional model construction module, a global path planning module and a navigation data encapsulation module. According to the invention, the precision and stability of registration are improved, the problem of multi-modal image data fusion can be effectively solved, and accurate preoperative navigation data output is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of puncture robot navigation, and particularly relates to a planning and navigation method and system for a puncture robot. Background Art

[0002] With the rapid development of medical imaging technology and computer science, vision-guided puncture robots have been widely used in minimally invasive surgery. Especially in the preoperative stage, how to accurately plan and locate the puncture path is a key issue in the current field of medical robots. Traditional puncture robot navigation methods mostly rely on a single image data source, such as CT or MRI images. However, due to problems such as resolution limitations, imaging modality differences, and tissue distortion during acquisition, these images are difficult to meet the high-precision and high-robustness navigation requirements.

[0003] In recent years, robot calibration methods based on binocular vision have gradually become a research hotspot due to their high efficiency and cost-effectiveness. Binocular vision can provide richer depth information through the spatial relationship between cameras, thereby supporting higher-precision three-dimensional space reconstruction and positioning. However, existing binocular vision calibration methods still have certain technical challenges in practical applications, mainly including: insufficient image registration accuracy, error accumulation during three-dimensional reconstruction, large image noise interference, and the problem of fusing different modality images.

[0004] Currently, for the preoperative stage, especially the accuracy of puncture robot positioning, the market demand is increasing day by day. Precise preoperative image registration and path planning can significantly improve the safety and efficiency of surgery, while reducing the operation time and postoperative recovery period. Therefore, developing a high-precision calibration method based on binocular vision that can achieve precise registration and effective fusion of preoperative image data is an urgent problem to be solved in the current field of medical robots.

[0005] The defects of the existing technical solutions are mainly reflected in the following aspects: (1) Insufficient accuracy. Traditional image registration methods often have problems such as low registration accuracy and poor algorithm stability. Especially in the case of low-resolution or noisy images, it is difficult to achieve high-precision registration results; (2) Difficult image fusion. The contrast, texture, and resolution differences between different modality images make it difficult for existing technologies to effectively eliminate information loss or errors during multi-modal data fusion; (3) Complicated calibration process. Current calibration methods usually require the cooperation of multiple sensor data, and the operation process is complex and vulnerable to external environmental influences, resulting in unstable calibration results; (4) Poor real-time performance. Existing vision-based puncture robot calibration methods usually have a large amount of calculation and poor real-time performance, making it difficult to provide real-time feedback during the operation. Summary of the Invention

[0006] The object of the present invention is to provide a planning and navigation method and system for a puncture robot, which is used to solve at least one of the above technical problems. By fusing high-resolution images and ultrasound simulation data and adopting a dual-channel convolutional neural network for accurate image registration and contrast alignment, the accuracy and stability of registration are improved. Through the construction of a semantically enhanced three-dimensional model and global path planning, the problem of multi-modal image data fusion can be effectively solved, and accurate preoperative navigation data output can be realized.

[0007] The embodiments of the present invention are implemented as follows:

[0008] A planning and navigation method for a puncture robot, which includes:

[0009] Register the preoperative high-resolution image and ultrasound simulation data through a dual-channel convolutional neural network, perform high-precision spatial alignment, and generate multi-modal fused body image data I fuse 。

[0010] Perform topological structure analysis on the multi-modal fused body image data I fuse and construct a semantically enhanced three-dimensional model.

[0011] Based on the semantically enhanced three-dimensional model, perform global path planning through obstacle space encoding and reinforcement learning pre-planning.

[0012] According to the result of the global path planning, generate navigation parameters and a lightweight model that can be imported into the navigation system, and output preoperative navigation data.

[0013] In a preferred embodiment of the present invention, in the above planning and navigation method for a puncture robot, the registering the preoperative high-resolution image and ultrasound simulation data through a dual-channel convolutional neural network, performing high-precision spatial alignment, and generating multi-modal fused body image data includes:

[0014] Three-dimension the preoperative high-resolution image data into Three-dimension the ultrasound simulation data into

[0015] Perform isotropic resampling on the three-dimensionalized data of the preoperative high-resolution image and the three-dimensionalized ultrasound simulation data respectively to unify the resolution, and obtain the preoperative high-resolution image space standardized data and the ultrasound simulation space unified data

[0016] Design a dual-channel convolutional neural network, and map the gray-scale distribution of the ultrasound simulation unified data to the preoperative high-resolution image space standardized data Perform inter-modal contrast alignment to obtain the ultrasound simulation space standardized data

[0017] Normalize the preoperative high-resolution image spatial data Perform elastic transformation to generate the preoperative high-resolution registered image where tri is the trilinear interpolation kernel and N(x) is the neighborhood voxels of x.

[0018] The preoperative high-resolution registered image and the ultrasound simulation spatial normalization data are weighted and fused to generate multimodal fusion volume image data where w pre and w us are weights, and the values of the weights are adaptively adjusted according to the local signal-to-noise ratio.

[0019] Its technical effect is as follows: The registration is performed using a dual-channel convolutional neural network, which fully excavates the deep features between two different modal images. Through the nonlinear mapping ability of the neural network, the accuracy of spatial alignment is effectively improved. The registration algorithm based on deep learning can automatically complete the spatial alignment of images and the contrast alignment between modalities. The preoperative high-resolution image and ultrasound simulation data are fused into multimodal fusion volume image data, maximizing the advantages of different modal data and providing richer anatomical structure information. The high-precision spatial alignment and modal contrast alignment provide high-quality input data for subsequent topological structure analysis, three-dimensional model construction, and path planning.

[0020] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the dual-channel convolutional neural network includes:[[]]

[0021] A preoperative image encoder branch and an ultrasound image encoder branch. The preoperative high-resolution image spatial normalization data is input into the preoperative image encoder branch to extract preoperative image features The ultrasound simulation spatial normalization data is input into the ultrasound image encoder branch to extract ultrasound image features

[0022] A cross-modal attention fusion module. For the preoperative image features and the ultrasound image features at each level, calculate the preoperative image channel weights and the ultrasound image channel weights respectively, where GAP is global average pooling, σ is the Sigmoid function, and W is a learnable weight matrix. The preoperative image features and the ultrasound image features are weighted feature concatenated to obtain dynamically fused multimodal features

[0023] The deformation field prediction decoder upsamples the dynamically fused multi-modal features and outputs a three-dimensional dense deformation field The pre-operative high-resolution image spatial normalization data is elastically transformed under the three-dimensional dense deformation field φ.

[0024] Its technical effect is that: the cross-modal attention fusion module can dynamically adjust the importance of each modal feature by calculating the channel weights for the pre-operative image features and ultrasound image features at each level, and weighting different channels according to global average pooling (GAP) and the Sigmoid function, thus ensuring the optimal expression of the fused features. The deformation field prediction decoder upsamples the dynamically fused multi-modal features to generate a three-dimensional dense deformation field. In this way, the network can predict the deformation law of the image in space and accurately achieve the spatial registration between the pre-operative image and the ultrasound simulation image.

[0025] In a preferred embodiment of the present invention, in the above method for planning and navigation of a puncture robot, the step of performing topological structure analysis on the multi-modal fusion volume image data I fuse and constructing a semantic enhanced three-dimensional model includes:

[0026] Input the multi-modal fusion volume image data I fuse , perform voxel-level semantic segmentation, and output a voxel-level semantic segmentation label map L∈{0,1,...,C} H×W×D .

[0027] Using the semantic segmentation label map L as input, construct a topological feature map G=(V,E,W), where V is the set of nodes, and each node v i represents an anatomical structure, E is the set of edges, representing the spatial adjacency relationship of anatomical structures, and W is the adjacency matrix, and the element w ij is the edge weight element.

[0028] Fuse the topological feature map with the segmentation result, construct a graph convolutional network, and assign a semantic attribute vector a(x)=[a 1 (x),a 2 (x),a 3 (x)] to each voxel x, and design the feature vector of node v i The graph convolutional layer H (l +1) -1 / 2 =σ(D -1 / 2 WD -1 / 2 H (l) Θ (l) ), where a 1(x) is the anatomical category to which it belongs, a 2 (x) is the distance to the nearest blood vessel, a 3 (x) is the estimated value of tissue elastic modulus, f geo (v i ) is the geometric feature, H (l) is the node feature matrix of the l-th layer, D is the degree matrix, Θ (l) is the learnable parameter matrix, σ is the ReLU activation function, and the output of the graph convolutional network is the enhanced node feature H (L) .

[0029] Taking the node feature H (L) and the semantic segmentation label map L as inputs, for each voxel x, fuse the semantic attributes and the graph features of the nearest node Extract the isosurface based on the marching cubes algorithm to generate an enhanced 3D model with semantic labels.

[0030] Its technical effect is as follows: Voxel-level semantic segmentation can accurately identify and distinguish different types of tissues or anatomical regions, such as tumors, blood vessel branches, bones, etc., by classifying the anatomical structures of each voxel. Using the weighted cross-entropy loss function and the smooth term weight effectively balances the segmentation errors of different classes, reduces the situation of unclear boundaries or ambiguous classes, and improves the segmentation accuracy. By constructing the topological feature map of the semantic segmentation results, each anatomical structure (such as a tumor, blood vessel branch, etc.) is represented as a node, and the adjacency relationship between nodes (i.e., the spatial connection between anatomical structures) is described as an edge set. The setting of edge weights can precisely capture the spatial distance and connection relationship between anatomical structures by controlling the attenuation of spatial correlation. Especially in complex anatomical regions, such as the blood vessel network or bone structure near a tumor, it can carefully reflect the relationship between each part. When constructing the graph convolutional network, each node (anatomical structure) is assigned a semantic attribute vector, which contains not only anatomical category information but also structural information such as tissue elastic modulus, geometric features (such as volume, surface area), and the distance to the nearest blood vessel. The introduction of this information greatly enhances the anatomical perception ability of the model, making the generated 3D model more accurately reflect the physical properties and mutual relationships of different tissues. The fusion of node features and the semantic segmentation label map further enhances the model's semantic understanding of anatomical regions, enabling each voxel to not only have information about its spatial position but also rich semantic features, such as whether it is a tumor, blood vessel, or other important anatomical structures. Based on these fused features, the enhanced 3D model with semantic labels generated by extracting the isosurface using the marching cubes algorithm can carefully and accurately express each anatomical structure and its spatial relationship.

[0031] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, based on the semantic-enhanced three-dimensional model, through obstacle space encoding and reinforcement learning pre-planning, global path planning is performed, including:

[0032] Taking the semantic-enhanced three-dimensional model as input, calculate the risk value of each voxel x according to the anatomical category and adjacent structures to which each voxel x belongs

[0033] where d vessel (x) is the Euclidean distance from voxel x to the nearest blood vessel, and d bone (x) is the Euclidean distance from voxel x to the nearest bone.

[0034] Divide the three-dimensional space into voxel grids Each voxel stores the risk value r(x), generating a probability occupancy map Complete obstacle space encoding.

[0035] Based on the probability occupancy map P occ , set the starting point p start and the target point p goal , generate and output multiple candidate rough paths {P start} between the starting point p goal and the target point p k , and each path consists of a sequence of key waypoints.

[0036] Set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path P * ={p 1 , p 2 ,..., p N} through reinforcement learning.

[0037] Smooth the optimal path P * through the B-spline basis function, and output a continuous B-spline curve u∈[0,1], where B i,k (u) is the B-spline basis function, and P i is the control point, completing the global path planning.

[0038] Its technical effects are as follows: By calculating risk values based on the anatomical categories and adjacent structures of each voxel, accurate risk assessment is achieved. This path planning not only takes into account the spatial geometric structure but also includes the functional attributes of tissues and organs, such as the risk information of blood vessels and bones, and can avoid key areas, improving the safety of the puncture process. By dividing the three-dimensional space into a voxel grid, each voxel stores its risk value and generates a probability occupancy map, clearly marking the obstacle areas (such as blood vessels, bones, etc.) in the space. According to the layout of the probability occupancy map, dangerous areas are identified and avoided, ensuring that the path planning system can fully consider various risks, optimize the path selection, and avoid collisions with important structures during the puncture process. The global path planning based on the semantic-enhanced three-dimensional model not only considers the geometric shape of the space (such as the specific positions of blood vessels and bones) but also dynamically optimizes according to the risk information of the anatomical structure. When the puncture robot performs tasks, it can automatically and intelligently adjust the path to avoid potential dangerous areas, maximizing the safety and success rate of the puncture.

[0039] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the state space and action space for local adjustment layers are set, a reward function is set, and through reinforcement learning, an optimal puncture path P is generated * including:

[0040] Construct a state space, including the voxel storage risk value r(x) in the obstacle space encoding, the tissue elastic modulus E(x) of real-time ultrasound feedback, and the global coarse path reference curvature κ global .

[0041] Define an action space, including the robot pitch angle adjustment parameter Δθ pitch ∈[-5°, 5°], the yaw angle adjustment parameter Δθ yaw ∈[-5°, 5°], the needle insertion step Δd ∈ [1mm, 5mm]. If the risk value r(p t ) > 0.7, then the needle insertion step Δd is restricted to Δd ≤ 2mm. If the distance to the adjacent bone d bone < 5mm, then the adjustment amplitude of the pitch angle adjustment parameter Δθ pitch and the yaw angle adjustment parameter Δθ yaw is halved.

[0042] Set a multi-objective reward function, take the state space and the action space as the input of the multi-objective reward function, and the reward items include the target approach reward R goal = 10·exp(-Δd / 50), the safety reward R safe = ∑ r(x)＜0.3 (1 - r(x)), the smoothing penalty R smooth = -20·(||Δθpitch || 2 +||Δθ yaw || 2 +||Δκ global || 2 ) and tissue damage penalty R damage = -0.1·E(x)·Δd, output the state vector, generate the optimal puncture path P * = {p 1 , p 2 ,..., p N}.

[0043] Its technical effect lies in that: by introducing the tissue elastic modulus and the global coarse path reference curvature of real-time ultrasonic feedback, the construction of the state space can reflect the soft and hard characteristics (such as elastic modulus) of the tissue and the geometric characteristics (such as curvature) of the path in real time, and the local path adjustment is more flexible and accurate, and can dynamically adapt to the characteristics of different tissues and the adjustment requirements of the puncture path. Through the multi-objective reward function, multiple objectives are comprehensively considered, including target approach reward, safety reward, smoothing penalty and tissue damage penalty, etc., ensuring that in the process of path optimization, the robot not only has to consider the proximity of the path to the target point, but also ensures the smoothness of the path and the safety during the puncture process. The target approach reward guides the robot to approach the target position more quickly and accurately; the safety reward encourages the robot to avoid high-risk areas, such as blood vessels and bones, etc.; the smoothing penalty reduces the sharp turns in the path to ensure the smoothness of the path; the tissue damage penalty ensures that the path will not cause damage to sensitive tissues, thus balancing various requirements and generating a safe and efficient optimal path.

[0044] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, according to the result of the global path planning, generating navigation parameters and a lightweight model that can be imported into the navigation system, the preoperative navigation data output includes:

[0045] Design a trapezoidal velocity profile v(s), maximizing the needle insertion speed while satisfying the acceleration constraint, where s is the arc length of the continuous B-spline curve, v max is the maximum allowable speed, is the arc length of the acceleration section, a max is the maximum acceleration value, s total is the total arc length, s d = s total - s a is the starting point of the deceleration section.

[0046] Calculate the total puncture time

[0047] Encode the trapezoidal velocity profile v(s) and the total puncture time T into a time-position lookup table and input it into the real-time controller.

[0048] Convert the semantic-enhanced three-dimensional model into a sparse octree structure and define node merging rules. Obtain the octree model T, where O i is the spatial region corresponding to the octree node. is the mean value of the attributes within the node, and δ is the attribute merging threshold.

[0049] Take the trapezoidal velocity profile v(s) and the octree model as inputs and encapsulate them into a MATLAB navigation parameter package.

[0050] Its technical effects are as follows: The design of the trapezoidal velocity profile enables the needle insertion process to maximize the needle insertion speed while satisfying the constraints of acceleration and deceleration, ensuring the smoothness and safety of path execution. By reasonably planning the proportions of the acceleration section, constant-speed section, and deceleration section, the stability during the puncture process is ensured, and errors caused by drastic speed changes are avoided. Encoding the trapezoidal velocity profile and puncture time through the time-position lookup table can quickly import this information into the real-time controller, ensuring the smoothness and efficiency of the puncture operation. The introduction of the octree structure transforms the originally complex three-dimensional space model into a lightweight structure, simplifies data processing and storage, effectively compresses the data volume, reduces the computational and storage burdens while still retaining key spatial information, enabling the spatial regions of different anatomical structures to be efficiently decomposed and processed. The mean value of the attributes within each node and the merging threshold control the balance between data accuracy and efficiency, ensuring that the model has higher efficiency during the calculation process while maintaining sufficient accuracy.

[0051] A planning and navigation system for a puncture robot, which includes:

[0052] A dual-channel convolutional neural network registration module for registering the preoperative high-resolution image and the ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multi-modal fusion volume image data I fuse .

[0053] A semantic-enhanced three-dimensional model construction module for performing topological structure analysis on the multi-modal fusion volume image data I fuse and constructing a semantic-enhanced three-dimensional model.

[0054] A global path planning module for performing global path planning based on the semantic-enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning.

[0055] A navigation data encapsulation module, which is used to generate navigation parameters and a lightweight model that can be imported into the navigation system according to the results of global path planning, and output preoperative navigation data.

[0056] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the planning and navigation method for a puncture robot as described above.

[0057] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the planning and navigation method for a puncture robot as described above.

[0058] The beneficial effects of the embodiments of the present invention are:

[0059] By registering preoperative high-resolution images and ultrasound simulation data through a dual-channel convolutional neural network, the present invention can achieve accurate spatial alignment. Combining the advantages of preoperative images and real-time ultrasound data, it provides accurate anatomical structure information for the puncture robot, thereby greatly improving the accuracy and safety of the puncture operation. Especially during the puncture process around important anatomical structures such as tumors and blood vessels, it can effectively avoid damage.

[0060] By analyzing the topological structure of multi-modal fusion volume image data and constructing a semantic-enhanced three-dimensional model, the present invention can accurately reflect the anatomical structure and its spatial relationship in the body. It not only improves the depth of semantic information of the model, but also effectively improves the positioning accuracy of the puncture robot, providing more detailed anatomical guidance. When constructing the topological feature map, enhanced feature information is given to the nodes through a graph convolutional network, further improving the accuracy and adaptability of the model, helping the puncture robot to more accurately identify and avoid potential obstacles in a complex environment, and improving the safety of the puncture process.

[0061] By obstacle space encoding and reinforcement learning pre-planning, the present invention can intelligently generate the optimal puncture path according to the actual situation. The introduction of reinforcement learning enables the system to automatically adjust the path in a complex environment, avoid dangerous areas, and optimize the operation path during the puncture process to ensure the efficiency and safety of the puncture. By setting the state space, action space, and multi-objective reward function, the path is dynamically adjusted according to real-time feedback, and the local path is optimized in combination with actual feedback, further improving the accuracy and stability of the puncture process.

[0062] By designing a trapezoidal velocity profile, the present invention maximizes the needle insertion speed while ensuring that the acceleration and deceleration comply with physical constraints, thereby improving the efficiency and smoothness of the puncture, effectively reducing errors caused by excessive or insufficient speed, and enhancing the stability of the puncture operation. By calculating the total puncture time and combining the trapezoidal velocity profile data input by the real-time controller, the puncture time is accurately controlled to avoid the influence of too long or too short puncture time on the surgical effect.

[0063] The present invention converts the semantic-enhanced three-dimensional model into a sparse octree structure, significantly reducing the complexity of data processing and enabling rapid response and execution of path planning tasks during real-time control. The introduction of the octree not only saves storage space but also greatly improves the computational efficiency. Especially in a dynamic environment, it helps to reduce latency and improve the response speed of the puncture robot. Based on the real-time ultrasound feedback information, the path and speed are dynamically adjusted to ensure that new situations can be adapted at any time during the operation, providing an efficient and safe puncture operation.

[0064] The present invention enhances the recognition ability of different tissues and improves the adaptability in complex environments by fusing ultrasound simulation data and preoperative high-resolution images, utilizing the complementary information from different image sources, enabling the robot to perform precise puncture operations on different patient individuals. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0066] Figure 1 It is a flowchart of the planning and navigation method for the puncture robot of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0068] Please refer to Figure 1, a first embodiment of the present invention provides a planning and navigation method for a puncture robot, which includes: registering a preoperative high-resolution image and ultrasound simulation data through a dual-channel convolutional neural network, performing high-precision spatial alignment, and generating multi-modal fusion volume image data I fuse ; performing topological structure analysis on the multi-modal fusion volume image data I fuse to construct a semantic-enhanced three-dimensional model; based on the semantic-enhanced three-dimensional model, performing global path planning through obstacle space encoding and reinforcement learning pre-planning; according to the result of the global path planning, generating navigation parameters and a lightweight model that can be imported into the navigation system, and outputting preoperative navigation data.

[0069] In a preferred embodiment of the present invention, in the above planning and navigation method for a puncture robot, the registering the preoperative high-resolution image and ultrasound simulation data through a dual-channel convolutional neural network, performing high-precision spatial alignment, and generating multi-modal fusion volume image data includes: three-dimensionally converting the preoperative high-resolution image data into three-dimensionally converting the ultrasound simulation data into respectively performing isotropic resampling on the three-dimensionalized preoperative high-resolution image data and the three-dimensionalized ultrasound simulation data to unify the resolution, and obtaining the preoperative high-resolution image spatially standardized data and the ultrasound simulation spatially unified data where the steps of isotropic resampling are where Δ is the target voxel spacing, and the sinc function is used for anti-aliasing; designing a dual-channel convolutional neural network to map the gray-scale distribution of the ultrasound simulation unified data to the preoperative high-resolution image spatially standardized data for inter-modal contrast alignment to obtain the ultrasound simulation spatially standardized data performing elastic transformation on the preoperative high-resolution image spatially standardized data to generate a preoperative high-resolution registered image where tri is the trilinear interpolation kernel and N(x) is the neighborhood voxels of x; weighting and fusing the preoperative high-resolution registered image with the ultrasound simulation spatially standardized data to generate multi-modal fusion volume image data where w pre and w us are weights, and the values of the weights are adaptively adjusted according to the local signal-to-noise ratio.

[0070] Its technical effects are as follows: The dual-channel convolutional neural network is used for registration, fully mining the deep features between two different modality images. Through the non-linear mapping ability of the neural network, the accuracy of spatial alignment is effectively improved. The registration algorithm based on deep learning can automatically complete the spatial alignment of images and the contrast alignment between modalities. The preoperative high-resolution images and ultrasound simulation data are fused into multi-modal fused volume image data, maximizing the advantages of different modality data and providing richer anatomical structure information. The high-precision spatial alignment and modality contrast alignment provide high-quality input data for subsequent topological structure analysis, three-dimensional model construction, and path planning.

[0071] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the dual-channel convolutional neural network includes: a preoperative image encoder branch and an ultrasound image encoder branch. The preoperative high-resolution image spatial normalization data is input into the preoperative image encoder branch to extract preoperative image features The ultrasound simulation spatial normalization data is input into the ultrasound image encoder branch to extract ultrasound image features A cross-modal attention fusion module calculates the preoperative image channel weights and the ultrasound image channel weights respectively for the preoperative image features and the ultrasound image features at each level. where GAP is global average pooling, σ is the Sigmoid function, and W is a learnable weight matrix. The preoperative image features and the ultrasound image features are subjected to weighted feature splicing to obtain dynamically fused multi-modal features A deformation field prediction decoder upsamples the dynamically fused multi-modal features and outputs a three-dimensional dense deformation field The preoperative high-resolution image spatial normalization data is elastically transformed under the three-dimensional dense deformation field φ.

[0072] Its technical effects are as follows: The cross-modal attention fusion module can dynamically adjust the importance of each modality feature by calculating the channel weights for the preoperative image features and the ultrasound image features at each level and weighting different channels according to global average pooling (GAP) and the Sigmoid function, thus ensuring the optimal expression of the fusion features. The deformation field prediction decoder upsamples the dynamically fused multi-modal features to generate a three-dimensional dense deformation field. In this way, the network can predict the deformation law of the images in space and accurately achieve the spatial registration between the preoperative images and the ultrasound simulation images.

[0073] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the multi-modal fusion body image data I fuse is subjected to topological structure analysis, and semantic-enhanced three-dimensional model construction includes: inputting the multi-modal fusion body image data I fuse , performing voxel-level semantic segmentation, and outputting a voxel-level semantic segmentation label map L∈{0,1,...,C} H×W×D , where the segmentation loss function is C is the number of categories, y c (x) is the one-hot encoding of the true category c of voxel x, p c (x) is the category probability predicted by the network, w c is the category weight, N(x) is the 6-neighborhood of voxel x, λ = 0.1 is the weight of the smoothing term, represents weighted cross-entropy, represents boundary smoothing constraint; taking the semantic segmentation label map L as the input, constructing a topological feature map G=(V, E, W), where V is the set of nodes, and each node v i represents an anatomical structure (such as a tumor, blood vessel branch), E is the set of edges, representing the spatial adjacency relationship of anatomical structures, W is the adjacency matrix, and the element w ij is the edge weight element (specifically, by extracting the centroid coordinates for each anatomical category, such as the tumor centroid performing key point extraction, setting up edge connection rules, if there is a connected path between two key points and the distance d < d max , taking d max = 20mm, then establishing an edge e ij , and the edge weight σ = 5mm, controlling the attenuation of spatial correlation, and establishing the topological feature map G=(V, E, W)); fusing the topological feature map with the segmentation result, constructing a graph convolutional network, and assigning a semantic attribute vector a(x)=[a 1 (x), a 2 (x), a 3 (x)] to each voxel x, designing the feature vector i of node v graph convolutional layer H (l+1) = σ(D -1 / 2 WD -1 / 2 H (l) Θ (l) ), where a 1 (x) is the anatomical category to which it belongs, a 2 (x) is the distance to the nearest blood vessel, a 3 (x) is the estimated tissue elastic modulus value, f geo (v i ) is the geometric feature (volume, surface area), H(l) is the node feature matrix of the l-th layer, D is the degree matrix, Θ (l) is the learnable parameter matrix, σ is the ReLU activation function, and the output of the graph convolutional network is the enhanced node feature H (L) ; Taking the node feature H (L) and the semantic segmentation label map L as inputs, for each voxel x, fusing the semantic attributes and the graph feature h of the nearest node i (L) , extracting the isosurface based on the marching cubes algorithm to generate an enhanced three-dimensional model with semantic labels.

[0074] Its technical effect is as follows: Voxel-level semantic segmentation can accurately identify and distinguish different types of tissues or anatomical regions, such as tumors, blood vessel branches, bones, etc., by classifying the anatomical structure of each voxel. Using the weighted cross-entropy loss function and the smooth term weight effectively balances the segmentation errors of different classes, reduces the situation of unclear boundaries or ambiguous classes, and improves the segmentation accuracy. By constructing the topological feature map of the semantic segmentation results, each anatomical structure (such as a tumor, a blood vessel branch, etc.) is represented as a node, and the adjacency relationship between nodes (i.e., the spatial connection between anatomical structures) is described as an edge set. The setting of the edge weight controls the decay of spatial correlation, enabling the topological feature map to accurately capture the spatial distance and connection relationship between anatomical structures. Especially in complex anatomical regions, such as the blood vessel network or bone structure near a tumor, it can carefully reflect the relationship between each part. When constructing the graph convolutional network, each node (anatomical structure) is assigned a semantic attribute vector, which contains not only anatomical category information but also structural information such as tissue elastic modulus, geometric features (such as volume, surface area), and the distance to the nearest blood vessel. The introduction of this information greatly enhances the anatomical perception ability of the model, enabling the generated three-dimensional model to more accurately reflect the physical properties and mutual relationships of different tissues. The fusion of node features and the semantic segmentation label map further enhances the model's semantic understanding of anatomical regions, enabling each voxel to not only have information about its spatial position but also rich semantic characteristics, such as whether it is a tumor, a blood vessel, or other important anatomical structures. Based on these fused features, the enhanced three-dimensional model with semantic labels generated by extracting the isosurface using the marching cubes algorithm can express each anatomical structure and its spatial relationship carefully and accurately.

[0075] In a preferred embodiment of the present invention, in the above method for planning and navigation of a puncture robot, based on the semantic-enhanced three-dimensional model, through obstacle space encoding and reinforcement learning pre-planning, global path planning is performed, including: taking the semantic-enhanced three-dimensional model as an input, and calculating the risk value of each voxel x according to the anatomical category and adjacent structures to which each voxel x belongs where dvessel The Euclidean distance from voxel x to the nearest blood vessel is d(x). bone The Euclidean distance from voxel x to the nearest bone is d(x); the three-dimensional space is divided into a voxel grid Each voxel stores a risk value r(x) to generate a probability occupancy map Complete the obstacle space encoding; based on the probability occupancy map P occ , set the starting point p start and the target point p goal , at the starting point p start and the target point p goal generate multiple candidate rough paths {P k} between them, each path consisting of a sequence of key waypoints; set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path P * ={p 1 ,p 2 ,...,p N}; smooth the optimal path P * through the B-spline basis function to output a continuous B-spline curve where B i,k (u) is the B-spline basis function, P i is the control point, and complete the global path planning.

[0076] Its technical effect is as follows: by calculating the risk value according to the anatomical category and adjacent structures of each voxel, accurate risk assessment is realized. This path planning not only considers the spatial geometric structure but also includes the functional attributes of tissues and organs, such as the risk information of blood vessels and bones, and can avoid key areas to improve the safety of the puncture process. By dividing the three-dimensional space into a voxel grid, each voxel stores its risk value and generates a probability occupancy map, clearly marking the obstacle areas (such as blood vessels, bones, etc.) in the space, and according to the layout of the probability occupancy map, identifying and avoiding dangerous areas to ensure that the path planning system can fully consider various risks, optimize the path selection, and avoid collisions with important structures during the puncture process. The global path planning based on the semantic-enhanced three-dimensional model not only considers the geometric shape of the space (such as the specific positions of blood vessels and bones) but also dynamically optimizes according to the risk information of the anatomical structure. When the puncture robot performs tasks, it can automatically and intelligently adjust the path to avoid potential dangerous areas and maximize the safety and success rate of the puncture.

[0077] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, the state space and action space for the local adjustment layer are set, the reward function is set, and the optimal puncture path P *Including: constructing a state space, including the voxel storage risk value r(x) in the obstacle space encoding, the tissue elastic modulus E(x) of real-time ultrasonic feedback, and the global coarse path reference curvature κ global ; defining an action space, including the robot pitch angle adjustment parameter Δθ pitch ∈[-5°, 5°], the yaw angle adjustment parameter Δθ yaw ∈[-5°, 5°], the needle insertion step Δd ∈ [1mm, 5mm]. If the risk value r(p t ) > 0.7, then the needle insertion step Δd is restricted to Δd ≤ 2mm. If the distance to the adjacent bone d bone < 5mm, then the adjustment amplitude of the pitch angle adjustment parameter Δθ pitch and the yaw angle adjustment parameter Δθ yaw is halved; setting a multi-objective reward function, taking the state space and the action space as the inputs of the multi-objective reward function. The reward terms include the target approach reward R goal = 10·exp(-Δd / 50), the safety reward R safe = ∑ r(x)＜0.3 (1 - r(x)), the smoothing penalty R smooth = -20·(||Δθ pitch || 2 + ||Δθ yaw || 2 + ||Δκ global || 2 ) and the tissue damage penalty R damage = -0.1·E(x)·Δd, outputting a state vector, and generating an optimal puncture path P * = {p 1 , p 2 ,..., p N}.

[0078] Its technical effects are as follows: By introducing the tissue elastic modulus with real-time ultrasound feedback and the global coarse path reference curvature, the construction of the state space can reflect the soft and hard characteristics of the tissue (such as the elastic modulus) and the geometric characteristics of the path (such as the curvature) in real time. The local path adjustment is more flexible and precise, and can dynamically adapt to the characteristics of different tissues and the adjustment requirements of the puncture path. Through the multi-objective reward function, multiple objectives are comprehensively considered, including the target approach reward, safety reward, smoothness penalty, and tissue damage penalty, etc., ensuring that during the path optimization process, the robot not only considers the proximity of the path to the target point, but also ensures the smoothness of the path and the safety during the puncture process. The target approach reward guides the robot to approach the target position more quickly and precisely; the safety reward encourages the robot to avoid high-risk areas, such as blood vessels and bones, etc.; the smoothness penalty reduces sharp turns in the path to ensure the smoothness of the path; the tissue damage penalty ensures that the path does not cause damage to sensitive tissues, thus balancing various requirements and generating a safe and efficient optimal path.

[0079] In a preferred embodiment of the present invention, in the above-mentioned planning and navigation method for a puncture robot, according to the result of the global path planning, navigation parameters and a lightweight model that can be imported into the navigation system are generated, and the preoperative navigation data output includes: designing a trapezoidal velocity profile v(s) to maximize the needle insertion speed while satisfying the acceleration constraint, where s is the arc length of the continuous B-spline curve, v max is the maximum allowable speed, is the arc length of the acceleration section, a max is the maximum acceleration value, s total is the total arc length, s d = s total - s a is the starting point of the deceleration section; calculate the total puncture time Encode the trapezoidal velocity profile v(s) and the total puncture time T into a time-position lookup table and input it to the real-time controller; convert the semantic-enhanced three-dimensional model into a sparse octree structure, define the node merging rule, to obtain the octree model T, where O i is the spatial region corresponding to the octree node, is the mean value of the attributes within the node, and δ is the attribute merging threshold; package the trapezoidal velocity profile v(s) and the octree model as inputs into a MATLAB navigation parameter package.

[0080] Its technical effects are as follows: The design of the trapezoidal velocity profile enables the needle insertion process to maximize the needle insertion speed while satisfying the constraints of acceleration and deceleration, ensuring the smoothness and safety of path execution. By reasonably planning the proportions of the acceleration section, constant velocity section, and deceleration section, the stability during the puncture process is ensured, and errors caused by drastic speed changes are avoided. By encoding the trapezoidal velocity profile and puncture time through a time-position lookup table, this information can be quickly imported into the real-time controller, guaranteeing the smoothness and efficiency of the puncture operation. The introduction of the octree structure transforms the originally complex three-dimensional space model into a lightweight structure, simplifies data processing and storage, effectively compresses the data volume, reduces the computational and storage burdens while still retaining key spatial information, enabling the spatial regions of different anatomical structures to be efficiently decomposed and processed. The attribute mean value and merging threshold within each node control the balance between data precision and efficiency, ensuring that the model has higher efficiency during the calculation process while maintaining sufficient precision.

[0081] The second embodiment of the present invention provides a planning and navigation system for a puncture robot, which includes: a dual-channel convolutional neural network registration module for registering the preoperative high-resolution image and ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multi-modal fusion volume image data I. fuse ; a semantic-enhanced three-dimensional model construction module for performing topological structure analysis on the multi-modal fusion volume image data I. fuse to construct a semantic-enhanced three-dimensional model; a global path planning module for performing global path planning based on the semantic-enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning; a navigation data encapsulation module for generating navigation parameters and a lightweight model that can be imported into the navigation system according to the result of the global path planning and outputting preoperative navigation data.

[0082] The third embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the planning and navigation method for a puncture robot as described above.

[0083] The fourth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the planning and navigation method for a puncture robot as described above.

[0084] The computer program product of the planning and navigation method and device for a puncture robot provided by the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the foregoing method embodiments. For specific implementation, reference can be made to the method embodiments, which will not be elaborated here.

[0085] Specifically, the storage medium can be a general storage medium, such as a removable disk, a hard disk, etc. When the computer program on the storage medium is run, it can execute the above-mentioned planning and navigation method for the puncture robot, thereby improving the accuracy and stability of registration, effectively solving the problem of multi-modal image data fusion, and realizing accurate preoperative navigation data output.

[0086] If the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0087] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments or easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A planning and navigation method for a puncture robot, characterized in that: include: The preoperative high-resolution images and ultrasound simulation data are registered through a dual-channel convolutional neural network for high-precision spatial alignment to generate multimodal fusion image data. fuse ; The multimodal fusion image data I fuse Perform topological structure analysis and construct semantically enhanced 3D models; Based on the semantically enhanced three-dimensional model, global path planning is performed through obstacle space encoding and reinforcement learning pre-planning; Based on the results of global path planning, navigation parameters and a lightweight model that can be imported into the navigation system are generated, and preoperative navigation data is output.

2. The planning and navigation method for a puncture robot according to claim 1, characterized in that: The method of registering the preoperative high-resolution image and the ultrasound simulation data through a dual-channel convolutional neural network to perform high-precision spatial alignment and generate multi-modal fusion volume image data includes: Convert preoperative high-resolution imaging data into three-dimensional Convert ultrasound simulation data into three-dimensional The preoperative high-resolution image 3D data and ultrasound simulation 3D data were isotropically resampled to unify the resolution and obtain the preoperative high-resolution image spatial standardized data. Unify data with ultrasound simulation space Design a dual-channel convolutional neural network to unify the ultrasound simulation data The grayscale distribution is mapped to the preoperative high-resolution image space normalized data Perform inter-modal contrast alignment to obtain ultrasound simulation spatial standardized data The preoperative high-resolution image spatially normalized data Perform elastic transformation to generate preoperative high-resolution registration images Among them, tri is the trilinear interpolation kernel, N(x) is the neighborhood voxel of x; The preoperative high-resolution registration image Spatially normalize data with the ultrasound simulation Weighted fusion to generate multi-modal fusion image data Among them, w pre and w us is the weight, and the weight value is adaptively adjusted according to the local signal-to-noise ratio.

3. The planning and navigation method for a puncture robot according to claim 2, characterized in that: The dual-channel convolutional neural network includes: A preoperative image encoder branch and an ultrasound image encoder branch, wherein the preoperative high-resolution image spatial normalization data Input the preoperative image encoder branch to extract the preoperative image features The ultrasound simulation spatial normalization data Input the ultrasound image encoder branch to extract the ultrasound image features Cross-modal attention fusion module, for each level of preoperative image features and ultrasound imaging features Calculate the preoperative image channel weights separately and ultrasound image channel weights Among them, GAP is the global average pooling, σ is the Sigmoid function, and W is the learnable weight matrix. and the ultrasound imaging features Perform weighted feature concatenation to obtain dynamic fusion multimodal features Deformation field prediction decoder, which dynamically fuses multimodal features Upsample and output a three-dimensional dense deformation field The preoperative high-resolution image spatial normalization data An elastic transformation is performed under the three-dimensional dense deformation field φ.

4. The planning and navigation method for a puncture robot according to claim 1, characterized in that: The multi-modal fusion image data I fuse Topological structure analysis and semantic enhanced 3D model construction include: Input multimodal fusion image data I fuse , perform voxel-level semantic segmentation and output voxel-level semantic segmentation label map L∈{0,1,...,C} H×W×D ,; The semantic segmentation label graph L is used as input to construct a topological feature graph G = (V, E, W), where V is a node set and each node v i represents an anatomical structure, E is an edge set, representing the spatial adjacency relationship of the anatomical structure, W is the adjacency matrix, and the element w ij is the edge weight element; The topological feature map is integrated with the segmentation result to construct a graph convolutional network, and a semantic attribute vector a(x)=[a1(x), a2(x), a3(x)] is assigned to each voxel x. The node v is designed. i The eigenvector of Graph convolutional layer H (l+1) =σ(D -1 / 2 WD -1 / 2 H (l) Θ (l) ), where a1(x) is the anatomical category, a2(x) is the distance to the nearest blood vessel, a3(x) is the estimated value of the tissue elastic modulus, and f geo (v i ) is the geometric feature, H (l) is the feature matrix of the l-th layer node, D is the degree matrix, Θ (l) is a learnable parameter matrix, σ is a ReLU activation function, and the output of the graph convolutional network is the enhanced node feature H (L) ; The node feature H (L) And the semantic segmentation label map L as input, for each voxel x, the semantic attributes are fused with the graph features of the nearest node Isosurfaces are extracted based on the marching cubes algorithm to generate enhanced 3D models with semantic labels.

5. The planning and navigation method for a puncture robot according to claim 1, characterized in that: The global path planning is performed based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning, including: Taking the semantically enhanced 3D model as input, the risk value of each voxel x is calculated according to the anatomical category and neighboring structures to which each voxel x belongs. Among them, d vessel (x) is the Euclidean distance from voxel x to the nearest vessel, d bone (x) is the Euclidean distance from voxel x to the nearest bone; Divide the 3D space into a voxel grid Each voxel stores a risk value r(x), generating a probability occupancy map Complete obstacle space coding; Based on the probability of occupying the map P occ , set the starting point p start and the target point p goal , at the starting point p start and the target point p goal Generate and output multiple candidate rough paths {P k }, each path consists of a sequence of key waypoints; Set the state space and action space for the local adjustment layer, set the reward function, and generate the optimal puncture path P through reinforcement learning. * ={p1,p2,...,p N }; For the optimal path P * The path is smoothed by B-spline basis function and a continuous B-spline curve is output. Among them, B i,k (u) is the B-spline basis function, P i As control points, complete global path planning.

6. The planning and navigation method for a puncture robot according to claim 5, characterized in that: The setting is used to locally adjust the state space and action space of the layer, set the reward function, and generate the optimal puncture path P through reinforcement learning. * include: Construct a state space, including the voxel storage risk value r(x) in the obstacle space encoding, the tissue elastic modulus E(x) of real-time ultrasound feedback, and the global rough path reference curvature κ global ; Define the action space, including the robot pitch angle adjustment parameter Δθ pitch ∈[-5°,5°], yaw angle adjustment parameter Δθ yaw ∈[-5°,5°], needle insertion step length Δd∈[1mm,5mm], if the current position risk value r(p t )>0.7, the needle insertion step length Δd≤2mm. bone <5mm, then the pitch angle adjustment parameter Δθ pitch and the yaw angle adjustment parameter Δθ yaw The adjustment is halved; A multi-objective reward function is set, and the state space and the action space are used as inputs of the multi-objective reward function. The reward item includes a target proximity reward R goal =10·exp(-Δd / 50), safety bonus R safe =∑ r(x)＜0.3 (1-r(x)), smoothing penalty R smooth =-20·(||Δθ pitch || 2 +||Δθ yaw || 2 +||Δκ global || 2 ) and tissue damage penalty R damage =-0.1·E(x)·Δd, output state vector, generate optimal puncture path P * ={p1,p2,...,p N }.

7. The planning and navigation method for a puncture robot according to claim 6, characterized in that: The generating of navigation parameters and a lightweight model that can be imported into the navigation system according to the result of the global path planning, and the output of preoperative navigation data include: Design a trapezoidal velocity profile v(s) to maximize the needle velocity while satisfying the acceleration constraint. Where, s is the arc length of the continuous B-spline curve, v max is the maximum permissible speed, is the arc length of the acceleration segment, a max is the maximum acceleration, s total is the total arc length, s d =s total -s a is the starting point of the deceleration section; Calculate the total puncture time Encoding the trapezoidal velocity profile v(s) and the total puncture time T into a time position lookup table, and inputting it into a real-time controller; Convert the semantically enhanced 3D model into a sparse octree structure, define node merging rules, Get the octree model T, where O i is the spatial area corresponding to the octree node, is the attribute mean within the node, δ is the attribute merging threshold; The trapezoidal velocity profile v(s) and the octree model are taken as input and encapsulated into a MATLAB navigation parameter package.

8. A planning and navigation system for a puncture robot, characterized in that: include: The dual-channel convolutional neural network registration module is used to register the preoperative high-resolution image with the ultrasound simulation data through a dual-channel convolutional neural network, perform high-precision spatial alignment, and generate multi-modal fusion image data I fuse ; A semantically enhanced three-dimensional model building module is used to transform the multimodal fusion volume image data I fuse Perform topological structure analysis and construct semantically enhanced 3D models; A global path planning module, used for performing global path planning based on the semantically enhanced three-dimensional model through obstacle space encoding and reinforcement learning pre-planning; The navigation data encapsulation module is used to generate navigation parameters and a lightweight model that can be imported into the navigation system based on the results of global path planning, and output preoperative navigation data.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the planning and navigation method for a puncture robot according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the planning and navigation method for a puncture robot according to any one of claims 1 to 7 is implemented.

Images