Craniotomy robot collaborative navigation method and system

Abstract

The invention discloses a surgical robot collaborative navigation method and system, and the method comprises the steps: S1, carrying out the three-dimensional reconstruction of a surgical site of a patient, so as to obtain a planned surgical path; s2, predicting a shielding track point of the surgical instrument on the surgical path when the surgical instrument moves on the planned surgical path and any marking ball on the first positioning tool is shielded and cannot pass through to cause tracking and positioning of the optical tracking system; s3, shielding the trajectory points, and planning to obtain an optimal collaborative navigation trajectory of the optical tracking system; the surgical robot collaborative navigation system comprises a surgical robot module, a navigation robot module and a collaborative navigation control module composed of a preoperative preparation module, a preoperative shielding prediction module and a collaborative navigation path planning module. The collaborative navigation method and system for the craniotomy operation robot can adapt to the complex operation environment, the planned path has optimality and smoothness, and the adjustment time is short.

Description

Technical Field

[0001] This invention relates to the field of medical navigation robot technology, and in particular to a collaborative navigation system for a craniotomy robot. Background Technology

[0002] Robot-assisted surgery can significantly improve the precision and safety of procedures, with optical positioning systems (OTS) being the core component for achieving high-precision navigation. However, existing OTS systems mostly employ a passive tracking mode with a fixed pose. Taking the skull as an example, the dynamic and variable pose of the surgical robot's end effector (such as a milling cutter) can easily lead to visual occlusion between the positioning markers on the end effector and the fixed OTS, resulting in the loss of real-time instrument position information and disruption of navigation continuity. This not only increases surgical time but may also cause drifting cuts at the bone window boundary, damaging the dura mater and brain tissue, seriously endangering patient safety.

[0003] To address the visual occlusion problem, existing technologies have proposed multi-camera fusion or multi-sensor fusion solutions. However, these methods increase system complexity and cost, and fail to fundamentally achieve active, blind-spot-free tracking. Although a few studies have attempted to mount OTS (Optical Trajectory System) on robotic arms for active navigation, these methods are mostly geared towards orthopedic surgeries. They lack targeted occlusion prediction and collaborative trajectory planning strategies for operations such as craniotomies, which involve complex paths, frequent instrument posture changes, and small surgical areas. Therefore, it is of great significance to develop a robotic collaborative navigation system capable of actively predicting and avoiding occlusion, ensuring continuous navigation throughout craniotomies. Summary of the Invention

[0004] The purpose of this invention is to provide a collaborative navigation system for craniotomy robots that solves the problem of visual obstruction caused by the changing positions of instruments.

[0005] Therefore, the technical solution of the present invention is as follows.

[0006] On one hand, this invention proposes a cooperative navigation method for surgical robots, the steps of which include:

[0007] S1. Perform three-dimensional reconstruction of the patient's surgical site to obtain the planned surgical path; the optical tracking system registers the first positioning tool and the second positioning tool;

[0008] S2. When the surgical instrument moves along the planned surgical path, if any marker ball on the first positioning tool is blocked and cannot pass through, causing the optical tracking system to track and position the instrument, the obstruction trajectory point of the surgical instrument on the surgical path is determined, thereby determining the obstruction path segment on the surgical path.

[0009] S3. Based on the occlusion trajectory points obtained in step S2, the optimal cooperative navigation trajectory of the optical tracking system is planned, which includes: 1) constructing a path planning network based on deep reinforcement learning of a multi-task network; 2) defining a reward function consisting of step size reward, joint reward, and first constraint reward. Based on the pose of the first localization tool at the endpoint trajectory point of the occluded path segment, a high-dimensional state vector is initialized and generated. and reward function 3) Define the trajectory planning reward consisting of target approach reward, motion smoothing reward, and second constraint reward for the first stage of training, so as to obtain the optimal observation point of the optical tracking system; and in the high-dimensional state vector Added joint velocity and acceleration ∈ 6, ∈ 6. Form a high-dimensional state vector Initialize and generate high-dimensional state vectors Trajectory planning rewards The input path planning network is then used for a second-stage training process to obtain the optimal cooperative navigation trajectory for the optical tracking system.

[0010] Furthermore, the surgical robot collaborative navigation method also includes an optical positioning system fine-tuning step, so as to make timely adjustments to the optical tracking system when the movement trajectory of the surgical instruments deviates slightly.

[0011] On the other hand, the present invention also proposes a surgical robot cooperative navigation system, comprising:

[0012] The surgical robot module includes a surgical robotic arm, a first positioning tool, and a second positioning tool; a clamp is fixed to the end of the surgical robotic arm, the first positioning tool is fixed to the clamp, and the second positioning tool is fixed to the side adjacent to the patient's surgical site.

[0013] A navigation robot module includes a navigation robotic arm with an optical tracking system fixed at its end for continuously tracking and positioning a first positioning tool and a second positioning tool;

[0014] The collaborative navigation control module includes a preoperative preparation module, a preoperative occlusion prediction module, and a collaborative navigation path planning module. The preoperative preparation module performs 3D reconstruction of the patient's surgical site to plan the surgical path and maps the planned surgical path to the patient's surgical site in the actual surgical space, mapping the end effector of the surgical instrument to the planned surgical path in the actual surgical space. The preoperative occlusion prediction module predicts the occlusion trajectory point when any marker ball on the first positioning tool is occluded during the movement of the surgical instrument along the planned surgical path. The collaborative navigation path planning module constructs and trains a path planning network based on deep reinforcement learning for multi-task networks to plan the optimal collaborative navigation trajectory of the optical tracking system based on the occlusion trajectory point.

[0015] Furthermore, the surgical robot's collaborative navigation system also includes an optical tracking system pose fine-tuning module, which is used to make timely adjustments to the optical tracking system when the movement trajectory of the surgical instruments deviates slightly.

[0016] Compared with existing technologies, this collaborative navigation method and system for craniotomy robots can pre-estimate occlusion trajectory points through preoperative occlusion prediction and intraoperative path planning. At the same time, it combines path planning using deep reinforcement learning algorithms to achieve adaptive operation in complex surgical environments. The planned path has both optimality and smoothness, enabling the navigation robotic arm to actively adjust the pose of the optical tracking system. The adjustment time is shorter than the instrument adjustment time, meeting the real-time requirements of surgery. This fundamentally avoids the visual occlusion problem caused by the changing pose of surgical instruments, ensuring the continuity and consistency of intraoperative navigation. Attached Figure Description

[0017] Figure 1 This is an architectural diagram of the surgical robot collaborative navigation system of the present invention;

[0018] Figure 2 This is a schematic diagram of the surgical robot collaborative navigation system in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of multiple key angles in the line of sight formed by the right lens of the optical tracking system relative to the clamp and surgical instruments in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the observation angle of the optical tracking system relative to two positioning tools before planning, based on the cooperative navigation path planning module in an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the observation angle of the planned optical tracking system relative to two positioning tools based on the cooperative navigation path planning module in an embodiment of the present invention;

[0022] Figure 6This is a schematic diagram of the interference angle of the left lens of the optical tracking system based on the cooperative navigation path planning module before planning, relative to the two positioning tools in an embodiment of the present invention.

[0023] Figure 7 This is a schematic diagram of the interference angle of the left lens of the planned optical tracking system relative to the two positioning tools, based on the cooperative navigation path planning module in an embodiment of the present invention.

[0024] Figure 8 This is a schematic diagram of the cooperative navigation path planning results in a simulation experiment of an embodiment of the present invention;

[0025] Figure 9 This is a comparison chart of the second-stage learning results of four methods in the simulation experiment of an embodiment of the present invention;

[0026] Figure 10 This is a comparison chart of the second-stage training results of four methods in the simulation experiment of an embodiment of the present invention. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0028] Example 1

[0029] See Figure 1 and Figure 2 A surgical robot collaborative navigation system includes a surgical robot module, a navigation robot module, and a collaborative navigation control module.

[0030] The surgical robot module includes a surgical robotic arm 1, a first positioning tool 3, and a second positioning tool 4. Specifically, the surgical robotic arm 1 is a six-degree-of-freedom industrial robotic arm (JAKA zu7), with a gripper 2 fixed at its end for setting surgical instruments. The first positioning tool 3 is fixed to the gripper 2 to track and position the gripper 2 and the surgical instruments on it. The second positioning tool 4 is fixed to the side adjacent to the patient's surgical site to track and position the patient's surgical site.

[0031] In this embodiment, the first positioning tool 3 and the second positioning tool 4 have the same structure and adopt commercially available positioning tools that can be used with optical tracking systems; specifically, the positioning tool includes an X-shaped fixing frame, and a marker ball is fixed at each of the four ends of the X-shaped fixing frame.

[0032] The navigation robot module includes a navigation robotic arm 6 and an optical tracking system 5. Specifically, the navigation robotic arm 6 adopts a six-degree-of-freedom industrial robotic arm (UR5), and its end effector is fixed to the optical tracking system (OTS) 5 through a rigid connector. In practical applications, the navigation robotic arm 6 actively adjusts the pose of the OTS to achieve continuous tracking and positioning of the first positioning tool 3 and the second positioning tool 4.

[0033] The collaborative navigation control module includes a preoperative preparation module, a preoperative occlusion prediction module, and a collaborative navigation path planning module.

[0034] The preoperative preparation module is used to perform three-dimensional reconstruction of the patient's surgical site to plan the surgical path, and uses the optical tracking system 5 to register the first positioning tool 3 and the second positioning tool 4 to map the planned surgical path to the patient's surgical site in the real surgical space, and to make the end of the surgical instrument act on the planned surgical path in the real surgical space.

[0035] Specifically, the preoperative preparation module includes:

[0036] The 3D reconstruction module is used to reconstruct a 3D model from the patient's surgical site imaging data (such as CT data or MRI data), and can draw and confirm the planned surgical path on the 3D model;

[0037] The registration module is used to uniformly register the coordinate sets on the first positioning tool 3, the second positioning tool 4, the navigation robotic arm 6, and the optical tracking system 5 to the same coordinate system, which may be, but is not limited to, the base coordinate system of the surgical robotic arm 1; then, it maps the planned surgical path to the surgical site of the patient in the actual surgical space, and maps the end of the surgical instrument to the planned surgical path in the actual surgical space.

[0038] The preoperative occlusion prediction module is used to predict, before surgery, the trajectory point of the corresponding surgical instrument end on the planned surgical path when any marker ball on the first positioning tool 3 is occluded during the movement of the surgical instrument along the planned surgical path, that is, the occlusion trajectory point.

[0039] Specifically, the preoperative occlusion prediction module includes:

[0040] The first positioning tool pose acquisition module is used to calculate the pose of the fixture at each trajectory point and convert it into the pose of the four marker balls on the first positioning tool at the trajectory points.

[0041] The spatial point set generation module is used to generate spatial point sets that describe the positions of clamps and surgical instruments on each trajectory point;

[0042] The parameter acquisition module is used to acquire the vector lengths from each spatial point in the spatial point set to the center of the left and right lenses, the vector lengths from the center point of each marker ball to the center of the left and right lenses, the difference angle between marker balls based on the left lens, the difference angle between marker balls based on the right lens, the difference angle between the marker ball and the fixture based on the left lens, and the difference angle between the marker ball and the fixture based on the right lens when the surgical instruments are located at each trajectory point of the planned surgical path.

[0043] The prediction module is used to determine whether each trajectory point is an occluded trajectory point based on the parameters obtained by the parameter acquisition module.

[0044] The collaborative navigation path planning module constructs a path planning network based on deep reinforcement learning of multi-task networks and trains it in two stages to plan the optimal collaborative navigation trajectory of the optical tracking system 5 based on the occlusion points predicted by the preoperative occlusion prediction module. This enables the optical tracking system 5 to actively move to the best observation pose when the surgical instruments move to the occlusion points on the planned surgical path.

[0045] Specifically, the collaborative navigation path planning module includes:

[0046] The network building module is used to build multi-task deep reinforcement learning path planning networks.

[0047] The first parameter generation module is used to define the reward function for the first stage of training. Furthermore, by acquiring the occlusion trajectory points predicted by the prediction module, a high-dimensional state vector is initialized and generated for the first stage of training. and reward function ;

[0048] The second parameter generation module is used to define the reward for the second-stage training trajectory planning. And based on high-dimensional state vectors Initialize and generate high-dimensional state vectors for the second stage of training. Trajectory planning rewards ;

[0049] The training module sequentially calls the first parameter generation module and the second parameter generation module to perform two-stage training on the deep reinforcement learning path planning network of the multi-task network in order to obtain the optimal cooperative navigation trajectory of the optical tracking system.

[0050] As a preferred technical solution in this embodiment, the cooperative navigation control module also includes an optical tracking system pose fine-tuning module, which is used to make timely adjustments to the optical tracking system when the movement trajectory of the surgical instrument deviates slightly.

[0051] Specifically, the pose fine-tuning module of the optical tracking system includes:

[0052] The status acquisition module is used to acquire the current motion posture of surgical instruments in real time. And the left lens - the first positioning tool observation angle And the right lens - the first positioning tool observation angle ;

[0053] The judgment module is used to calculate the observation angle of the left lens and the first positioning tool. The difference in observation angle between the planned observation angle of the left camera and the first positioning tool, and the observation angle of the right camera and the first positioning tool. The difference between the observation angles of the right camera and the first positioning tool in the corresponding plan is determined, and the adjustment drive module is activated when any observation angle difference exceeds the preset difference threshold.

[0054] The adjustment drive module is used to fine-tune the optical tracking system from the planned pose to the pose corresponding to the surgical instrument after its offset.

[0055] Example 2

[0056] A collaborative navigation method for surgical robots, the specific implementation steps of which are described below.

[0057] S1. Perform three-dimensional reconstruction of the patient's surgical site to plan the surgical path; the optical tracking system registers the first positioning tool and the second positioning tool.

[0058] In step S1, three-dimensional reconstruction of the patient's surgical site is performed based on imaging data (such as CT data and MRI data) of the patient's surgical site.

[0059] In step S1, the optical tracking system registers the first and second positioning tools using hand-eye calibration technology; specifically, see... Figure 2 In the aforementioned surgical robot collaborative navigation system, the surgical robotic arm includes, but is not limited to, its base coordinate system {Bs}, the first positioning tool corresponds to the surgical instrument coordinate system {T1}, the second positioning tool corresponds to the surgical site coordinate system {T2), the navigation robotic arm includes, but is not limited to, its base coordinate system {Bn}, and the optical tracking system corresponds to the optical coordinate system {O}. Since the optical tracking system uses a binocular camera, its left lens corresponds to the left lens coordinate system {O}. L}, its right camera corresponds to the right camera coordinate system {O R To unify the multiple coordinate systems in the usage scenario, all operations are registered to the base coordinate system {Bs} of the surgical robot arm through the transformation matrix between coordinate systems.

[0060] Therefore, the planned surgical path can be mapped to the patient's surgical site in the actual surgical space, and the end of the surgical instrument can also be mapped to the planned surgical path in the actual surgical space.

[0061] S2. When the surgical instrument moves along the planned surgical path, if any marker ball on the first positioning tool is blocked and cannot pass through, the optical tracking system will track and position the obstruction point of the surgical instrument on the surgical path.

[0062] Based on the above analysis, the situations where the marker balls on the first positioning tool are obstructed can be specifically divided into: ① the situation where the marker balls obstruct each other, and ② the situation where the clamps (including surgical instruments) obstruct the marker balls.

[0063] See Figure 3 The specific steps for determining whether the marker ball on the first positioning tool is obstructed when the surgical instrument moves to each trajectory point on the surgical path are as follows:

[0064] S201. Based on the planned surgical path, the pose of the fixture at the current trajectory point is calculated using the forward kinematics model of the surgical robot arm, and then converted into the pose of the four marker balls on the first positioning tool at the trajectory point.

[0065] S202. Obtain the spatial point set describing the position of the fixture and surgical instruments at the current trajectory point using linear interpolation. (j=1,2,...,n), where n is a set of points in space. The number of points in the midspace;

[0066] S203. Based on the imaging principles of the left lens L and right lens R of the binocular camera in the optical tracking system, calculate the spatial point set respectively. The vector length from each spatial point to the center of the left camera. Each marked ball Vector length from center point to left lens center spatial point set The vector length from each spatial point to the center of the right camera Each marked ball Vector length from the center point to the center of the right lens Where j is a set of spatial points. The index of the midspace point, i is the marker sphere The serial number;

[0067] S204. Based on the imaging principle of the left lens L and right lens R of the binocular camera in the optical tracking system 5, a pattern is formed pointing to each marker ball on the first positioning tool 3. The line of sight (i=1,2,3,4) is used to construct a virtual view frustum; similarly, the left lens L and right lens R of the binocular camera can also form a set of pointing points in space. The line of sight to each spatial point is determined, and a virtual view frustum is constructed based on this line of sight.

[0068] Based on the line of sight from the center of the left camera L to the center points of any two marked balls, the angle between the left camera L and any two marked balls can be obtained. ( ), and according to the left lens L to the marker ball half angle of the cone And left lens L to the marker ball half angle of the cone The difference angle between the marker balls based on the left lens was calculated. To be used to determine the marked ball With the marker ball Whether there is occlusion between them; similarly, based on the line of sight from the right lens R of the binocular camera to the center points of any two marker balls, the angle between the right lens R and any two marker balls can also be obtained. Right camera R to the marker ball half angle of the cone And right camera R to the marker ball half angle of the cone To calculate the value used to determine the marked ball With the marker ball The difference angle between the marker balls indicating whether occlusion occurs. .

[0069] Based on the spatial point set from the left lens L of the binocular camera The line of sight from any point in space can be used to obtain the distance from the center of the left camera to the specified marker ball. Angle between the center points Furthermore, based on the center of the left camera to the marked ball half angle of the cone The difference angle between the marker ball and the fixture based on the left lens can be calculated. To determine whether the clamp is obstructing the marker ball. Similarly, based on the spatial point set from the center of the right lens of a binocular camera... The line of sight from any point in space can be used to obtain the distance from the center of the right camera to the space point and the designated marker ball. Angle between the center points Furthermore, based on the center of the right camera to the marked ball half angle of the cone The difference angle between the marker ball and the fixture based on the right lens can be calculated. And used to determine whether the clamp obstructs the marker ball. .

[0070] See Figure 4 The right lens R of the binocular camera points to the marker ball. and the marker ball Taking the formed visual cone as an example, based on the lines of sight from the center point of the right lens R to the center points of any two marker spheres, the angle between the right lens R and any two marker spheres is obtained. Right camera R to the marker ball half angle of the cone And right camera R to the marker ball half angle of the cone The difference angle between the marker spheres based on the right lens was calculated. This is used to determine whether occlusion occurs between the marker balls from the perspective of the right lens; based on the spatial point set from the left lens L of the binocular camera. The line of sight from any point in space can be used to obtain the distance from the left camera L to the point in space and the marker sphere. Angle between the center points The difference angle between the marker ball and the fixture based on the right lens was calculated. This is used to determine whether the clamp obstructs the marker ball from the perspective of the right camera. .

[0071] S205. Based on the calculation results of steps S204 and S205 above, determine whether, at the current trajectory point, the marker ball of the first positioning tool is obscured by other marker balls, or by clamps and their surgical instruments.

[0072] Specifically, for each marked ball in turn (i=1,2,3,4) Perform the following judgment:

[0073] Condition 1: ,and or ;

[0074] Condition 2: ,and ;

[0075] Condition 3: ,and ;

[0076] Condition 4: ,and ;

[0077] When a marker ball is specified If any one of conditions 1 to 4 above is true, then the designated marked ball is proven to be true. If an area is occluded, the current trajectory point is marked as an occluded trajectory point; otherwise, it proves that the designated marker ball is not obscured. There is no obstruction in any form, and the current trajectory point is marked as a successful navigation trajectory point.

[0078] Of the four conditions mentioned above, conditions 1 and 2 are used to determine whether the marked ball is obscured by the clamp. If either condition 1 or condition 2 is true, it indicates that the point set... Compared to a marker sphere, which is closer to the camera, it's necessary to consider interference between view cones and point sets. The case of entering the cone region; while conditions 3 and 4 are used to determine whether there is a case where a marked ball is occluded by another marked ball. When either condition 3 or condition 4 is true, it indicates that the point set... "Hidden" behind the marker ball, which is closer to the camera, it means that only the interference between the view cones needs to be considered.

[0079] Repeat steps S201 to S205 above, using the clamp to move the surgical instrument to each trajectory point on the surgical path for the judgment as described above, and extract all occluded trajectory points on the surgical path for use in the subsequent collaborative navigation path planning steps.

[0080] S3. Construct and train a path planning network based on deep reinforcement learning of multi-task network to plan the optimal cooperative navigation trajectory of the optical tracking system based on the predicted occlusion trajectory points, so that the optical tracking system can actively move to the best observation pose when the surgical instrument moves to the occlusion point of the surgical path.

[0081] Specifically, the implementation steps of step S3 are as follows:

[0082] S301. Construct a path planning network (MTN-SAC network) for multi-task network deep reinforcement learning to realize cooperative navigation path planning.

[0083] The path planning network includes an Actor network and a Critic network; among them...

[0084] The Actor network consists of a shared network and a task-specific branch network connected in sequence. The shared network uses a multilayer perceptron, and the task-specific branch network consists of N task branch modules. The shared features extracted by the shared network are input into each task branch module to output the average action value corresponding to each task.

[0085] The expression for this Actor network is:

[0086] ,

[0087] ,

[0088] In the above two formulas, To share features, SN() indicates that it is implemented through a multilayer perceptron. Given a high-dimensional state vector as input. To share network parameters; This represents a task-specific branch network, where k represents the task branch module of task k in the network. For specific parameters of task k in the Actor network;

[0089] The Critic network consists of a shared network and a task-specific branch network connected sequentially. The task-specific branch network comprises N task branch modules. By inputting the shared features extracted by the shared network into each task branch module, the Q-value corresponding to each task is output. Its expression is as follows:

[0090] ,

[0091] ,

[0092] In the above two formulas, To share features, SN() indicates that it is implemented through a multilayer perceptron. Given a high-dimensional state vector as input. To share network parameters; Represents a task-specific branch network. Let i be the i-th specific parameter of task k, where i = 1 or 2.

[0093] In this embodiment, the shared network extracts features from the input information to obtain shared features that can be transferred and shared among multiple tasks. Specifically, it consists of a 256-dimensional first fully connected layer, a LayerNorm normalization layer, a ReLU activation function, a 256-dimensional second fully connected layer, a LayerNorm normalization layer, and a ReLU activation function connected in sequence. In the task-specific branch network, each task branch module consists of a 256-dimensional first fully connected layer, a LayerNorm normalization layer, a ReLU activation function, and a 256-dimensional second fully connected layer connected in sequence. The number N of task branch modules is determined according to the number of reward function types during network training.

[0094] The path planning network also includes a multi-task deep Q-network, which consists of N Q-subnetworks, each corresponding to a task branch module in the Critic network. Based on the shared features output from the Critic network, each Q-subnetwork outputs two independent Q-values; then, the smaller Q-value is selected. This is used to calculate the loss function for the corresponding task branch in the Critic network, and its expression is:

[0095] ,

[0096] In the formula, This represents a multi-task deep Q-network. The Q-network parameters for task k; The Q-value output by the first Q-network for task k. The smaller of the two Q-values ​​is the Q-value output by the second Q-network for task k. Feedback is sent to the Critic network and used for loss function calculation in each task branch module.

[0097] In an Actor network, to ensure that each task branch can be optimized for a specific objective, a loss function is defined that applies to each task branch module in the task-specific branch network. Its expression is:

[0098] ,

[0099] In the formula, For temperature parameters, Let k be the policy function for task k. This represents the logarithmic approximation of the action. Let Q be the value of task k. Let k be the current state of task k. This represents the current action of task k.

[0100] In the Critic network, a loss function is defined that is applicable to each task branch module in the task-specific branch network. Its expression is:

[0101] ,

[0102] ,

[0103] In the formula, For the next state The action of sampling under the current policy, This is the output of the Critic network. It's a temperature parameter. It is a discount factor. The immediate reward for task k. The target Q-value for task k is output by the multi-task deep Q-network.

[0104] As a feature extractor for multiple tasks, the shared network does not require a separately set loss function. Instead, it relies on the overall loss of each task-specific branch module in the subsequent task-specific branch network to backpropagate and optimize the parameters. The overall loss is expressed as:

[0105] ,

[0106] In the formula, N represents the number of tasks. Let the Actor network loss be for task k. Let be the Critic network loss for task k. Then, the parameters are updated by calculating the gradient of the overall loss. Optimize the shared network.

[0107] To simultaneously optimize Q-value estimation for all tasks, a temporal difference learning approach is used to achieve collaborative learning among tasks. Accordingly, a loss function suitable for each Q-subnetwork in a multi-task deep Q-network is defined. Its expression is:

[0108] ,

[0109] In the formula, The immediate reward for task k. As a discount factor, The Q-value of task k output by the Critic network. The target Q-value for task k is output by the multi-task deep Q-network. For entropy regularization, This is the next state for task k. This is the next action for task k.

[0110] The goal of constructing this MTN-SAC network is to achieve efficient feature sharing and task-independent optimization in multi-task learning of cooperative navigation trajectories. This improves computational efficiency while ensuring the accuracy of task optimization through the synergistic effect of three types of networks, effectively solving complex tasks.

[0111] S302. The newly constructed MTN-SAC network is trained in the first stage to output the optimal observation point of the optical tracking system. The specific implementation steps are described below.

[0112] S3021. Determine the high-dimensional state vector for the first stage of training. It consists of the positioning tool state T, the navigation robot state G, and environmental constraints. Its composition, its expression is:

[0113] .

[0114] in,

[0115] The positioning tool state T is composed of the states of the first positioning tool (T1) and the second positioning tool (T2), and its expression is:

[0116] ,

[0117] in, and These represent the position and four-element orientation of the first positioning tool. and These represent the position and four-element pose of the second positioning tool. In practical applications, the second positioning tool is fixed adjacent to the patient's surgical site, and its pose remains unchanged throughout the entire process, while the pose of the first positioning tool is constantly changing.

[0118] The expression for the state G of the navigation robotic arm is:

[0119] ,

[0120] Where θ is the current joint angle of the navigation robot arm, θ∈ 6; Pee and Qee are the position and quaternion pose of the end effector of the navigation robot, respectively, Pee∈ 3, Qee∈ 3; P L and Q L These are the center position and pose quaternions of the left camera, respectively, P L ∈ 6, Q L ∈ 6; P R and Q R These are the center position and pose quaternions of the right camera, respectively, P R ∈ 6, Q R ∈ 6.

[0121] Environmental constraints The expression is:

[0122] ,

[0123] in, For the flip angle of the optical tracking system, , Let {Bs} be the z-axis of the base coordinate system. Let {O} be the y-axis of the optical coordinate system. ∈ ;

[0124] The interference angle of the left lens. , , Let be the angle formed by the center of the left lens to the center of the smallest enclosing sphere of the first positioning tool and the center of the smallest enclosing sphere of the second positioning tool. The angle formed by the line from the center of the left lens to the center of the smallest enclosing sphere of the first positioning tool and the tangent line from the center of the left lens to the smallest enclosing sphere of the first positioning tool. The angle between the center of the left lens and the center of the minimum wrapping sphere of the second positioning tool and the tangent line between the center of the left lens and the minimum wrapping sphere of the second positioning tool; wherein, the minimum wrapping sphere of the positioning tool is defined as: the center of the minimum wrapping sphere is the origin of the positioning tool coordinate system, and the radius is the distance from the origin of the positioning tool coordinate system to the farthest marker sphere.

[0125] Similarly, The interference angle of the right lens. , , Let be the angle formed by the center of the right lens to the center of the smallest enclosing sphere of the first positioning tool and the center of the smallest enclosing sphere of the second positioning tool. The angle between the center of the right lens and the center of the smallest enclosing sphere of the first positioning tool, and the tangent line from the center of the right lens to the smallest enclosing sphere of the first positioning tool. The angle between the center of the right lens and the center of the smallest enclosing sphere of the second positioning tool and the tangent line between the center of the right lens and the smallest enclosing sphere of the second positioning tool.

[0126] like Figure 6 and Figure 7 As shown, the interference angle of the left lens Interference angle with the right lens It can be used to determine whether there is obstruction between two positioning tools. Specifically, when the calculated results of the two interference angles are both ≥0, it can be determined that the two positioning tools do not obstruct each other.

[0127] Left lens - first positioning tool observation angle, , ∈ , The vector from the center of the left camera to the center of the first positioning tool. ∈ , Let be the vector length from the center of the left lens to the center of the first positioning tool. The normal vector of the first positioning tool. ∈ , The normal vector length of the first positioning tool;

[0128] The observation angle of the left lens - the second positioning tool. , ∈ 4, The vector from the center of the left camera to the center of the second positioning tool. , This is the vector length from the center of the left camera to the center of the second positioning tool. The normal vector of the second positioning tool. , The normal vector length of the second positioning tool;

[0129] The observation angle of the right lens - the first positioning tool. , ∈ , The vector from the center of the right camera to the center of the first positioning tool. , The vector length from the center of the right camera to the center of the first positioning tool;

[0130] The observation angle of the right lens - the second positioning tool. , ∈ 4, The vector from the center of the right camera to the center of the second positioning tool. , It is the vector length from the center of the right camera to the center of the second positioning tool.

[0131] like Figure 4 and Figure 5 As shown, the left lens - the observation angle of the first positioning tool Left lens - second positioning tool observation angle Right lens - first positioning tool observation angle Right lens - second positioning tool observation angle In binocular cameras used in quantization optical tracking systems, the question arises regarding whether collinear marker spheres exist in the field of view of either the left or right lens; specifically, when the observation angle... , , , If any value is ≥90°, the intraoperative navigation is considered to be obstructed.

[0132] For the field of view plane parameters, , Π∈ 40;

[0133] In the formula, A, B, C, and D are the field-of-view plane parameters of the optical tracking system, and the subscripts 1 to 10 of each field-of-view plane parameter correspond to the serial numbers of the 5 field-of-view planes of the optical tracking system.

[0134] d T1 Let be the distance from the minimum outer boundary of the first positioning tool to the field of view plane, and its expression is:

[0135] ,

[0136] d T2 The distance from the minimum outer boundary of the second positioning tool to the field of view plane is expressed as: ,

[0137] In the formula, A i B i C i D i These are the parameters of the i-th field of view plane of the optical tracking system, i=1,2,…,10; , and These are the coordinates of the origin of the coordinate system of the first positioning tool. , and These are the coordinates of the origin of the coordinate system for the second positioning tool. The minimum outer radius of the sphere encompassing the first positioning tool. The minimum outer radius of the sphere for the second positioning tool.

[0138] S3022. Construct the reward function for the first stage of training. Its reward is based on step length. Joint rewards and first constraint reward Composition; based on this reward function Composed of three types of rewards, the corresponding multi-task network deep reinforcement learning path planning network has three task molecule modules and three Q-sub-networks, corresponding to step-size reward task optimization, joint reward task optimization, and first constraint reward task optimization, respectively.

[0139] reward function The expression is:

[0140] ,

[0141] Among them, step size reward Set the pose of the optical tracking system at time t Compared with the initial pose The Euclidean distance is used to minimize the spatial movement of the optical tracking system and reduce dynamic positioning errors caused by movement. Its expression is: ;

[0142] Joint rewards The expression used to prevent the navigation robotic arm from exceeding joint limits is:

[0143] ,

[0144] In the formula, The minimum and maximum joint angles allowed for the movement of the joints of the navigation robot arm, where i represents the number of joints; in this embodiment, based on the navigation robot arm being a six-degree-of-freedom robot arm, i=1,2,…,6, which correspond to the six joints of the navigation robot arm respectively;

[0145] First constraint reward It acts as a constraint, and its expression is: ,in, The weighting coefficient for the observation angle. For all observation angles reward function The sum, its expression is:

[0146] ,

[0147] In the formula, For the lens The observation angle of phase j, Ti, is determined by the positioning tool; where j is either L or R. L stands for left lens. The right camera is represented by Ti; Ti represents the positioning tool, where i is 1 or 2, T1 is the first positioning tool, and T2 is the second positioning tool.

[0148] These are the weighting coefficients for the interference angle. For all interference angles reward function The sum, its expression is:

[0149] ,

[0150] In the formula, For the lens The interference angle of j;

[0151] , For d Ti Corresponding field of view coverage reward function The sum, its expression is:

[0152] ,

[0153] In the formula, d Ti For positioning tools The distance from the minimum outer boundary of the enclosing sphere to the field of view plane; For the minimum enclosing sphere radius of the positioning tool Ti, specifically, The minimum outer radius of the sphere encompassing the first positioning tool. The minimum outer radius of the sphere for the second positioning tool.

[0154] The weighting coefficient for the reversal angle. For the reverse angle The corresponding flip reward function is expressed as follows:

[0155] .

[0156] S3023, Perform the first stage training on the deep reinforcement learning path planning network of the multi-task network;

[0157] 1) Based on the multiple trajectory occlusion points obtained in step S2, determine the occlusion path segments that may be occluded, and extract the pose of the first localization tool under the endpoint trajectory point of each occlusion path segment to initialize the generation of a high-dimensional state vector. and reward function ;

[0158] Based on the actual application, the multiple trajectory occlusion points obtained in step S2 are presented as multiple path segments on the surgical path. Each occlusion path segment has a start point and an end point, and corresponds to the process from the start of occlusion to the end of occlusion on the marker ball of the first positioning tool. Since the surgical planning path (such as the bone milling path) is generally non-zigzag and smooth, the "severity" of occlusion increases continuously from the start of occlusion to the end of occlusion. Therefore, for each occlusion path segment, it is only necessary to consider the pose of the first positioning tool under the end trajectory point.

[0159] 2) Initialize and generate high-dimensional state vectors for the first stage of training. and reward function ;

[0160] 3) Transform the high-dimensional state vector and reward function The input is fed into the multi-task deep reinforcement learning path planning network constructed in step S301 to minimize the network's three loss functions and maximize the reward function. The first phase of training was conducted on the network with the goal of achieving this.

[0161] Once the first phase of training of the network is completed, its output is the optimal observation pose of the optical tracking system, which is also the optimal pose of the termination occlusion point.

[0162] S303. Perform a second-stage training on the deep reinforcement learning path planning network of the multi-task network to plan the optimal cooperative navigation trajectory of the optical tracking system.

[0163] The specific implementation steps for the second phase of training are described below.

[0164] S3031. Determine the high-dimensional state vector for the second stage of training. , its in Add joint velocity and acceleration to the base. ∈ 6, ∈ 6, specifically, consists of the positioning tool state T, the navigation robotic arm state G, and environmental constraints. Navigation robotic arm joint speed and the joint acceleration of the navigation robotic arm Its composition, its expression is:

[0165] .

[0166] S3032. The purpose of the second phase of training is to plan the cooperative navigation trajectory of the optical tracking system relative to the pose of the surgical instruments within the corresponding time period. The pose adjustment is completed within the time frame, and based on this, a trajectory planning reward is constructed for the second phase of training. Its reward is based on the proximity of the target. Smooth motion reward Second Constraint Reward Composition; similarly, rewards are planned based on this trajectory. Composed of three types of rewards, in the second stage of training of the deep reinforcement learning path planning network of the multi-task network, the three task molecular modules and the three Q sub-networks correspond to the target proximity reward task optimization, motion smoothing reward task optimization and second constraint reward task optimization, respectively.

[0167] Specifically, the trajectory planning reward The expression is:

[0168] ;

[0169] Among them, the target proximity reward The expression is: In the formula,

[0170] ,

[0171] ,

[0172] ,

[0173] in, It is a range reward, used to reward OTS behavior that meets position and attitude accuracy requirements at the end of the current round. , These are the reward weights, The Euclidean distance between the current position of the optical tracking system and the optimal observation point. The direction error is calculated based on unit quaternions; δpos and δrot are the success thresholds, respectively.

[0174] To facilitate the approach to rewards, it is used to provide continuous guidance for each step of the behavior toward the goal, avoiding the problem of sparse rewards making learning difficult;

[0175] It is a time-based reward, used to reward users who meet a fixed set value. Limitations to ensure that the trained network... The cooperative navigation trajectory of the optical tracking system can be completed within a short period of time, among which, It is the time of step i.

[0176] Motion smoothing reward The expression is: In the formula,

[0177] ,

[0178] , ,

[0179] , ,

[0180] in, The joint angle reward is used to ensure trajectory smoothness and is achieved by penalizing excessive joint angle changes; among which, and These are the joint angles at time step t+1 and time step t, respectively, where i is the joint number.

[0181] This is a joint angular velocity reward, used to constrain the instantaneous motion velocity of joints, preventing excessive joint speed and ensuring smooth movement. It penalizes the absolute value of the velocity of each joint in each step. It represents the maximum permissible angular velocity of the i-th joint; N is the total number of joints, N=6; joint angular velocity is the angular velocity of the i-th joint at the current time step, which can be obtained from the joint angles of two consecutive steps using the finite difference method; Δt is the time step size. and These are the joint angles at time steps t and t-1, respectively.

[0182] Joint acceleration reward, used to constrain instantaneous joint acceleration, aims to avoid excessive acceleration, reduce mechanical shock, and ensure smooth movement. Specifically, it penalizes the absolute value of acceleration for each joint in each step to encourage the network to learn smooth acceleration trajectories during training; among these, joint angular acceleration... It is the angular acceleration of the i-th joint at the current time step. It is the maximum permissible angular acceleration of the i-th joint.

[0183] Second constraint reward The expression is: In the formula, and These are the constraint reward and joint reward in the reward function of the first training phase, respectively.

[0184] The trajectory planning reward It serves as the basis for decision-making in the second training phase, transforming abstract requirements such as the optical tracking system's ability to quickly and accurately approach the optimal observation point, smooth motion, and safety constraints into learnable information.

[0185] S3033, Perform the second stage training on the deep reinforcement learning path planning network of the multi-task network;

[0186] 1) Initialize and generate high-dimensional state vectors for the second stage of training. Trajectory planning rewards ;

[0187] 2) Transform the high-dimensional state vector Trajectory planning rewards The input is fed into the multi-task deep reinforcement learning path planning network after the first stage of training, in order to minimize the network's three loss functions and maximize the trajectory planning reward. The network is then trained in the second phase with the goal of achieving this.

[0188] After the second phase of training of the network is completed, the three loss functions of the network converge to a minimum, while the trajectory planning reward... Reaching its maximum value and also exhibiting convergence, the network's output is the result obtained through optimal planning. The cooperative navigation trajectory of the optical tracking system can be planned within a time limit.

[0189] The training process for the second training phase is the same as that for the first phase. In this embodiment, the training platform for both the first and second phases is equipped with an NVIDIA-RTX-4090 GPU (24GB VRAM) and an Intel-i9-13900K processor; the software environment uses PyTorch 3.9 and PyTorch 2.0.0 frameworks, along with the CUDA 11.8 acceleration library to achieve efficient neural network training. The network training employs a phased optimization strategy and uses the Adam optimizer; the learning rate of the Actor network is set to 1×10⁻⁶. -4 , β1=0.9, β2=0.999, ε=1×10 -8 The learning rate of the Critic network was set to 1×10. -3 The entropy coefficient α was optimized using the Adam optimizer with a learning rate of 1×10⁻⁶. -4 For training hyperparameters, the experience replay mechanism uses a capacity of 1×102. 5 The circular buffer is configured with a batch size of 128. A discount factor γ = 0.98, a soft update coefficient τ = 0.005, and a maximum gradient norm limit of 0.5 are used to effectively prevent gradient explosion during training. The first training phase (finding the optimal observation point) consists of 100 training episodes, with a maximum of 100 steps per episode. During the warm-up phase, a random policy is used to fill the experience replay buffer until the batch size is doubled. Convergence is achieved when both the policy loss and value loss stabilize, and the reward function reaches a plateau. The second training phase (cooperative navigation trajectory planning) is conducted based on the convergence of the first phase, maintaining the same training cycle to ensure sufficient learning by the network.

[0190] Considering the influence of interaction forces or vibrations between the milling cutter and the skull during actual surgical instrument operation (taking bone milling as an example), unexpected situations may occur where the milling path deviates slightly from the planned path, forcing the optical positioning system to fine-tune its cooperative navigation trajectory. Although the probability of this situation is very small, for the safety of cooperative navigation, this surgical robot cooperative navigation method also includes a fine-tuning step of the optical positioning system to achieve real-time correction of the cooperative navigation path.

[0191] Specifically, in the fine-tuning step of the optical positioning system, since the operating space of the surgical instruments is much smaller than the field of view of the optical positioning system, the fine-tuning trigger mechanism of the optical positioning system only needs to consider the offset of the milling cutter's posture. This posture offset can be converted into a change in the difference between the real-time observation angle and the observation angle on the pre-planned trajectory. Based on practical surgical experience, when the difference in observation angle reaches 5° or more, it can be determined that the surgical instrument offset is too large, and the optical positioning system needs to perform motion compensation to ensure the effectiveness and safety of navigation.

[0192] Based on this, the specific implementation steps of the fine-tuning process of the optical positioning system are as follows:

[0193] 1) During the movement of surgical instruments, the OTS can monitor the movement posture of the surgical instruments in real time. And obtain the observation angle of the left camera - the first positioning tool in real time. And the right lens - the first positioning tool observation angle ;

[0194] 2) Real-time calculation of the planned observation angle of the left camera-first positioning tool along the surgical path and the observation angle of the left camera-first positioning tool. The difference, and the planned observation angle of the right lens - first positioning tool and the observation angle of the right lens - first positioning tool. The difference;

[0195] 3) Set the observation angle difference threshold to 5°, and when the difference between any of the above observation angles is ≥5°, move the optical tracking system from the planned pose. Fine-tune to the position corresponding to the surgical instrument offset ;

[0196] Specifically, given that the base coordinate system of the cooperative navigation system is {Bs}, the transformation matrix of the positioning tool T in coordinate system {Bs} can be obtained through the forward kinematics of the surgical robot. The transformation matrix of the OTS observation and positioning tool T is: Therefore, the transformation matrix of OTS in coordinate system {Bs} for:

[0197] ,

[0198] If the milling cutter trajectory deviates during the procedure, the transformation matrix of the positioning tool in the OTS will be used. Compared to the transformation matrix of preoperative planning It will definitely change. The correct observation location for OTS should be... :

[0199] ,

[0200] Therefore, the motion compensation for OTS fine-tuning is the OTS from Adjust to .

[0201] To further demonstrate the effectiveness of the robot cooperative navigation method and system of the present invention, simulation experiments were conducted. As a control, the simulation experiments also used three deep reinforcement learning algorithms suitable for continuous motion spaces: DDPG, SAC, and TD3.

[0202] like Figure 8 The diagram shows the simulation results of collaborative path planning using the method of the present invention (MTN-SAC): Under the condition of the same initial planned path, taking the surgical instrument as a milling cutter as an example, it first obtains the potential occlusion points on the osteotomy path through the preoperative occlusion prediction module. After the milling cutter reaches the key hole B, the navigation robotic arm starts to drive the OTS to move along the planned collaborative navigation path, and can complete the adjustment time at the key hole B in advance. This ensures the continuous capture of the milling cutter by the OTS and avoids unnecessary dynamic errors caused by the OTS adjusting the observation pose when the milling cutter is milling bone.

[0203] like Figure 9 The diagram shows the simulation results of collaborative path planning using four deep reinforcement learning algorithms applicable to continuous action spaces: the present invention (MTN-SAC), DDPG, SAC, and TD3. The results show that, under the same shared parameters, the method of this application is significantly superior to the other three algorithms. Specifically, during the second stage of training, the network of the present invention converges at approximately 270 episodes, while the curves of the other three models only begin to show signs of convergence at the end of training. This indicates that more training segments are needed to reach a stable state, meaning that the convergence speed of the present invention is at least 45% faster than the other three methods during training.

[0204] Next, to demonstrate the effectiveness of different algorithms in cooperative navigation path planning for optical tracking systems, the OTS cooperative paths planned by the four algorithms were compared under the same starting point and optimal observation point. The results are as follows: Figure 10 As shown in the figure, the blue and red dots represent the OTS starting point and the optimal observation point, respectively. The black dashed line representing the demonstration path planned by this invention (MTN-SAC) is the smoothest and has the closest shape match compared to the other three planned paths, with the smallest deviation from the optimal observation point. In contrast, the path smoothing effect of the other three algorithms is significantly worse, and their similarity to the demonstration path differs greatly.

Claims

1. A method for cooperative navigation of a surgical robot, characterized by the following steps: include: S1. Perform three-dimensional reconstruction of the patient's surgical site to obtain the planned surgical path; The optical tracking system registers the first and second positioning tools; S2. When the surgical instrument moves along the planned surgical path, if any marker ball on the first positioning tool is blocked and cannot pass through, causing the optical tracking system to track and position the instrument, the obstruction trajectory point of the surgical instrument on the surgical path is determined, thereby determining the obstruction path segment on the surgical path. S3. Based on the occlusion trajectory points obtained in step S2, the optimal cooperative navigation trajectory of the optical tracking system is planned, which includes: 1) Constructing a path planning network based on deep reinforcement learning of multi-task networks; 2) Define a reward function consisting of step size reward, joint reward, and first constraint reward. Based on the pose of the first localization tool at the endpoint trajectory point of the occluded path segment, a high-dimensional state vector is initialized and generated. and reward function 3) Define the trajectory planning reward consisting of target approach reward, motion smoothing reward, and second constraint reward for the first stage of training, so as to obtain the optimal observation point of the optical tracking system; and in the high-dimensional state vector Joint velocities and accelerations are added to form a high-dimensional state vector. Initialize and generate high-dimensional state vectors Trajectory planning rewards The input path planning network is then used for a second-stage training process to obtain the optimal cooperative navigation trajectory for the optical tracking system.

2. The surgical robot cooperative navigation method according to claim 1, characterized in that, In step S1, the specific steps for determining whether the marker ball on the first positioning tool is obstructed when the surgical instrument moves to each trajectory point on the surgical path are as follows: S201. Based on the planned surgical path, the pose of the four marker balls on the first positioning tool at the current trajectory point is obtained through the positive kinematics model of the surgical robot arm. S202. Obtain the set of spatial points describing the positions of the fixture and surgical instruments at the current trajectory point. (j=1,2,...,n), where n is a set of points in space. The number of points in the midspace; S203, Calculate the spatial point sets respectively The vector lengths from each spatial point in the image to the center of the left and right shots, respectively. and and each marked ball Vector lengths from the center point to the center of the left and right shots, respectively. and j is a set of points in space. The index of the midspace point, i is the marker sphere The serial number; S204. Calculate the difference angle between the marker balls based on the left lens. And the difference angle between the marker balls based on the right lens , ; Calculate the angle difference between the marker ball and the fixture based on the left lens respectively. And the difference angle between the marker ball and the clamp based on the right lens. ; S205, sequentially check each marked ball (i=1,2,3,4) Perform the following conditional judgment: when any marked ball If any one of the above conditions 1 to 4 is true, then the current trajectory point is determined to be an occluded trajectory point. Condition 1: ,and or ; Condition 2: ,and ; Condition 3: ,and ; Condition 4: ,and .

3. The surgical robot cooperative navigation method according to claim 2, characterized in that, In step S204, based on the line of sight of the left or right camera, any two marker balls on the first positioning tool... and The formula for calculating the angle difference between the marked balls is: ， ， In the formula, The difference angle between the marked balls is based on the left lens. For the left camera and two marker balls , The angle between them From the left camera to the marker ball The half-angle of the cone, From the left camera to the marker ball The half-angle of the cone; The difference angle between the marked spheres is based on the right lens. For the right camera and two marker balls , The angle between them To the right camera to the marker ball The half-angle of the cone, To the right camera to the marker ball The half-angle of the cone; Based on the viewpoint of the left or right camera, any marked sphere and the set of spatial points The formula for calculating the angle difference between the marker sphere and the fixture formed between any two spatial points is as follows: ， ， In the formula, The angle difference between the marker ball and the clamp is based on the left lens. From the center of the left camera to the spatial point and the marker ball The angle between the center points, Left camera center to marker ball The half-angle of the cone, The angle difference between the marker ball and the clamp is based on the right lens. From the center of the right camera to the spatial point and the marker ball The angle between the center points, From the center of the right camera to the marker ball The half-angle of the visual cone.

4. The surgical robot cooperative navigation method according to claim 1, characterized in that, In step S3, the path planning network of the multi-task deep reinforcement learning network includes an Actor network and a Critic network. Both the Actor and Critic networks consist of interconnected shared networks and task-specific branch networks. Each task-specific branch network comprises N task branch modules, which, based on shared features extracted from the shared network, output the action mean and Q-value corresponding to each task. The path planning network also includes a multi-task deep Q-network, which consists of N Q-subnetworks. Each Q-subnetwork includes two independent Q-networks, which output two Q-values ​​based on the shared features of the Critic network, and the smaller value is used to calculate the loss function of the corresponding task branch in the Critic network. The loss function expression for each task branch module in the Actor network is as follows: ， In the formula, For temperature parameters, Let k be the policy function for task k. This is the logarithmic approximation of the action. Let Q be the value of task k; The loss function expressions for each task branch module in the Critic network are as follows: ， ， In the formula, For the next state The action of sampling under the current policy, This is the output of the Critic network. It's a temperature parameter. It is a discount factor. The immediate reward for task k. Let i be the target Q-value of task k output by the multi-task deep Q-network, and let i represent the i-th specific parameter of task k. The loss function expression for a shared network is: ， In the formula, N represents the number of tasks; The loss function expressions for each Q-subnetwork in a multi-task deep Q-network are as follows: ， In the formula, The immediate reward for task k. As a discount factor, The Q-value of task k output by the Critic network. For entropy regularization, This is the next state for task k. This is the next action for task k.

5. The surgical robot cooperative navigation method according to claim 1, characterized in that, In the first stage of training the path planning network reward function The expression is: ;in, Step length reward Set the pose of the optical tracking system at time t Compared with the initial pose Euclidean distance, ; Joint rewards Configure joint restrictions for the navigation robotic arm. , The minimum and maximum joint angles allowed for the movement of the joints of the navigation robotic arm, where i represents the number of joints; Constraints and Rewards Set to: , The weighting coefficient for the observation angle. For all observation angles reward function The sum of , For the lens j is the observation angle of the positioning tool Ti, j is L or R, Ti represents the positioning tool, and i is 1 or 2; These are the weighting coefficients for the interference angle. For all interference angles reward function The sum of , For the lens The interference angle of j; , For d Ti Corresponding field of view coverage reward function The sum of d Ti For positioning tools The distance from the minimum outer boundary of the enclosing sphere to the field of view plane; Let Ti be the minimum radius of the enclosing sphere of the positioning tool; The weighting coefficient for the reversal angle. For the reverse angle The corresponding flip reward function, ; High-dimensional state vector The expression is: ,in, T represents the positioning tool status. , and These represent the position and four-element orientation of the first positioning tool. and These represent the position and four-element orientation of the second positioning tool, respectively. G represents the state of the navigation robotic arm. θ is the current joint angle of the navigation robot arm, θ∈ 6; Pee and Qee are the position and quaternion pose of the end effector of the navigation robot, respectively, Pee∈ 3, Qee∈ 3; P L and Q L These are the center position and pose quaternions of the left camera, respectively, P L ∈ 6, Q L ∈ 6; P R and Q R These are the center position and pose quaternions of the right camera, respectively, P R ∈ 6, Q R ∈ 6; Due to environmental constraints, , in, For the flip angle of the optical tracking system, The interference angle of the left lens. The interference angle of the right lens. Left lens - first positioning tool observation angle, The observation angle of the left lens - the second positioning tool. The observation angle of the right lens - the first positioning tool. The observation angle of the right lens - the second positioning tool. Let d be a parameter of the field of view plane. T1 d is the distance from the minimum outer boundary of the first positioning tool to the field of view plane. T2 This is the distance from the minimum outer boundary of the sphere to the field of view plane of the second positioning tool.

6. The surgical robot cooperative navigation method according to claim 5, characterized in that, In the second stage of training the path planning network Trajectory planning reward The expression is: ;in, Target proximity reward Set to: , For range rewards, , , These are the reward weights, The Euclidean distance between the current position of the optical tracking system and the optimal observation point. The direction error is calculated based on unit quaternions; δpos and δrot are the success thresholds, respectively. To get closer to the reward, , As a time reward, , It is the time of step i; Motion smoothing reward Set to: ,in, For joint angle reward, , and These are the joint angles at time step t+1 and time step t, respectively, where i is the joint number. For joint angular velocity bonus, , , It is the maximum permissible angular velocity of the i-th joint, where N is the total number of joints and the joint angular velocity. It is the angular velocity of the i-th joint at the current time step, and Δt is the time step size. and These are the joint angles at time steps t and t-1, respectively. For joint acceleration rewards, , Joint angular acceleration It is the angular acceleration of the i-th joint at the current time step. It is the maximum permissible angular acceleration of the i-th joint; Second constraint reward Set to: , and Reward functions The first constraint reward and joint reward in the process.

7. The surgical robot cooperative navigation method according to claim 1, characterized in that, It also includes fine-tuning steps for the optical positioning system: 1) During the movement of the surgical instruments, the optical positioning system monitors the movement posture of the surgical instruments in real time and acquires the observation angles of the left lens and the first positioning tool in real time; 2) Calculate in real time the difference between the planned observation angle of the left camera-first positioning tool and the current observation angle of the left camera-first positioning tool, as well as the difference between the planned observation angle of the right camera-first positioning tool and the current observation angle of the right camera-first positioning tool; 3) Set the observation angle difference threshold, and when the difference of any of the above observation angles is greater than or equal to the observation angle difference threshold, fine-tune the optical tracking system from the planned pose to the pose corresponding to the surgical instrument after the offset.

8. A surgical robot collaborative navigation system, characterized in that, include: The surgical robot module includes a surgical robotic arm (1), a first positioning tool (3), and a second positioning tool (4); the surgical robotic arm (1) has a clamp (2) fixed at its end, the first positioning tool (3) is fixed on the clamp (2), and the second positioning tool (4) is fixed on the side adjacent to the patient's surgical site; The navigation robot module includes a navigation robotic arm (6) with an optical tracking system (5) fixed at its end for continuously tracking and positioning a first positioning tool (3) and a second positioning tool. The collaborative navigation control module includes a preoperative preparation module, a preoperative occlusion prediction module, and a collaborative navigation path planning module. The preoperative preparation module performs three-dimensional reconstruction of the patient's surgical site to plan the surgical path and maps the planned surgical path to the patient's surgical site in the real surgical space, and maps the end of the surgical instrument to the planned surgical path in the real surgical space. The preoperative occlusion prediction module is used to predict the occlusion trajectory point corresponding to any marker ball on the first positioning tool (3) when it is occluded during the movement of the surgical instrument in the planned surgical path. The cooperative navigation path planning module constructs and completes the training of a path planning network based on deep reinforcement learning of multi-task networks, so as to plan the optimal cooperative navigation trajectory of the optical tracking system (5) according to the occlusion trajectory points.

9. The surgical robot collaborative navigation system according to claim 8, characterized in that, The preoperative occlusion prediction module includes: The first positioning tool pose acquisition module is used to calculate the pose of the fixture at each trajectory point and convert it into the pose of the four marker balls on the first positioning tool at the trajectory points. The spatial point set generation module is used to generate spatial point sets that describe the positions of clamps and surgical instruments on each trajectory point; The parameter acquisition module is used to acquire the vector lengths from each spatial point in the spatial point set to the center of the left and right lenses, the vector lengths from the center point of each marker ball to the center of the left and right lenses, the difference angle between marker balls based on the left lens, the difference angle between marker balls based on the right lens, the difference angle between the marker ball and the fixture based on the left lens, and the difference angle between the marker ball and the fixture based on the right lens when the surgical instruments are located at each trajectory point of the planned surgical path. The prediction module is used to determine whether each trajectory point is an occluded trajectory point based on the parameters obtained by the parameter acquisition module.

10. The surgical robot collaborative navigation system according to claim 8, characterized in that, The collaborative navigation path planning module includes: The network building module is used to build multi-task deep reinforcement learning path planning networks. The first parameter generation module is used to define the reward function for the first stage of training. Furthermore, by acquiring the occlusion trajectory points predicted by the prediction module, a high-dimensional state vector is initialized and generated for the first stage of training. and reward function ; The second parameter generation module is used to define the reward for the second-stage training trajectory planning. And based on high-dimensional state vectors Initialize and generate high-dimensional state vectors for the second stage of training. Trajectory planning rewards ; The training module sequentially calls the first parameter generation module and the second parameter generation module to perform two-stage training on the deep reinforcement learning path planning network of the multi-task network in order to obtain the optimal cooperative navigation trajectory of the optical tracking system.

Images