Method and device for dynamically adjusting motion control mode based on intraoperative view environment, medium and program product

Abstract

The embodiment of the invention provides a method and equipment for dynamically adjusting a motion control mode based on an intraoperative visual field environment, a medium and a program product, and relates to the field of intelligent medical treatment. The method comprises the following steps: acquiring an intraoperative view image and a preset basic force threshold value according to the tissue type of a to-be-operated site; calculating an intraoperative tissue real-time deformation rate based on the intraoperative view image, and calculating a real-time force safety threshold according to a preset basic force threshold and the tissue real-time deformation rate; receiving real-time contact force of the instrument and the tissue, comparing the real-time contact force with a real-time force safety threshold value, and if the real-time contact force is smaller than or equal to the real-time force safety threshold value, outputting an instruction of damping force in direct proportion to the real-time contact force; if the real-time contact force is larger than the real-time force safety threshold value, an emergency braking program instruction is output. According to the method, factors, such as tissue deformation and illumination change, which cause instability of the surgical field of view in the surgical environment are introduced, so that the environmental adaptability and dynamic adjustment of the field of view mechanism in the robot-assisted surgical operation process are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical care, and more specifically, to a method, device, medium, and program product for dynamically adjusting motion control based on the intraoperative visual environment. Background Technology

[0002] With the deep integration of medical technology and robotics, neurosurgery is gradually breaking through the precision and safety boundaries of traditional procedures, moving towards "minimally invasive, precise, and intelligent" approaches. However, the safety and interactive control technology of existing neurosurgical robots still faces many bottlenecks that urgently need to be overcome:

[0003] The system suffers from poor environmental adaptability and visualization capabilities. During surgery, factors such as tissue deformation, changes in lighting, and instrument obstruction can lead to instability in the surgical field of view. Existing systems lack a mechanism for dynamically adjusting the field of view, often resulting in instruments moving out of the field of view or key anatomical structures appearing blurred. Furthermore, traditional visualization methods can only present two-dimensional images or static three-dimensional models, failing to overlay planned paths, instrument trajectories, and real-time anatomical structures. Surgeons must mentally construct spatial relationships, increasing operational difficulty and the risk of misjudgment. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention provides a method, device, medium, and program product for dynamically adjusting motion control based on the intraoperative visual environment. The method of this invention increases environmental adaptability and the dynamic adjustment mechanism of the visual field during robot-assisted surgery by introducing factors in the surgical environment, such as tissue deformation and changes in illumination, that cause instability in the surgical field.

[0005] The first aspect of this application discloses a method for dynamically adjusting motion control based on the intraoperative visual environment, the method comprising:

[0006] S101, acquire intraoperative visual images and preset baseline force thresholds based on the tissue type of the surgical site;

[0007] S102, calculate the real-time tissue deformation rate ε based on the intraoperative visual image, calculate the real-time force safety threshold according to the preset basic force threshold and the real-time tissue deformation rate, and match the real-time force safety threshold with the real-time tissue state;

[0008] S103 receives the real-time contact force between the instrument and the tissue, compares the real-time contact force with the real-time force safety threshold, and outputs a damping force command proportional to the real-time contact force if the real-time contact force is less than or equal to the real-time force safety threshold; if the real-time contact force is greater than the real-time force safety threshold, it outputs an emergency braking procedure command.

[0009] In some embodiments, the method further includes:

[0010] S104, acquire the positional deviation of the surgical instrument end effector, tissue deformation rate and ambient light fluctuation value;

[0011] S105: Based on the tissue deformation rate, the device end position deviation, the ambient light fluctuation value, and the shortest distance between the device and the sensitive tissue, select any one of the following modes to execute the control command: continuous trajectory mode, incremental fine-tuning mode, or safety obstacle avoidance mode.

[0012] Optionally, the positional deviation of the surgical instrument end effector is calculated by the distance between the actual positional position of the surgical instrument and the pre-planned positional position.

[0013] Optionally, when the tissue deformation rate, instrument end position deviation, and ambient light fluctuation value meet the first requirement, the continuous trajectory mode is executed; in the continuous trajectory mode, the control command of the optimal path is executed, and the movement speed is controlled at 0.5-1mm / s; optionally, the first requirement is when the tissue deformation rate is ≤5%, the instrument end position deviation is ≤0.2mm, and the ambient light fluctuation is ≤100lux.

[0014] Optionally, when the tissue deformation rate and the device end position deviation meet the second requirement, the incremental fine-tuning mode is executed; in the incremental fine-tuning mode, control commands for multiple fine-tuning steps are executed; optionally, the second requirement is when the tissue deformation rate is >5% or the device end position deviation is >0.2mm.

[0015] Optionally, when the shortest distance between the instrument and the sensitive tissue meets the third requirement, a safety obstacle avoidance mode is executed. In the safety obstacle avoidance mode, the repulsive force between the robotic arm and the sensitive tissue is calculated by the artificial potential field method, and the movement path of the robotic arm is adjusted according to the direction of the repulsive force. At the same time, the radius of motion of the microscopic instrument is limited to ≤5mm and the speed threshold is ≤0.2mm / s to ensure that the instrument is always away from the sensitive tissue.

[0016] Optional, repulsive force = 10 N·mm² / square of the shortest distance between the instrument and the sensitive tissue.

[0017] In some embodiments, the method further includes:

[0018] Calculate the distance between the robotic arm end effector and the AR system's field of view boundary. If the distance is less than or equal to the first threshold, output a program command to adjust the pose of the camera arm end effector, drive the camera arm to move to the new pose, and make the surgical arm end effector return to the center of the AR field of view.

[0019] Using the end effector as the tracking target, the camera focal length and angle are adjusted according to the real-time pose of the end effector to control the tracking error to be less than or equal to the second threshold.

[0020] In some embodiments, the optimal path is generated as follows: based on the preoperative three-dimensional anatomical model and intraoperative visual field images, anatomical structure data including key structural features is extracted and segmented; the anatomical structure data is input into a depth deterministic policy gradient (DDPG) model with surgical time constraints, and the model is guided to iteratively optimize the path through a reward function until the set path optimization objective is met, thereby generating the optimal path;

[0021] Optional key structural features include one or more of the following: gray matter, white matter, cerebral blood vessels, and nerve bundles.

[0022] In some embodiments, the method is performed in a dynamic coordinate system generated by registering key feature points in the preoperative three-dimensional anatomical model and intraoperative visual images.

[0023] Optional methods for constructing a dynamic coordinate system include:

[0024] Obtain a preoperative three-dimensional anatomical model and intraoperative visual field images of the surgical site;

[0025] The intraoperative actual coordinates Pi and model feature point coordinates Qi of A feature points in the intraoperative visual field image and the preoperative three-dimensional anatomical model are extracted respectively, where A is a natural number greater than 0;

[0026] The average registration error between the actual coordinates Pi and the coordinates Qi of the model feature points is calculated. The average registration error is compared with a first threshold. Registration begins when the average registration error is less than or equal to the first threshold. A dynamic coordinate system is generated based on the registered feature points.

[0027] In some embodiments, the emergency braking procedure commands include: commands to control the force feedback device to output a reverse damping force and / or commands to cut off the motion drive signal.

[0028] In some embodiments, the optical flow method is used to calculate the real-time tissue deformation rate ε.

[0029] A second aspect of this application discloses a computer device, comprising: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the above-described method.

[0030] A third aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0031] The fourth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0032] This application has the following beneficial effects:

[0033] 1. This application innovatively discloses a method for dynamically adjusting motion control based on the intraoperative visual environment. This method introduces factors such as tissue deformation and light changes in the surgical environment that cause instability in the surgical field of view, thereby increasing the environmental adaptability and dynamic adjustment mechanism of the field of view during robot-assisted surgery. This can effectively overcome the shortcomings of weak environmental adaptability and visualization capabilities in the prior art.

[0034] This application can effectively overcome the following shortcomings of the existing technology:

[0035] 2. The method disclosed in this application can also effectively overcome the problem of imperfect safety monitoring mechanisms currently existing. Although existing systems can monitor instrument position and contact force, they mostly use fixed safety thresholds and do not consider the impact of intraoperative tissue deformation (such as deformation of brain tissue due to traction) and differences in instrument types (such as the different force tolerances of micro-forceps and electrocoagulation hooks) on the safety threshold. When the contact force between the instrument and the tissue exceeds the threshold, the emergency braking response time is long (more than 200ms in some systems), and it can only be indicated by audible and visual alarms, which cannot intuitively inform the doctor of the risk location, making it easy to miss the best time to avoid risks.

[0036] 3. The method disclosed in this application can also effectively overcome the current problem of insufficient human-computer interaction and collaboration. Existing systems mostly adopt a control mode of "doctor preset parameters + robot passive execution", which cannot capture the doctor's operating intentions in real time and dynamically adjust the motion strategy. For example, when the doctor adjusts the position of the instrument by operating the handle, the system response delay often exceeds 150ms, and it is difficult to match the amplitude of the movement according to the subtle changes in the doctor's operating force and speed, resulting in "operational gaps" in human-computer collaboration, affecting the smoothness and accuracy of the surgery. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the method flow provided in the first aspect of the present invention;

[0039] Figure 2 This is a schematic diagram of a system for dynamically adjusting motion control based on the intraoperative visual environment, provided in the second aspect of the present invention.

[0040] Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention;

[0041] Figure 4This is a schematic diagram of the architecture of an exemplary computing device provided in an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the storage medium provided in an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0044] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Figure 1 This is a flowchart illustrating a method for dynamically adjusting motion control based on the intraoperative visual environment, as provided in an embodiment of the present invention. Specifically, the method includes the following steps:

[0047] S101, acquire intraoperative visual images and preset baseline force thresholds based on the tissue type of the surgical site;

[0048] S102, calculate the real-time tissue deformation rate ε based on the intraoperative visual image, calculate the real-time force safety threshold according to the preset basic force threshold and the real-time tissue deformation rate, and match the real-time force safety threshold with the real-time tissue state;

[0049] In some embodiments, the optical flow method is used to calculate the real-time tissue deformation rate ε.

[0050] S103 receives the real-time contact force between the instrument and the tissue, compares the real-time contact force with the real-time force safety threshold, and outputs a damping force command proportional to the real-time contact force if the real-time contact force is less than or equal to the real-time force safety threshold; if the real-time contact force is greater than the real-time force safety threshold, it outputs an emergency braking procedure command.

[0051] In some embodiments, the emergency braking procedure commands include: commands to control the force feedback device to output a reverse damping force and / or commands to cut off the motion drive signal.

[0052] In some embodiments, the method further includes:

[0053] S104, acquire the positional deviation of the surgical instrument end effector, tissue deformation rate and ambient light fluctuation value;

[0054] In some embodiments, the ambient light fluctuation value is the dynamic manifestation of ambient light intensity under changes in angle, medium, or distance. The two are coupled through physical laws (cosine relationship) and environmental factors. There is a direct mathematical relationship between the ambient light fluctuation value and the ambient light intensity: when light is incident at an angle, the actual illuminance (intensity) will fluctuate with the change of the incident angle, following the cosine theorem (Eθ=E0·cosθ), where θ is the incident angle and E0 is the reference illuminance when incident perpendicularly.

[0055] S105: Based on the tissue deformation rate, the device end position deviation, the ambient light fluctuation value, and the shortest distance between the device and the sensitive tissue, select any one of the following modes to execute the control command: continuous trajectory mode, incremental fine-tuning mode, or safety obstacle avoidance mode.

[0056] Optionally, the positional deviation of the surgical instrument end effector is calculated by the distance between the actual positional position of the surgical instrument and the pre-planned positional position.

[0057] Optionally, when the tissue deformation rate, instrument end position deviation, and ambient light fluctuation value meet the first requirement, the continuous trajectory mode is executed; in the continuous trajectory mode, the control command of the optimal path is executed, and the movement speed is controlled at 0.5-1mm / s; optionally, the first requirement is when the tissue deformation rate is ≤5%, the instrument end position deviation is ≤0.2mm, and the ambient light fluctuation is ≤100lux.

[0058] In some embodiments, the optimal path is generated as follows: based on the preoperative three-dimensional anatomical model and intraoperative visual field images, anatomical structure data including key structural features is extracted and segmented; the anatomical structure data is input into a depth deterministic policy gradient (DDPG) model with surgical time constraints, and the model is guided to iteratively optimize the path through a reward function until the set path optimization objective is met, thereby generating the optimal path;

[0059] Optional key structural features include one or more of the following: gray matter, white matter, cerebral blood vessels, and nerve bundles.

[0060] Optionally, when the tissue deformation rate and the device end position deviation meet the second requirement, the incremental fine-tuning mode is executed; in the incremental fine-tuning mode, control commands for multiple fine-tuning steps are executed; optionally, the second requirement is when the tissue deformation rate is >5% or the device end position deviation is >0.2mm.

[0061] Optionally, when the shortest distance between the instrument and the sensitive tissue meets the third requirement, a safety obstacle avoidance mode is executed. In the safety obstacle avoidance mode (when the electromagnetic positioning module in step 1 detects that the shortest distance between the instrument and the sensitive tissue (cerebral blood vessels, nerve bundles) is ≤3mm), the repulsive force between the robotic arm and the sensitive tissue is calculated by the artificial potential field method (repulsion coefficient / square of the shortest distance between the instrument and the sensitive tissue, with a repulsion coefficient of 10N·mm²). The movement path of the robotic arm is adjusted according to the direction of the repulsive force, while limiting the radius of motion of the microscopic instrument to ≤5mm and the speed threshold to ≤0.2mm / s, to ensure that the instrument is always away from the sensitive tissue.

[0062] Optional, repulsive force = 10 N·mm² / square of the shortest distance between the instrument and the sensitive tissue.

[0063] In some embodiments, the method further includes:

[0064] Calculate the distance between the robotic arm's end effector and the AR system's field of view boundary. If the distance is less than or equal to a first threshold, output a program command to adjust the pose of the camera arm's end effector, driving the camera arm to move to a new pose so that the surgical arm's end effector is once again in the center region of the AR field of view. Here, the AR system's field of view boundary (FOV) refers to the range of virtual content that the AR device can display in the real environment, usually expressed in terms of horizontal and vertical angles.

[0065] Using the end effector as the tracking target, the camera focal length and angle are adjusted according to the real-time pose of the end effector to control the tracking error to be less than or equal to the second threshold.

[0066] In some embodiments, the method is performed in a dynamic coordinate system generated by registering key feature points in the preoperative three-dimensional anatomical model and intraoperative visual images.

[0067] Optional methods for constructing a dynamic coordinate system include:

[0068] Obtain a preoperative three-dimensional anatomical model and intraoperative visual field images of the surgical site;

[0069] In some embodiments, the surgical site herein refers to the surgical area of ​​any animal (e.g., a mammal), including but not limited to humans, non-human primates, rodents, etc., who will become the recipient of a particular treatment. Generally, the terms "subject" and "patient" are used interchangeably herein when referring to a human subject. Preferably, the subject is a human.

[0070] The intraoperative actual coordinates Pi and model feature point coordinates Qi of A feature points in the intraoperative visual field image and the preoperative three-dimensional anatomical model are extracted respectively, where A is a natural number greater than 0. In some specific embodiments, the intraoperative actual coordinates Pi are obtained by: continuously acquiring the three-dimensional coordinates of any feature point M times, and calculating the average value of the three-dimensional coordinates of any feature point as the intraoperative actual coordinates Pi of any feature point, where M is a natural number greater than 0. Optionally, the A feature points are obtained by filtering them as follows: receiving N feature points of the surgical site in the intraoperative visual field image, and performing preprocessing operations such as removing noise points and eliminating outliers on the N feature points to obtain A feature points, where N is a natural number greater than 0, and A is less than N.

[0071] The average registration error between the actual coordinates Pi and the coordinates Qi of the model feature points is calculated. The average registration error is compared with a first threshold. Registration begins when the average registration error is less than or equal to the first threshold. A dynamic coordinate system is generated using the registered feature points as the reference. Optionally, the average registration error can be calculated by: calculating the sum of the Euclidean distances of Pi and Qi to obtain the total registration deviation; or calculating the ratio of the total registration deviation to the total number of feature points to obtain the average registration error.

[0072] In a dynamic coordinate system, the system receives pose change data of the operating handle and the optimal path for the surgical site. Optionally, the pose change data includes the movement speed and rotational angular velocity along the X, Y, and Z axes. The method further includes preprocessing the pose change data by performing Kalman filtering to remove operational jitter interference, resulting in preprocessed pose change data. This preprocessed pose change data is then input into an intent recognition model. The intent recognition model combines pre-set surgical step parameters, including the tumor resection range and vascular protection requirements, with optional operational intent information including: approaching the tumor boundary and fine-tuning the instrument angle.

[0073] The pose change data is input into the intention recognition model to analyze the operator's intention, obtain the intention information, and combine the intention information with the optimal path to generate control instructions to guide the robotic arm.

[0074] Optionally, the method further includes: when the average registration error is greater than the first threshold, repeating steps S102-103 until the average registration error is less than or equal to the first threshold, then starting registration, and generating a dynamic coordinate system based on the registered feature points.

[0075] In some embodiments, the threshold in this document is obtained by training with training set samples. It can be a specific threshold or an interval range. The specific form is not specifically limited in this embodiment.

[0076] In some embodiments, the model in this paper is constructed using conventional methods. The model is iteratively trained based on training samples with relevant parameters; when the trained model reaches the training stopping condition, the trained model is used as the final model.

[0077] Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention, such as... Figure 3 As shown, the device 2000 may include: one or more processors 2010 and one or more memories 2020; wherein the memories store computer-readable code that, when run by the one or more processors, can perform the methods described above.

[0078] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, operations, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor, and can be based on an x86 or ARM architecture.

[0079] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0080] For example, the method or apparatus according to embodiments of this disclosure can also be used by means of Figure 4 The architecture of the computing device 3000 shown is used for implementation. For example... Figure 4As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 4 One or more components in the computing device shown.

[0081] This invention also includes a computer-readable storage medium, such as... Figure 5 The diagram illustrates a storage medium 4000 provided in an embodiment of the present invention. The computer storage medium 4020 stores computer-readable instructions 4010. When the computer-readable instructions 4010 are executed by a processor, the method described above according to embodiments of the present disclosure can be performed. The computer-readable storage medium in the embodiments of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), and Direct Memory Bus Random Access Memory (DR RAM). It should be noted that the memory used in the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0082] This disclosure also provides a computer program product or system, including a computer program that, when executed by a processor, implements the steps of the above-described method, such as... Figure 2 As shown, the computer program product or computer program includes:

[0083] The first data acquisition module 201 is used or configured to acquire intraoperative visual field images and a preset basic force threshold based on the tissue type of the surgical site.

[0084] The real-time force safety threshold calculation module 202 is used or configured to calculate the real-time tissue deformation rate ε based on the intraoperative field of view image, calculate the real-time force safety threshold according to the preset basic force threshold and the real-time tissue deformation rate, and match the real-time force safety threshold with the real-time tissue state.

[0085] The instruction generation module 203 is used or configured to receive the real-time contact force between the instrument and the tissue, compare the real-time contact force with the real-time force safety threshold, and if the real-time contact force is less than or equal to the real-time force safety threshold, output an instruction for a damping force proportional to the real-time contact force; if the real-time contact force is greater than the real-time force safety threshold, output an emergency braking procedure instruction.

[0086] In some embodiments, the system further includes:

[0087] The second data acquisition module is used or configured to acquire surgical instrument end-position deviation, tissue deformation rate, and ambient light fluctuation value.

[0088] The mode selection execution module is used or configured to execute control commands by selecting one of the following modes: continuous trajectory mode, incremental fine-tuning mode, or safety obstacle avoidance mode, based on tissue deformation rate, instrument end-effector pose deviation, ambient light fluctuation value, and the shortest distance between the instrument and sensitive tissue.

[0089] Specifically, the following section uses the clinical application of surgical robots as an example to explain this solution in detail:

[0090] Step 1: Real-time acquisition of surgical environment data via multimodal sensors. These sensors include a high-frame-rate optical camera integrated into the robotic arm end effector, with a frame rate ≥200fps and a resolution ≥1920×1080, used to capture the surgical field of view in real time; a six-dimensional force sensor with a range of 0-50N and an accuracy of ±0.01N, used to detect the contact force and torque between the instrument and tissue; and an electromagnetic positioning module with a positioning accuracy ≤0.3mm, used to track the position and orientation of the instrument end effector in three-dimensional space. The acquired data includes high-precision visual images, force feedback signals, instrument end effector position information, and the three-dimensional spatial coordinates of the patient's anatomical structures.

[0091] Step 2: Based on the optical positioning subsystem and the improved nearest-point iterative ICP algorithm, the feature points of the patient's surgical site are registered with the preoperatively constructed three-dimensional anatomical model to generate a dynamic coordinate system for the surgical area; during the registration process, the average registration error is calculated using formula (1), and the average registration error is controlled to be ≤0.1mm.

[0092] Formula (1): E reg=Total registration deviation / Total number of feature points;

[0093] Among them, E reg The average registration error is expressed in mm; the total registration deviation is the sum of the distances between the three-dimensional coordinates of each feature point acquired by the intraoperative optical positioning subsystem and the corresponding three-dimensional coordinates of the feature point in the three-dimensional anatomical model; the total number of feature points is ≥20.

[0094] Step 3: Path planning is achieved by combining deep learning and reinforcement learning models: First, the Attention U-Net neural network with spatial attention mechanism is used to segment key anatomical structures in the operation, including gray matter, white matter, cerebral blood vessels and nerve bundles; then, the deep deterministic policy gradient (DDPG) model with surgical time constraints is used to optimize the path smoothness and obstacle avoidance ability; at the same time, by collecting the pose change rate of the doctor's operating handle at a sampling frequency ≥100Hz, the doctor's operating intention is fused to generate robot control commands.

[0095] Step 4: Based on the contact force signal collected by the six-dimensional force sensor in Step 1, calculate the real-time force safety threshold using formula (2), and monitor the interaction force between the surgical instrument and the tissue in real time; if the interaction force exceeds the real-time force safety threshold, trigger the force feedback device to complete emergency braking within ≤100ms, and generate a visual alarm through a red warning box in the surgical field of view:

[0096] Formula (2): T F =Basic force threshold * tissue deformation coefficient;

[0097] Among them, T F The real-time force safety threshold is expressed in N; the basic force threshold is determined based on the tissue type of the surgical site: 5 N for gray matter, 3 N for white matter, and 2 N for cerebral blood vessels; the tissue deformation coefficient = 1 - 0.2 × intraoperative real-time tissue deformation rate, with the intraoperative real-time tissue deformation rate ranging from 0 to 0.5.

[0098] Step 5: The planned path in Step 3, the registered key anatomical structures in Step 2, and the real-time motion trajectory of the instruments are superimposed onto the doctor's operating field of view using an augmented reality (AR) system. The AR display delay is ≤50ms. When the distance between the end of the surgical arm and the current AR field of view boundary is ≤5mm, the camera arm pose is replanned using an inverse kinematics algorithm based on the pseudo-inverse of the Jacobian matrix. At the same time, visual servo control is adopted with a control error of ≤0.05mm to achieve smooth tracking of the instruments at the end of the image and ensure full visualization of the surgical arm movement.

[0099] Step 6: Adaptively switch motion control modes based on the real-time tissue deformation rate, instrument end pose deviation, and ambient light changes collected during the operation; the tissue deformation rate is obtained through the deformation detection algorithm of the visual image in Step 1, the instrument end pose deviation is calculated through the dynamic coordinate system in Step 2, and the ambient light changes are obtained through the light intensity sensor of the camera in Step 1.

[0100] Step 7: Before the operation, the operator's identity is verified by fingerprint and iris dual biometric identification. Based on the operator's identity information, the corresponding instrument motion constraint parameters and permission level are automatically loaded: the basic motion parameters are loaded for the chief physician's permission, the parameters with a speed threshold reduced by 10% are loaded for the attending physician's permission, and the parameters with a speed threshold reduced by 20% and a movement radius reduced by 15% are loaded for the resident physician's permission.

[0101] This method effectively overcomes the current problems of insufficient operational permissions and parameter compatibility. Doctors of different professional titles have varying surgical experience and proficiency, but existing systems do not set differentiated motion constraint parameters based on operator qualifications. For example, if resident physicians and chief physicians use the same instrument speed thresholds and activity radii, insufficient operational experience can easily lead to safety hazards. Furthermore, the lack of a strict preoperative identity verification mechanism poses a risk of unauthorized personnel operating the system.

[0102] As an improvement, the DDPG model with surgical time constraints introduced in step 3 has a reward function defined as: R = 0.6 * path deviation coefficient + 0.4 * time compliance coefficient;

[0103] Where R is the reward value for each action, ranging from 0 to 1; Path deviation coefficient = 1 - deviation rate between the current path and the optimal path, where the deviation rate between the current path and the optimal path = distance between the current path and the optimal path / length of the optimal path; Time compliance coefficient = 1 - execution time of the current action / maximum allowed execution time of the action, where the maximum allowed execution time of the action is preset according to the surgical steps, ranging from 1 to 5 seconds.

[0104] As an improvement, the specific triggering condition for the adaptive switching of motion control mode in step 6 is as follows:

[0105] Continuous trajectory mode: Triggered when the tissue deformation rate is ≤5%, the instrument end position deviation is ≤0.2mm, and the ambient light intensity is stable with fluctuations ≤100 lux. It is used to perform continuous surgical operations with a length ≥10mm, and the movement speed is controlled at 0.5-1mm / s.

[0106] Incremental fine-tuning mode: Triggered when tissue deformation rate > 5% or instrument end position deviation > 0.2mm, single fine-tuning step size ≤ 0.05mm, and pause for 500ms after each operation to wait for force feedback confirmation;

[0107] Safety obstacle avoidance mode: When the shortest distance between the instrument and sensitive tissue, including cerebral blood vessels and nerve bundles, is detected to be ≤3mm, it is triggered. The repulsive force between the robotic arm and the sensitive tissue is calculated by the artificial potential field method. The repulsive force = repulsion coefficient / square of the shortest distance between the instrument and the sensitive tissue. The repulsion coefficient is 10N·mm². The movement radius of the microscopic instrument is limited to ≤5mm and the speed threshold is ≤0.2mm / s.

[0108] As an improvement, the electromagnetic positioning module in step 1 adopts a dual-antenna design and eliminates electromagnetic interference from the metal parts of surgical instruments through a multi-source signal fusion algorithm, ensuring that the positioning accuracy remains ≤0.5mm when the metal instruments are close and the distance is ≤20mm.

[0109] As an improvement, the AR system described in step 5 allows doctors to adjust the transparency and display ratio of the overlaid information via voice commands. The voice command recognition accuracy is ≥95%, the transparency adjustment range is 30%-80%, the display ratio adjustment range is 0.8-1.2 times, and the voice command response time is ≤300ms.

[0110] As an improvement, the force feedback device in step 4 uses a linear actuator. During emergency braking, the output reverse damping force is proportional to the force exceeding the threshold, with a proportionality coefficient of 0.8, and the damping force change rate is ≤0.5N / ms, thus avoiding secondary damage to the tissue caused by mechanical impact.

[0111] As an improvement, the improved ICP algorithm in step 2 reduces registration error through a preprocessing step, which includes feature point denoising and outlier removal. The outlier removal adopts a method based on the 3σ criterion: calculate the standard deviation σ of the distance of all feature points, and remove points whose distance exceeds the range of average value - 3σ to average value + 3σ. The average value is the average of the distances of all feature points.

[0112] As an improvement, it also includes an intraoperative data backup step: the surgical environment data, including force signals, pose data, and AR overlay images, is automatically backed up every 5 seconds; the backup data is stored on both a local encrypted hard drive and a cloud backup, with the local encrypted hard drive having an AES-256 encryption level; and it supports real-time retrieval of historical data for comparative analysis during the operation.

[0113] More specifically,

[0114] Multimodal sensor data acquisition operation:

[0115] Sensor Deployment: Multimodal sensors are integrated into the end effector of the surgical robot's robotic arm. The high-frame-rate optical camera uses a CMOS image sensor, with the frame rate adjusted to 200fps and the resolution set to 1920×1080. The lens focal length is adapted to the surgical field of view (covering a surgical area with a diameter of 10-15cm) to ensure real-time capture of detailed images of the surgical site. The six-dimensional force sensor adopts a strain gauge structure with a range set to 0-50N and an accuracy calibrated to ±0.01N. It is installed at the connection end between the instrument and the robotic arm to directly detect the axial force, radial force, and torque when the instrument comes into contact with the tissue. The electromagnetic positioning module adopts a dual-antenna design. The positioning base station is fixed in the operating room at a location away from metal equipment (≥2m from the operating table). The positioning probe is embedded in the end effector of the instrument. The positioning accuracy is adjusted to ≤0.3mm before surgery through a calibration procedure.

[0116] Data acquisition process: After the surgery begins, the sensor synchronous acquisition program is activated. The high frame rate optical camera acquires one frame of surgical field image every 5ms and transmits it to the image processing unit for noise reduction and enhancement preprocessing. The six-dimensional force sensor acquires contact force and torque data every 2ms, converts it into digital signals, and uploads it to the force control unit in real time. The electromagnetic positioning module updates the three-dimensional coordinates (X, Y, Z axis coordinates and attitude angle) of the instrument end every 1ms and matches them with the image coordinates acquired by the optical camera. At the same time, the three-dimensional model of the patient's anatomical structure reconstructed from the preoperative CT / MRI images is combined with the feature point data acquired in real time during the operation to supplement the acquisition of the three-dimensional spatial coordinates of the anatomical structure of the patient's surgical site. All acquired data is uniformly stored in a local real-time database, and the data transmission delay is controlled within ≤10ms.

[0117] Operation for generating dynamic coordinate system of surgical area:

[0118] Feature point acquisition: Using the laser locator of the optical positioning subsystem, at least 20 feature points are marked at the surgical site of the patient (preferably anatomically stable points such as skull landmarks and vascular bifurcation points). The three-dimensional coordinates of each feature point are continuously acquired 5 times by an optical camera, and the average value is taken as the actual intraoperative coordinates of the feature point (denoted as Pi). At the same time, the coordinates of the model feature points corresponding one-to-one with the intraoperative feature points are extracted from the three-dimensional anatomical model constructed before the operation (denoted as Qi).

[0119] Registration Calculation: The improved Intermediate Closest Point Iteration (ICP) algorithm is run. First, the feature points acquired during the operation are preprocessed—noise points are removed by Gaussian filtering (the filter window size is set to 3×3), and outliers are removed using the 3σ criterion (the average distance ¯d and standard deviation σ of all feature points are calculated, and points whose distances exceed the range [¯d-3σ,¯d+3σ] are removed). Then, the average registration error is calculated using the formula Ereg = total registration deviation / total number of feature points (where the total registration deviation is the sum of the Euclidean distances of all Pi and Qi). If Ereg > 0.1 mm, feature points are reacquired and the registration process is repeated until Ereg ≤ 0.1 mm. After registration, a dynamic coordinate system for the surgical area is generated based on the registered feature points. This coordinate system is updated in real time with slight changes in the patient's position during the operation (calibrated every 10 ms) to ensure that the spatial correspondence between the coordinate system and the surgical site is always accurate.

[0120] Path planning and control instruction generation operation

[0121] Anatomical structure segmentation: The preoperative 3D anatomical model and the visual images acquired in real time during the operation are input into the Attention U-Net neural network with spatial attention mechanism. The network input layer receives 512×512 pixel image data. The attention module enhances the feature extraction of key structures such as cerebral blood vessels and nerve bundles. The output layer generates segmentation results labeled gray matter, white matter, cerebral blood vessels and nerve bundles. The segmentation accuracy needs to reach ≥95% to provide anatomically contraindicated areas for path planning.

[0122] Path optimization: The segmented anatomical structure data is loaded into a Depth Deterministic Policy Gradient (DDPG) model with surgical time constraints. During model initialization, the path optimization objectives are set as follows: shortest path length (≥10% shorter than manually planned path), shortest distance between the path and sensitive tissues ≥3mm, and single-step action execution time ≤5s. The model is guided to iteratively optimize the path using a reward function path deviation coefficient and time compliance coefficient R = 0.6 × path deviation coefficient + 0.4 × time compliance coefficient (where path deviation coefficient = 1 - deviation rate between the current path and the optimal path, and time compliance coefficient = 1 - current action execution time / maximum allowed action execution time). The path smoothness (radius of curvature at path inflection points ≥2mm) is verified every 100 iterations until an optimal path that meets the surgical requirements is generated.

[0123] Intent fusion and command generation: A displacement sensor is installed on the doctor's operating handle to collect the handpiece's pose change rate (including the movement speed and rotational angular velocity of the X, Y, and Z axes) at a sampling frequency of 100Hz. After removing operation jitter interference through Kalman filtering, the data is input into the intent recognition model. The model combines the pre-set surgical steps of the doctor (such as the tumor resection range and vascular protection requirements) to analyze the doctor's operating intent (such as "approach the tumor boundary" or "fine-tune the instrument angle"). The intent parameters are then integrated into the optimized path data to generate control commands for the robotic arm (including movement direction, speed, and acceleration). The commands are transmitted to the robotic arm controller in real time via Ethernet with a transmission delay of ≤50ms.

[0124] Real-time force feedback monitoring and emergency braking operation:

[0125] Force safety threshold calculation: A baseline force threshold is preset based on the tissue type of the surgical site—5N for gray matter, 3N for white matter, and 2N for cerebral blood vessels. Using the deformation detection algorithm of the visual image in step 1 (employing optical flow to calculate tissue deformation), the real-time tissue deformation rate ε (range 0-0.5) is obtained and substituted into the formula for the baseline force threshold T. F =Base force threshold × (1 - 0.2 × ε) Calculate the real-time force safety threshold T F The threshold is updated every 50ms to ensure that the threshold matches the organization's real-time status.

[0126] Force monitoring and braking execution: A six-dimensional force sensor collects the contact force between the instrument and the tissue in real time, and compares it with the real-time force safety threshold T. F For comparison, if the contact force ≤ T F If the force feedback device outputs a weak damping force (proportional coefficient 0.3) proportional to the contact force, it provides the doctor with tactile feedback; if the contact force is greater than T... F If the force feedback device outputs a reverse damping force within 100ms (proportional to the force exceeding the threshold, with a proportionality coefficient of 0.8 and a damping force change rate ≤ 0.5N / ms), the robotic arm controller cuts off the motion drive signal, causing the instrument to stop moving immediately. In addition, a red warning box is generated on the AR display interface of the surgical field (covering the contact area exceeding the safety threshold), accompanied by a buzzer (frequency 2000Hz, lasting 1s), reminding the doctor to adjust the operation in time.

[0127] Augmented reality visualization and field of view adjustment operations:

[0128] AR overlay display: The augmented reality (AR) system is activated, and the optimal path planned in step 3 (displayed as a solid green line), the key anatomical structures registered in step 2 (displayed as light yellow for gray matter, light blue for white matter, red for cerebral blood vessels, and purple for nerve bundles), and the instrument movement trajectory tracked in step 1 (displayed as an orange dashed line, retaining the trajectory of the most recent 5 seconds) are overlaid onto the real-time surgical field of view captured by the high frame rate optical camera; the AR display latency is controlled to ≤50ms, the initial transparency of the overlaid information is set to 50%, and the display ratio is set to 1.0x to ensure that the details of the surgical site are not obscured.

[0129] Dynamic Field of View Adjustment: The AR system's field of view monitoring module calculates the distance between the surgical arm tip and the current AR field of view boundary in real time. If the distance is ≤5mm, the camera arm pose adjustment program is immediately initiated—based on the inverse kinematics algorithm of the Jacobian matrix pseudo-inverse, the optimal adjustment angle of the camera arm (X, Y, Z axis rotation angle range -15° to 15°) is calculated, driving the camera arm to move to the new pose, so that the surgical arm tip is once again in the center area of ​​the AR field of view. At the same time, a visual servo control algorithm is used to adjust the camera focal length and angle in real time with the instrument tip as the tracking target, controlling the tracking error to ≤0.05mm, ensuring that there are no blind spots in the field of view throughout the movement of the surgical arm. Doctors can adjust the transparency (30%-80%) and display ratio (0.8-1.2 times) of the overlaid information through voice commands (supporting preset commands such as "increase transparency" and "magnify display", with a recognition accuracy of ≥95%), with a voice command response time of ≤300ms.

[0130] Adaptive motion control mode switching operation:

[0131] Mode trigger condition monitoring: Real-time acquisition of three key parameters—tissue deformation rate obtained through deformation detection algorithm of visual image in step 1, device end pose deviation (distance between actual device pose and planned pose) calculated through dynamic coordinate system in step 2, and ambient light intensity obtained through light intensity sensor of camera in step 1 (sampling frequency 10Hz, calculating light fluctuation value every 1 second); the three parameters are compared with preset thresholds to determine whether the mode switching condition is met.

[0132] Mode switching execution:

[0133] Continuous trajectory mode: When the tissue deformation rate is ≤5%, the device end position deviation is ≤0.2mm, and the ambient light fluctuation is ≤100lux, the system will automatically switch to this mode. In this mode, the robotic arm will move continuously along the path planned in step 3, with the movement speed controlled at 0.5-1mm / s. This mode is suitable for continuous surgical operations with a length of ≥10mm, such as tumor resection and tissue separation. The position is calibrated every 20ms during the movement to ensure that the path deviation is ≤0.1mm.

[0134] Incremental fine-tuning mode: When the tissue deformation rate is >5% or the device end position deviation is >0.2mm, it automatically switches to this mode. In this mode, the single fine-tuning step size of the robotic arm is set to ≤0.05mm. After each fine-tuning step is completed, it pauses for 500ms to wait for the six-dimensional force sensor to feed back the contact force data. After confirming that there are no abnormalities, it will execute the next fine-tuning step. It is suitable for fine operation when the device is close to sensitive structures such as nerve bundles and blood vessels.

[0135] Safety obstacle avoidance mode: When the electromagnetic positioning module in step 1 detects that the shortest distance between the instrument and sensitive tissue (cerebral blood vessels, nerve bundles) is ≤3mm, it automatically switches to this mode. In this mode, the repulsive force between the robotic arm and the sensitive tissue is calculated by the artificial potential field method (repulsive force = 10N·mm² / square of the shortest distance between the instrument and the sensitive tissue). The movement path of the robotic arm is adjusted according to the direction of the repulsive force. At the same time, the radius of motion of the microscopic instrument is limited to ≤5mm and the speed threshold is ≤0.2mm / s to ensure that the instrument is always away from the sensitive tissue.

[0136] Preoperative safety verification and intraoperative data backup procedures:

[0137] Preoperative identity verification and parameter loading: Before the surgery begins, the doctor verifies their identity using the biometric recognition module of the surgical robot. First, the doctor presses their finger onto the fingerprint scanner (500dpi resolution, recognition time ≤1s) and simultaneously aligns it with the iris scanner (recognition distance 15-20cm, recognition accuracy ≥99%). The system compares the doctor's identity information in the database. If the verification is successful, the corresponding instrument motion constraint parameters are loaded according to the doctor's title: Chief physicians are loaded with basic motion parameters (speed threshold 1mm / s, radius of motion 10mm), attending physicians are loaded with parameters with a 10% reduction in speed threshold (0.9mm / s), and resident physicians are loaded with parameters with a 20% reduction in speed threshold (0.8mm / s) and a 15% reduction in radius of motion (8.5mm). If the verification fails (e.g., the fingerprint and iris information do not match), the system locks the operation permission and generates an operation log (recording the verification time and the reason for the failure).

[0138] Intraoperative data backup: The intraoperative data backup program is activated, automatically extracting surgical environment data (including force signals from the six-dimensional force sensor, instrument pose data from the electromagnetic positioning module, overlaid images from the AR system, and motion commands from the robotic arm) from the real-time database every 5 seconds. The data is compressed and stored on a local encrypted hard drive (using the AES-256 encryption algorithm, with the encryption key kept by the surgical supervisor), and simultaneously uploaded to a cloud server via a dedicated network for dual backup. The backup data supports real-time retrieval during the operation—the surgeon can select functions such as "view data from the last minute" and "compare with the preoperative planned path" through the operation interface. The system retrieves historical data within 300ms and displays it in chart form (force signals are displayed as line graphs, and pose data are displayed as 3D trajectory graphs) to assist the surgeon in judging the progress of the operation and adjusting the operation strategy.

[0139] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0140] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

Claims

1. A method for dynamically adjusting motion control based on the intraoperative visual environment, characterized in that, The method includes: S101, acquire intraoperative visual images and preset baseline force thresholds based on the tissue type of the surgical site; S102, calculate the real-time tissue deformation rate during surgery based on the intraoperative visual image, and calculate the real-time force safety threshold according to the preset basic force threshold and the real-time tissue deformation rate, wherein the real-time force safety threshold is matched with the real-time state of the tissue. S103: Receive the real-time contact force between the instrument and the tissue, compare the real-time contact force with the real-time force safety threshold, and if the real-time contact force is less than or equal to the real-time force safety threshold, output a damping force command proportional to the real-time contact force; if the real-time contact force is greater than the real-time force safety threshold, output an emergency braking procedure command.

2. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 1, characterized in that, The method further includes: S104, acquire the positional deviation of the surgical instrument end effector, tissue deformation rate and ambient light fluctuation value; S105, based on the tissue deformation rate, the device end position deviation, the ambient light fluctuation value and the shortest distance between the device and the sensitive tissue, select any one of the continuous trajectory mode, incremental fine-tuning mode or safety obstacle avoidance mode to execute the control command. Optionally, the positional deviation of the surgical instrument end is calculated by the distance between the actual position of the surgical instrument and the pre-planned position. Optionally, when the tissue deformation rate, instrument end-effector pose deviation, and ambient light fluctuation value meet the first requirement, a continuous trajectory mode is executed; in the continuous trajectory mode, the control command for the optimal path is executed to control the movement speed. Optionally, when the tissue deformation rate and the device end pose deviation meet the second requirement, the incremental fine-tuning mode control command with multiple fine-tuning steps is executed. Optionally, when the shortest distance between the instrument and the sensitive tissue meets the third requirement, a safe obstacle avoidance mode is executed; in the safe obstacle avoidance mode, the repulsive force between the robotic arm and the sensitive tissue is calculated by the artificial potential field method, the movement path of the robotic arm is adjusted according to the direction of the repulsive force, and the activity radius and speed threshold of the microscopic instrument are limited to ensure that the instrument is always away from the sensitive tissue.

3. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 1, characterized in that, The method further includes: Calculate the distance between the robotic arm end effector and the AR system's field of view boundary. If the distance is less than or equal to the first threshold, output a program command to adjust the pose of the camera arm end effector, drive the camera arm to move to the new pose, and make the surgical arm end effector return to the center of the AR field of view. Using the end effector as the tracking target, the camera focal length and angle are adjusted according to the real-time pose of the end effector to control the tracking error to be less than or equal to the second threshold.

4. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 2, characterized in that, The optimal path is generated as follows: based on the preoperative three-dimensional anatomical model and intraoperative visual field images, anatomical structure data including key structural features is extracted and segmented; the anatomical structure data is input into a depth deterministic strategy gradient model constrained by surgical time, and the model is guided by a reward function to iteratively optimize the path until the set path optimization objective is met, thereby generating the optimal path.

5. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 1, characterized in that, The method is performed in a dynamic coordinate system generated by registering key feature points in the preoperative three-dimensional anatomical model and intraoperative visual images. Optionally, the method for constructing the dynamic coordinate system includes: Obtain a preoperative three-dimensional anatomical model and intraoperative visual field images of the surgical site; The intraoperative actual coordinates Pi and model feature point coordinates Qi of A feature points in the intraoperative visual field image and the preoperative three-dimensional anatomical model are extracted respectively, where A is a natural number greater than 0; Calculate the average registration error between the actual intraoperative coordinates Pi and the model feature point coordinates Qi, compare the average registration error with a first threshold, and start registration when the average registration error is less than or equal to the first threshold. Use the registered feature points as a reference to generate a dynamic coordinate system.

6. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 1, characterized in that, The emergency braking procedure commands include: commands to control the force feedback device to output reverse damping force and / or commands to cut off the motion drive signal.

7. The method for dynamically adjusting motion control based on the intraoperative visual environment according to claim 1, characterized in that, The real-time deformation rate of tissue was calculated using the optical flow method.

8. A computer device, characterized in that, The device includes: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

Images