Intelligent cavity operation navigation method and system based on multi-mode perception fusion

Abstract

The invention discloses an intelligent cavity operation navigation method and system based on multi-mode perception fusion, and relates to the technical field of surgical medical treatment. Firstly, bionic instrument posture data and cavity environment data are obtained; then, based on the bionic instrument posture data and the orifice environment data, the target curvature of each section is calculated through a bionic kinematics model, and the magnetorheological fluid pressure is dynamically adjusted to control the local rigidity; meanwhile, the cavity point cloud topological graph is converted into an obstacle density graph, and path weight matrixes of different paths are quantified based on a swarm intelligence algorithm; and finally, according to the target curvature, the local rigidity and the path weight matrix, driving a bionic instrument to execute compliant motion and cooperative obstacle avoidance operation. According to the method, multi-modal data (curvature, rigidity and contact force) are fused through a bionic kinematics model, navigation deviation correction is achieved in combination with a curvature-force nonlinear mapping model and three-dimensional Euclidean distance monitoring, and the navigation precision is improved in combination with a path re-planning mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical medicine, and particularly to an intelligent cavity surgery navigation method and system based on multi-modal perception fusion. Background Art

[0002] Natural orifice minimally invasive surgery (such as transoral, nasal, rectal, etc.) relies on a high-precision navigation system to achieve safe operation of instruments in complex anatomical structures. Existing technologies mainly complete navigation through single-modal perception (such as vision or force feedback) combined with predefined path planning algorithms. Representative systems include Medtronic Hugo RAS, Da Vinci surgical robot, and Intuitive Surgical Ion system.

[0003] However, traditional navigation systems rely on rigid kinematic models and do not consider tissue dynamic deformation, resulting in large navigation errors and multi-instrument collaboration defects: lack of swarm intelligence obstacle avoidance mechanism, and path conflicts are likely to occur during multi-instrument operation, prolonging the operation time. Therefore, there is an urgent need for an intelligent cavity surgery navigation method and system based on multi-modal perception fusion to solve the above problems. Summary of the Invention

[0004] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract, and the title. However, such simplifications or omissions shall not be used to limit the scope of the present invention.

[0005] In view of the problems existing in the above-mentioned prior art, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solution: an intelligent cavity surgery navigation method based on multi-modal perception fusion, the method comprising the following steps:

[0007] First, obtain bionic instrument attitude data and cavity environment data;

[0008] Wherein, the bionic instrument attitude data includes curvature values of each segment collected by a shape memory alloy sensor and local stiffness values feedback by a magnetorheological fluid pressure sensor, and the cavity environment data includes a cavity point cloud topology map generated by millimeter wave scanning and a contact force vector measured by an optical fiber force feedback array;

[0009] Then, based on the bionic instrument attitude data and cavity environment data, solve the target curvature of each segment through a bionic kinematic model, and dynamically adjust the magnetorheological fluid pressure to control the local stiffness;

[0010] Meanwhile, convert the topological map of the cavity point cloud into an obstacle density map (convert the cavity point cloud data into a density value matrix, with the density value unit being points per cubic centimeter), and quantify the path weight matrix of different paths based on the swarm intelligence algorithm;

[0011] Finally, according to the target curvature, local stiffness, and path weight matrix, drive the bionic instrument to perform compliant motion and collaborative obstacle avoidance operations; and real-time monitor the deviation between the actual path and the planned path of the instrument. If the deviation exceeds the preset threshold, trigger path replanning.

[0012] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion of the present invention, wherein: the calculation process of the bionic kinematic model is as follows:

[0013] S101: Establish a curvature-force non-linear mapping relationship according to the curvature values of each segment and the applied external contact force;

[0014] S102: Dynamically calculate the local stiffness adjustment amount based on the magnetorheological fluid pressure value and the preset pressure-stiffness calibration curve;

[0015] S103: Associate the local stiffness adjustment amount with the target contact force threshold through a PID controller to ensure that the contact force is less than the first threshold.

[0016] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion of the present invention, wherein: the establishment of the curvature-force non-linear mapping relationship includes: experimental calibration and mathematical modeling, specifically:

[0017] Among them, the experimental calibration is specifically to calibrate the corresponding relationship between the SMA deformation amount and the contact force under different curvatures through in vitro animal tissue experiments;

[0018] Then perform mathematical modeling to fit the calibration data into a function, with the curvature value as the input of the function and the predicted contact force as the output.

[0019] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion of the present invention, wherein: the implementation of the swarm intelligence algorithm includes:

[0020] S201: Generate an initial pheromone distribution based on the obstacle density map, and set the pheromone attenuation rate τ in the high-density area of the obstacle density map to 2 times that in the low-density area;

[0021] S202: Dynamically allocate path priorities according to the real-time positions of multiple instruments, and preferentially select the path with the lowest pheromone concentration;

[0022] S203: If the path distance between instruments is detected to be less than the set threshold, trigger the asynchronous motion mode, pause and then enter the instrument until the previous instrument passes.

[0023] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion according to the present invention, wherein: the specific implementation of the asynchronous motion mode includes:

[0024] Pause all motion commands for the instrument.

[0025] After the front instrument passes through the conflict area, update the path weight matrix: P new = P old × 1.5, and send a path weight matrix update command to the later-entering instrument;

[0026] The later-entering instrument recalculates the obstacle avoidance path based on the updated weight matrix.

[0027] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion according to the present invention, wherein: the control logic of the PID controller is: calculate the adjustment amount of the magnetorheological fluid pressure in real time;

[0028] Output the pressure adjustment amount according to the PID formula:

[0029]

[0030] Among them, ΔQ represents the adjustment amount of the magnetorheological fluid pressure; e(t) represents the real-time error of the contact force, that is, the difference between the target contact force and the actual contact force; represents the rate of change of the error with time.

[0031] As a preferred solution of the intelligent cavity surgery navigation method based on multi-modal perception fusion according to the present invention, wherein: the trigger condition for path replanning is:

[0032] Calculate the three-dimensional Euclidean distance between the actual path and the planned path of the instrument in real time;

[0033] If the three-dimensional Euclidean distance is greater than 0.2 mm in three consecutive samples, it is determined that the deviation exceeds the limit.

[0034] A system applied to the above intelligent cavity surgery navigation method based on multi-modal perception fusion, the system includes: a multi-modal data acquisition module for obtaining multi-source perception data in real time, including: bionic instrument attitude data and cavity environment data;

[0035] A data preprocessing and fusion module for cleaning, registering and cross-modal aligning the original data;

[0036] A bionic kinematics modeling module for constructing a curvature-force mapping model, establishing a non-linear relationship based on SMA sensor data and external contact force, and adjusting the local stiffness in real time according to the magnetorheological fluid pressure value;

[0037] The swarm intelligence path planning module generates a multi-instrument collaborative obstacle avoidance path to avoid cross-interference.

[0038] The present invention also discloses a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned intelligent endoscopic surgery navigation method based on multi-modal perception fusion are realized.

[0039] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned intelligent endoscopic surgery navigation method based on multi-modal perception fusion are realized.

[0040] Advantages of the present invention:

[0041] 1. The present invention fuses multi-modal data (curvature, stiffness, contact force) through a bionic kinematic model, combines a curvature-force non-linear mapping model and three-dimensional Euclidean distance monitoring to achieve navigation correction, and combines a path replanning mechanism, thereby improving navigation accuracy;

[0042] 2. The present invention dynamically assigns path priorities through an algorithm (the path with the lowest pheromone concentration is prioritized), combines an asynchronous motion mode (pause for 0.5 seconds when the path distance < 1 mm), shortens the multi-instrument operation time, and avoids secondary injuries caused by cross-interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0044] Figure 1 It is a schematic diagram of the overall structure of the intelligent endoscopic surgery navigation method and system based on multi-modal perception fusion proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the drawings of the specification.

[0046] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0047] Second, the "one embodiment" or "embodiment" referred to herein means a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not all refer to the same embodiment, nor is it an individual or alternative embodiment that is mutually exclusive with other embodiments.

[0048] Referring to Figure 1 , an embodiment of the present invention provides an intelligent channel surgery navigation method based on multimodal perception fusion. This method includes the following steps:

[0049] This method includes the following steps:

[0050] Step 1: First, obtain bionic instrument attitude data and channel environment data;

[0051] Among them, the bionic instrument attitude data includes the curvature values of each segment collected by the shape memory alloy sensor and the local stiffness value feedback by the magnetorheological fluid pressure sensor. The channel environment data includes the channel point cloud topology map generated by millimeter-wave scanning and the contact force vector measured by the fiber optic force feedback array.

[0052] Step 2: Based on the bionic instrument attitude data and the channel environment data, calculate the target curvature of each segment through the bionic kinematic model, and dynamically adjust the magnetorheological fluid pressure to control the local stiffness.

[0053] The calculation process of the bionic kinematic model is as follows:

[0054] S101: Establish a curvature-force non-linear mapping relationship according to the curvature values of each segment and the applied external contact force;

[0055] Through experimental calibration (calibrating the correspondence between the SMA deformation amount and the contact force at different curvatures through in vitro animal tissue experiments) and mathematical modeling (fitting as a piecewise polynomial function), the relationship between curvature and contact force is quantified, such as:

[0056] Low curvature segment (0 - 0.4 radian / meter): The contact force and the curvature are linearly related, and the formula is:

[0057] F = k·θ;

[0058] High curvature segment (>0.4 radian / meter): The contact force and the curvature are quadratically related, and the formula is:

[0059] F = aθ 2 + bθ + c;

[0060] Among them, F is the predicted contact force (unit: Newton), θ is the curvature value (unit: radian / meter), and a, b, c, and k are all experimentally calibrated coefficients.

[0061] S102: Dynamically calculate the local stiffness adjustment amount based on the magnetorheological fluid pressure value and the preset pressure-stiffness calibration curve to ensure that the instrument maintains the optimal stiffness balance under different operating scenarios;

[0062] That is, determine the corresponding relationship between pressure and stiffness through experiments, such as:

[0063] When the pressure is 200 kPa, the stiffness is 150 kPa;

[0064] When the pressure is 400 kPa, the stiffness is 300 kPa.

[0065] S103: Associate the local stiffness adjustment amount with the target contact force threshold through a PID controller to ensure that the contact force is less than the first threshold.

[0066] The control logic of the PID controller is: Calculate the adjustment amount of the magnetorheological fluid pressure in real time;

[0067] Output the pressure adjustment amount according to the PID formula:

[0068]

[0069] Among them, ΔQ represents the adjustment amount of the magnetorheological fluid pressure; e(t) represents the real-time error of the contact force, that is, the difference between the target contact force and the actual contact force; represents the rate of change of the error with time (the derivative of the error).

[0070] Generally speaking, the adjusted pressure value is limited within the range of 50 kPa to 500 kPa to prevent tissue damage.

[0071] Step three: Convert the cavity point cloud topology map into an obstacle density map (convert the cavity point cloud data into a density value matrix, and the density value unit is points per cubic centimeter), and quantify the path weight matrix of different paths based on the swarm intelligence algorithm.

[0072] The implementation of the swarm intelligence algorithm includes:

[0073] S201: Generate the initial pheromone distribution based on the obstacle density map, and set the pheromone decay rate τ in the high-density area of the obstacle density map to 2 times that in the low-density area;

[0074] It should be noted that: The cavity point cloud topology map (generated by millimeter-wave scanning, with a resolution of 0.1 mm, and the data format is a three-dimensional coordinate point set {x i , y i , z i}.

[0075] The density calculation process is: Divide the cavity space into 1 cm 3For the cubic grid, count the number of point clouds in each grid, which is defined as the density value D (unit: points per cubic centimeter, points / cm 3 );

[0076] The formula is:

[0077] Where, N represents the total number of point clouds, and δ represents the indicator function (1 when the point falls within the grid, otherwise 0).

[0078] Pheromone distribution rule:

[0079] The initial pheromone concentration P0 is inversely proportional to the density: P0 = 1 / D + ∈.

[0080] ∈: A very small constant (such as 0.001) to prevent division by zero error.

[0081] Decay rate setting:

[0082] High-density area (D ≥ 50 points / cm 3 ): Decay rate τ high = 0.1 / s;

[0083] Low-density area (D < 50 points / cm 3 ): Decay rate τ low = 0.05 / s.

[0084] The purpose is to make the pheromone in the high-density area decay rapidly, reduce its path weight, and guide the instrument to detour.

[0085] S202: Dynamically allocate path priorities according to the real-time positions of multiple instruments, and preferentially select the path with the lowest pheromone concentration (representing the probability weight of the path being selected, dimensionless);

[0086] Specifically: The priority formula is: 1 / P i (t) + ∈;

[0087] Where, P i (t) represents the pheromone concentration (dimensionless) of path i at time t;

[0088] The higher the priority, the greater the probability of the path being selected.

[0089] It should be noted that: The positions of multiple instruments {x i , y i , z i} are collected every 100 ms;

[0090] According to the current position of the instrument, update the pheromone concentration of the paths around it: P i (t + 1) = P i (t) · e-τΔt 。

[0091] Among them, Δt represents the update time interval (0.1 second);

[0092] τ: τ is selected according to the current regional density high or τ low 。

[0093] S203: If the path distance between instruments is detected to be less than the set threshold, trigger the asynchronous motion mode. After pausing, enter the instrument until the previous instrument passes.

[0094] The specific implementation of the asynchronous motion mode includes:

[0095] All motion instructions for entering the instrument after pausing, that is, the conflict detection formula is:

[0096] Among them, d represents the three-dimensional Euclidean distance between the path points of the two instruments (unit: millimeter); the set threshold d t = 1mm; if d < 1mm, the instrument entering later pauses for 0.5 seconds;

[0097] After the previous instrument passes through the conflict area, update the path weight matrix: P new = P old ×1.5 (the pheromone concentration in the conflict area is increased by 50% to guide the instrument entering later to bypass), and send a path weight matrix update instruction to the instrument entering later;

[0098] The instrument entering later recalculates the obstacle avoidance path based on the updated weight matrix.

[0099] Step Four: According to the target curvature, local stiffness, and path weight matrix, drive the bionic instrument to perform compliant motion and cooperative obstacle avoidance operations; and real-time monitor the deviation between the actual path and the planned path of the instrument. If the deviation exceeds the preset threshold, trigger path replanning.

[0100] The trigger condition for path replanning is:

[0101] Real-time calculate the three-dimensional Euclidean distance between the actual path and the planned path of the instrument;

[0102] If the three-dimensional Euclidean distance is greater than 0.2mm in all three consecutive samplings (set based on clinical safety requirements, sampling interval: 50ms (20 times per second)), it is determined that the deviation exceeds the limit;

[0103] For example:

[0104] If the first sampling d1 = 0.22mm, the second d2 = 0.23mm, and the third d3 = 0.21mm, it is determined that the limit is not exceeded;

[0105] If d > 0.2 mm for three consecutive times (such as 0.25 mm, 0.22 mm, 0.23 mm), then re-planning is triggered.

[0106] During re-planning, the latest cavity point cloud topology map and the current instrument pose data are combined to generate a locally optimized path. Here, the RRT* algorithm and the fluid dynamics model are used for local optimization of the path. A brief overview of the algorithm is as follows:

[0107] Algorithm input:

[0108] Starting point: the current position of the instrument.

[0109] End point: the next key point of the original planned path (such as the edge of the lesion).

[0110] Constraints:

[0111] The minimum turning radius of the instrument (3 mm);

[0112] The tissue viscoelasticity parameter (mapped from the preoperative CT value);

[0113] Path exploration and screening:

[0114] RRT* expansion: Randomly generate path nodes, and preferentially select areas with low pheromone concentration (see the path weight matrix) and low resistance.

[0115] Fluid dynamics verification:

[0116] Use the Navier-Stokes equation (existing equation) to simplify the model to calculate the path resistance. If the resistance of a certain path segment is greater than (the safety threshold), then this path is excluded:

[0117] Finally, a new path coordinate sequence that bypasses high-risk areas and conforms to the instrument's movement ability is generated.

[0118] The path after re-planning is updated to the path weight matrix, the pheromone concentration is dynamically adjusted, and the instrument priority is re-allocated according to the new path. If the re-planned path still cannot meet the deviation requirements (such as due to severe deformation of the cavity), then switch to the manual intervention mode.

[0119] This embodiment also discloses an intelligent endoscopic surgery navigation system based on multimodal perception fusion, which applies the above-mentioned intelligent endoscopic surgery navigation method based on multimodal perception fusion, and is characterized in that: the system includes: a multimodal data acquisition module, which is used to obtain multi-source perception data in real time, including: bionic instrument attitude data and endoscopic environment data; a data preprocessing and fusion module, which cleans, registers and aligns cross-modal for the original data; a bionic kinematics modeling module, which constructs a curvature-force mapping model, establishes a non-linear relationship based on the SMA sensor data and the external contact force, and adjusts the local stiffness in real time according to the magnetorheological fluid pressure value; a swarm intelligence path planning module, which generates a multi-instrument collaborative obstacle avoidance path to avoid cross-interference.

[0120] This embodiment also provides a computer device applicable to the case of the intelligent endoscopic surgery navigation method based on multimodal perception fusion, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent endoscopic surgery navigation method based on multimodal perception fusion proposed in the above embodiment.

[0121] This computer device can be a terminal, and this computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of this computer device is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of this computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication) or other technologies. The display screen of this computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of this computer device can be a touch layer covering the display screen, or a button, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0122] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the intelligent channel surgery navigation method based on multi-modal perception fusion as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, magnetic disk or optical disc.

[0123] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An intelligent channel surgery navigation method based on multimodal perception fusion, characterized in that The method includes the following steps: First, obtain the bionic device attitude data and the channel environment data; Among them, the bionic device attitude data includes the curvature values of each segment collected by the shape memory alloy sensor and the local stiffness values feedback by the magnetorheological fluid pressure sensor, and the channel environment data includes the channel point cloud topology map generated by millimeter wave scanning and the contact force vector measured by the fiber optic force feedback array; Then, based on the bionic device attitude data and the channel environment data, solve the target curvature of each segment through the bionic kinematic model, and dynamically adjust the magnetorheological fluid pressure to control the local stiffness; At the same time, convert the channel point cloud topology map into an obstacle density map, and quantify the path weight matrix of different paths based on the swarm intelligence algorithm; Finally, according to the target curvature, local stiffness and path weight matrix, drive the bionic device to perform compliant motion and cooperative obstacle avoidance operations; and monitor the deviation between the actual path and the planned path of the device in real time. If the deviation exceeds the preset threshold, trigger path replanning.

2. The intelligent channel surgery navigation method based on multi-modal perception fusion according to claim 1, characterized in that: The calculation process of the bionic kinematic model is as follows: S101: Establish a curvature-force nonlinear mapping relationship according to the curvature values of each segment and the applied external contact force; S102: Dynamically calculate the local stiffness adjustment amount based on the magnetorheological fluid pressure value and the preset pressure-stiffness calibration curve; S103: Associate the local stiffness adjustment amount with the target contact force threshold through a PID controller to ensure that the contact force is less than the first threshold.

3. The intelligent channel surgery navigation method based on multi-modal perception fusion according to claim 2, characterized in that: The establishment of the curvature-force nonlinear mapping relationship includes: experimental calibration and mathematical modeling. Specifically: Among them, the experimental calibration is specifically to calibrate the corresponding relationship between the SMA deformation amount and the contact force under different curvatures through in vitro animal tissue experiments; Then, perform mathematical modeling to calibrate the data and fit it into a function. The input of the function is the curvature value, and the output is the predicted contact force.

4. The intelligent channel surgery navigation method based on multi-modal perception fusion according to claim 3, characterized in that: The implementation of the swarm intelligence algorithm includes: S201: Generate an initial pheromone distribution based on the obstacle density map, and set the pheromone attenuation rate τ in the high-density area of the obstacle density map to 2 times that in the low-density area; S202: Dynamically allocate path priorities according to the real-time positions of multiple devices, and preferentially select the path with the lowest pheromone concentration; S203: If it is detected that the path distance between devices is less than the set threshold, trigger the asynchronous motion mode, pause and then enter the device until the previous device passes.

5. The intelligent channel surgery navigation method based on multi-modal perception fusion according to claim 4, characterized in that: The specific implementation of the asynchronous motion mode includes: Pause all motion instructions of the device that enters later; After the front instrument passes through the conflict area, update the path weight matrix: P new = P old × 1.5, and send a path weight matrix update instruction to the rear entering instrument; The device that enters later recalculates the obstacle avoidance path based on the updated weight matrix.

6. The intelligent channel surgery navigation method based on multi-modal perception fusion according to claim 5, characterized in that: The control logic of the PID controller is: calculate the adjustment amount of the magnetorheological fluid pressure in real time; Output the pressure adjustment amount according to the PID formula: Among them, ΔQ represents the adjustment amount of the magnetorheological fluid pressure; e(t) represents the real-time error of the contact force, that is, the difference between the target contact force and the actual contact force; represents the change rate of the error over time.

7. The intelligent intracavitary surgical navigation method based on multi-modal perception fusion according to claim 6, characterized in that: The trigger condition for the path replanning is: Calculate the three-dimensional Euclidean distance between the actual path and the planned path of the device in real time; If the three-dimensional Euclidean distance is greater than 0.2 mm in three consecutive samples, it is determined that the deviation exceeds the limit.

8. The intelligent cavity surgery navigation system based on multi-modal perception fusion applies the intelligent cavity surgery navigation method based on multi-modal perception fusion according to any one of the above claims 1-7, and is characterized in that: The system includes: a multi-modal data acquisition module for acquiring multi-source perception data in real time, including: bionic device attitude data and channel environment data; A data preprocessing and fusion module for cleaning, registering and cross-modal aligning the original data; The bionic kinematics modeling module constructs a curvature-force mapping model, establishes a non-linear relationship based on the SMA sensor data and the external contact force, and adjusts the local stiffness in real time according to the magnetorheological fluid pressure value; The swarm intelligence path planning module generates a multi-instrument collaborative obstacle avoidance path to avoid cross interference.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the intelligent intracavitary surgery navigation method based on multi-modal perception fusion according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the intelligent intracavitary surgery navigation method based on multi-modal perception fusion according to any one of claims 1 to 7 are implemented.

Images