Digestive tract catheterization robot navigation method and system based on multimodal information fusion

Abstract

The present invention relates to the field of medical robot technology, and specifically to a digestive tract catheterization robot navigation method and system based on multimodal information fusion; the method comprises: using a multimodal sensor to collect real-time data of a human cavity; preprocessing the collected multimodal data; integrating the multimodal data, extracting cavity environment characteristics, and establishing a three-dimensional model of the human cavity in real time; analyzing the human cavity environment characteristics to generate local and global paths; outputting the path to a catheterization robot controller to drive the catheterization robot, monitoring navigation abnormalities in real time, and collecting and storing the path, sensor status and operation data during the navigation process; the system comprises a data acquisition module, a data processing module, a three-dimensional model establishment module, a path generation module and a data monitoring module; through the above-mentioned manner, comprehensive environmental perception is achieved, navigation accuracy and real-time performance are improved, and high-precision operation requirements in complex clinical environments are met.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a digestive tract catheterization robot navigation method and system based on multimodal information fusion. Background Art

[0002] Currently, gastrointestinal cannulation is widely used in gastroscopy, colonoscopy, and other interventional procedures, providing critical support for clinical diagnosis and treatment. However, traditional cannulation methods rely primarily on the experience of medical staff and are combined with a single type of sensor, such as vision or force perception. These methods are susceptible to factors such as the complexity of the luminal environment, the variability of organ morphology, and physiological differences among patients during operation.

[0003] Although existing catheter placement robotic systems have introduced certain automation technologies, they usually rely on single-modal sensors to obtain environmental information, such as simply using an endoscopic camera to capture the cavity morphology or only using force sensors to monitor contact force. This type of method has limitations in data dimensions and is difficult to fully reflect the detailed characteristics of the cavity environment. In addition, single-modal sensors have large noise and errors, which can easily lead to inaccurate navigation or misjudgment of high-risk areas. Existing technologies have obvious shortcomings in the following aspects:

[0004] 1. Limited environmental perception capabilities: A single sensor cannot comprehensively reflect the geometry, physical properties, and dynamic changes of the cavity, resulting in incomplete navigation data.

[0005] 2. Lack of intelligence in path planning: Most existing systems only support static or local path planning and fail to combine multimodal information for real-time optimization, making it easy for navigation failures or deviations to occur in complex cavity environments.

[0006] 3. Insufficient real-time performance: Due to the low efficiency of information fusion and processing, existing methods are difficult to meet real-time requirements, especially in complex cavity environments.

[0007] In summary, the existing gastrointestinal catheterization technology has problems such as incomplete environmental perception, low navigation accuracy, and poor real-time performance, which makes it difficult to meet the high-precision operation requirements in complex clinical environments. Summary of the Invention

[0008] The purpose of the present invention is to provide a gastrointestinal catheterization robot navigation method and system based on multimodal information fusion, aiming to solve the problems of incomplete environmental perception, low navigation accuracy, and poor real-time performance in the existing gastrointestinal catheterization technology in the prior art, and the technical problem that it is difficult to meet the high-precision operation requirements in complex clinical environments.

[0009] To achieve the above objectives, the present invention adopts a digestive tract catheterization robot navigation method based on multimodal information fusion, comprising the following steps:

[0010] Real-time data collection of human body cavities using multimodal sensors, including visual sensors, force sensors, ultrasonic sensors, and inertial measurement units.

[0011] Preprocess the collected multimodal data;

[0012] Integrate multimodal data, extract cavity environment characteristics, and build a three-dimensional model of the human cavity in real time;

[0013] Analyze the characteristics of the human cavity environment and generate local and global paths;

[0014] The path is output to the tube placement robot controller to drive the tube placement robot, monitor the navigation abnormality status in real time, and collect and store the path, sensor status and operation data during the navigation process.

[0015] Among them, in the step of using multimodal sensors to collect real-time data of human cavities:

[0016] The visual sensor is used to capture image information inside the cavity and identify the cavity shape, texture and dynamic changes;

[0017] The force sensor is used to monitor the contact force and direction between the catheter and the cavity wall to determine whether there is abnormal pressure;

[0018] The ultrasonic sensor is used to obtain depth information and physical properties of cavity tissue;

[0019] The inertial measurement unit is used to record the posture and motion trajectory of the catheter in the cavity and monitor the acceleration and angular velocity.

[0020] Among them, in the step of preprocessing the collected multimodal data:

[0021] Preprocessing methods include noise filtering, time synchronization and coordinate alignment.

[0022] Among them, in the step of integrating multimodal data, extracting cavity environment characteristics, and establishing a three-dimensional model of the human cavity in real time:

[0023] The time features and motion dynamic features are extracted separately, and the fused features are mapped into the three-dimensional geometric structure of the cavity through a three-dimensional modeling algorithm based on voxel representation, the cavity surface mesh is generated, and the three-dimensional model is output.

[0024] Among them, in the step of analyzing the characteristics of the human body cavity environment and generating local and global paths, the process of generating the local path is:

[0025] For each neighbor node n′ of the current node n, calculate its temporary cost g′(n′) = g(n) + c(n,n′); where c(n,n′) is the cost of moving between nodes;

[0026] Define impassable areas and dynamically adjust routes to avoid them;

[0027] Determine changes in the environment ahead and make real-time adjustments.

[0028] Among them, in the step of calculating the temporary cost g′(n′)=g(n)+c(n,n′) for each neighbor node n′ of the current node n:

[0029] If the neighbor node n′ is not in the open list and a better path is found, then f′(n′)=g′(n′)+h(n′), update g(n′) and f(n′), and set the parent node of n′ to n.

[0030] Among them, in the step of calculating the temporary cost g′(n′)=g(n)+c(n,n′) for each neighbor node n′ of the current node n:

[0031] If the neighbor node n′ is not in the open list, add it to the open list.

[0032] Among them, in the step of analyzing the characteristics of the human body cavity environment and generating local and global paths, the process of generating the global path is:

[0033] The human cavity is represented as a weighted undirected graph G(V,E); where V represents the cavity node set, E represents the edge set connecting the nodes, and the edge weight ω(e) represents the travel cost between two nodes.

[0034] Based on the overall human body cavity, the location of the target area, the size of the navigation device and the overall passability of the cavity are determined to generate a global path.

[0035] The present invention also provides a digestive tract catheterization robot navigation system based on multimodal information fusion, comprising a data acquisition module, a data processing module, a three-dimensional model building module, a path generation module and a data monitoring module; wherein:

[0036] The data acquisition module is used to collect real-time data of human cavities using a multimodal sensor;

[0037] The data processing module is used to preprocess the collected multimodal data; wherein the preprocessing method includes noise filtering, time synchronization and coordinate alignment;

[0038] The three-dimensional model building module is used to integrate multimodal data, extract cavity environment characteristics, and build a three-dimensional model of the human cavity in real time;

[0039] The path generation module is used to analyze the characteristics of the human body cavity environment and generate local and global paths;

[0040] The data monitoring module is used to output the path to the tube placement robot controller, drive the tube placement robot, monitor the navigation abnormality status in real time, and collect and store the path, sensor status and operation data during the navigation process.

[0041] The present invention provides a digestive tract catheterization robot navigation method and system based on multimodal information fusion, which collects real-time data on the human cavity by utilizing multimodal sensors; wherein the multimodal sensors include visual sensors, force sensors, ultrasonic sensors and inertial measurement units; preprocesses the collected multimodal data; integrates the multimodal data, extracts cavity environment characteristics, and establishes a three-dimensional model of the human cavity in real time; analyzes the human cavity environment characteristics, and generates local and global paths; outputs the path to the catheterization robot controller, drives the catheterization robot, and monitors navigation abnormalities in real time, collects and stores the path, sensor status and operation data during the navigation process; realizes comprehensive perception of the environment, improves navigation accuracy and real-time performance, and meets the high-precision operation requirements in complex clinical environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 It is a flowchart of the steps of the digestive tract catheterization robot navigation method based on multimodal information fusion of the present invention.

[0044] Figure 2 It is a flow chart of the steps of generating a local path of the present invention.

[0045] Figure 3 It is a flow chart of the steps of generating a global path of the present invention.

[0046] Figure 4 It is a structural principle diagram of the digestive tract catheterization robot navigation system based on multimodal information fusion of the present invention.

[0047] Figure 5 It is a structural principle diagram of the electronic device of the present invention.

[0048] 601-data acquisition module, 602-data processing module, 603-three-dimensional model building module, 604-path generation module, 605-data monitoring module. DETAILED DESCRIPTION

[0049] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this application.

[0050] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0051] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0052] See also Figures 1 to 3 ,in Figure 1 This is a flowchart of the steps of the digestive tract catheterization robot navigation method based on multimodal information fusion. Figure 2 is a flowchart of the steps for generating a local path, Figure 3 It is a flow chart of the steps to generate the global path.

[0053] The present invention provides a digestive tract catheterization robot navigation method based on multimodal information fusion, comprising the following steps:

[0054] S100: Use multimodal sensors to collect real-time data on human cavities; the multimodal sensors include visual sensors, force sensors, ultrasonic sensors and inertial measurement units.

[0055] In this embodiment, the visual sensor is used to capture image information inside the cavity and identify the cavity morphology, texture and dynamic changes; for example, an endoscope camera is used to continuously shoot a video stream inside the cavity and transmit the image data in real time. The image acquisition frequency is dynamically adjusted according to the complexity of the cavity environment to ensure that key features are not missed when moving at high speed; the image data is transmitted in the form of a video stream, and the frame rate is determined by the cavity environment complexity C e Dynamic adjustment, the relationship between the frame rate F can be expressed as:

[0056] F=max(F min ,k·C e ).

[0057] The force sensor is used to monitor the contact force and direction between the catheter and the lumen wall to determine whether there is abnormal pressure. It detects the contact force between the front end of the catheter and the lumen wall and transmits the six-dimensional force / torque data to the processing module F:

[0058] F=[F x ,F y ,F z ,T x ,T y ,T z ] T ;

[0059] By sampling at high frequencies, such as 1kHz, subtle force changes can be captured, facilitating subsequent analysis of potential risk areas.

[0060] The ultrasonic sensor is used to obtain the depth information and physical properties of the cavity tissue. For example, an ultrasonic probe is used to transmit ultrasonic signals, and after receiving the echo, the depth distribution and tissue characteristics of the cavity are calculated to generate tomographic information. The acquisition frequency can be adjusted according to the detection accuracy requirements. For example, the sampling density can be increased in narrow or complex cavities to calculate the depth distribution d(x,y) and physical properties ρ(x,y) of the cavity tissue. The depth d(x,y) is calculated using the ultrasonic propagation time v and the sound speed t:

[0061]

[0062] The inertial measurement unit is used to record the posture and motion trajectory of the catheter in the cavity, and monitor the acceleration and angular velocity, and record the three-dimensional acceleration of the catheter a = [a x ,a y ,a z ] and angular velocity ω=[ω x ,ω y ,ω z ]. Combined with the timestamp t, the motion trajectory and posture of the catheter can be updated in real time through the following integral formula:

[0063]

[0064] where v(t) and r(t) are velocity and position respectively.

[0065] The above real-time integration and storage of multimodal sensors:

[0066] M(t k )={I(t k ),F c (t k ),T c (t k ),d(x,y,t k ),a(t k ),ω(t k )};

[0067] where {I(t k ) is the image data, F c (t k ),T c (t k ) is the force data, d(x,y,t k ) is the ultrasonic depth information, a(t k ),ω(t k ) is the motion information.

[0068] S200: Preprocessing the collected multimodal data; wherein the preprocessing method includes noise filtering, time synchronization and coordinate alignment.

[0069] In this embodiment, noise filtering, time synchronization and coordinate alignment are used to improve data quality and ensure the spatiotemporal consistency of multimodal data. The image data is smoothed using a Gaussian filter. The Gaussian filter is:

[0070]

[0071] A low-pass filter is applied to remove the interference introduced by high-frequency vibration in the force data. The low-pass filter is:

[0072]

[0073] A synchronization method based on hardware timestamps is used to align data from different sensors to a unified time base, and Kalman filtering is applied to the timestamps.

[0074] Through sensor calibration and rigid transformation, the modal data are aligned to a unified reference coordinate system. After the above preprocessing steps, the original multimodal data are converted into a high-quality, temporally and spatially consistent format, which facilitates subsequent feature extraction and modeling analysis.

[0075] S300: Integrate multimodal data, extract cavity environment characteristics, and build a three-dimensional model of the human cavity in real time.

[0076] In this embodiment, time features and motion dynamic features are extracted separately, and the fused features are mapped to the three-dimensional geometric structure of the cavity through a three-dimensional modeling algorithm based on voxel representation, and the cavity surface mesh is generated to output a three-dimensional model.

[0077] Among them, a convolutional neural network is used to extract spatial features in the cavity image. The forward propagation process of the convolutional neural network is as follows:

[0078] y (l) =f(W (l) *x (l-1) +b (l) );

[0079] Among them, y (l) is the output feature of the lth layer, W (l) is the convolution kernel, b (l) is the bias, f is the activation function (such as ReLU), * represents the convolution operation, x (l-1) is the input of the previous layer.

[0080] Force perception and ultrasound time series data processing, using bidirectional long short-term memory network (Bi-LSTM) to extract time-related features.

[0081] Inertial data feature extraction uses a spatiotemporal convolutional network to process acceleration and angular velocity data to capture dynamic motion features.

[0082] The extracted features are fused through a multi-layer fully connected network to form a unified cavity feature representation. The operation of the fusion layer can be expressed as:

[0083] F fusion =f(W fusion ·[F vsion ,F force ,F ultrasound ,F interial ]+b fusion );

[0084] Among them, F fusion is the fused feature representation, F vsion ,F force ,F ultrasound ,F interial are visual, force, ultrasonic and inertial features respectively, W fusion and b fusion are weights and biases, and f is the activation function.

[0085] The fused features are mapped to the 3D geometric structure of the cavity through a voxel-based 3D modeling algorithm. The intensity value of each voxel V(i,j,k) is calculated from the fused features:

[0086] V(i,j,k)=φ(F fusion ).

[0087] The MarchingCube algorithm is used to generate the cavity surface mesh. The algorithm steps are as follows:

[0088] Vertex Position=Interp(V low ,V high ,Threshold);

[0089] Combining the above-mentioned deep learning algorithm and three-dimensional modeling method, the real-time generated cavity model can accurately reflect the spatial form and dynamic changes of the cavity, providing an important basis for subsequent path planning and navigation.

[0090] S400: Analyze the characteristics of the human body cavity environment and generate local and global paths;

[0091] Furthermore, the process of generating a local path is:

[0092] 411: For each neighbor node n′ of the current node n, calculate its temporary cost g′(n′) = g(n) + c(n,n′); where c(n,n′) is the cost of moving between nodes;

[0093] 412: Define inaccessible areas and dynamically adjust the path to avoid them;

[0094] 413: Determine if the environment ahead has changed and make real-time adjustments.

[0095] Furthermore, if the neighbor node n′ is not in the open list and a better path is found, then f′(n′)=g′(n′)+h(n′), g(n′) and f(n′) are updated, and the parent node of n′ is set to n.

[0096] Furthermore, if the neighbor node n' is not in the open list, it is added to the open list.

[0097] Furthermore, the process of generating the global path is:

[0098] 421: Represent the human cavity as a weighted undirected graph G(V,E); where V represents the cavity node set, E represents the edge set connecting the nodes, and the edge weight ω(e) represents the travel cost between two nodes;

[0099] 422: Based on the overall human body cavity, determine the location of the target area, the size of the navigation device, and the overall passability of the cavity to generate a global path.

[0100] In this embodiment, based on the cavity model established in real time, the local path planning module adopts the A* algorithm based on heuristic search to dynamically generate a local optimal path according to the current position of the navigation device and the cavity environment characteristics perceived in real time.

[0101] The goal of local path planning is to minimize the total cost function from the starting point to the target point, f(n) = g(n) + h(n), where g(n) is the cost from the starting point to the current node n, and h(n) is the estimated cost from the current node n to the target node.

[0102] For each neighbor node n' of the current node n, calculate its temporary cost g'(n') = g(n) + c(n, n'), where c(n, n') is the cost of moving between nodes. If the neighbor node is not in the open list, or a better path is found, calculate f'(n') = g'(n') + h(n'). Update g(n') and f(n'), and set n''s parent to n. If n' is not in the open list, add it to the list.

[0103] Based on the cavity model data collected by multimodal sensors, inaccessible areas are defined and the route is dynamically adjusted to avoid these areas. The heuristic estimate h(n) is typically calculated using Euclidean distance or weighted Manhattan distance.

[0104] If the environment in front of the navigation device changes, such as new obstacles appearing or the path being blocked, the D* algorithm is used to adjust the existing path in real time to ensure that the device always moves along the optimal path.

[0105] The global path planning module is based on the overall cavity anatomical model, comprehensively considering the location of the target area, the size of the navigation equipment and the overall passability of the cavity, and uses the Dijkstra algorithm or algorithm based on graph search to generate the global path.

[0106] The human cavity is represented as a weighted undirected graph G(V,E), where V represents the cavity node set, E represents the edge set connecting the nodes, and the edge weight ω(e) represents the travel cost between two nodes.

[0107] During actual navigation, the navigation device may encounter dynamic changes in the cavity or abnormal sensor data. Therefore, when the device detects a blocked path or a change in the target position, the local path planning module generates a new local path and updates the global path to ensure safe and efficient navigation.

[0108] S500: Output the path to the tube placement robot controller, drive the tube placement robot, monitor the navigation abnormality status in real time, collect and store the path, sensor status and operation data during the navigation process.

[0109] In this embodiment, feedback control and motion control strategies are used to drive the catheter placement robot to achieve precise navigation and complete the catheter placement operation; an abnormality detection mechanism is established to monitor high-risk signals during the navigation process in real time, and initiate emergency processing when necessary; the path, sensor status and operation data during the navigation process are collected, displayed and recorded for subsequent analysis and system optimization.

[0110] Corresponding to the aforementioned embodiment of the digestive tract catheterization robot navigation method based on multimodal information fusion, the present application also provides an embodiment of the digestive tract catheterization robot navigation system based on multimodal information fusion.

[0111] Figure 4 This is a block diagram of a digestive tract catheterization robot navigation system based on multimodal information fusion according to an exemplary embodiment. Figure 4 The system may include: a data acquisition module 601, a data processing module 602, a three-dimensional model building module 603, a path generation module 604 and a data monitoring module 605; wherein:

[0112] The data acquisition module 601 is used to collect real-time data of human cavities using a multimodal sensor;

[0113] The data processing module 602 is used to pre-process the collected multimodal data; wherein the pre-processing method includes noise filtering, time synchronization and coordinate alignment;

[0114] The three-dimensional model building module 603 is used to integrate multimodal data, extract cavity environment characteristics, and build a three-dimensional model of the human cavity in real time;

[0115] The path generation module 604 is used to analyze the characteristics of the human body cavity environment and generate local and global paths;

[0116] The data monitoring module 605 is used to output the path to the tube placement robot controller, drive the tube placement robot, monitor the navigation abnormality status in real time, and collect and store the path, sensor status and operation data during the navigation process.

[0117] In this embodiment, the data acquisition module 601 uses a multimodal sensor to collect real-time data on the human cavity; the data processing module 602 preprocesses the collected multimodal data; the preprocessing method includes noise filtering, time synchronization and coordinate alignment; the three-dimensional model establishment module 603 integrates multimodal data, extracts cavity environment characteristics, and establishes a three-dimensional model of the human cavity in real time; the path generation module 604 analyzes the human cavity environment characteristics and generates local and global paths; the data monitoring module 605 outputs the path to the catheter placement robot controller, drives the catheter placement robot, and monitors the navigation abnormality in real time, collects and stores the path, sensor status and operation data during the navigation process; it realizes comprehensive perception of the environment, improves navigation accuracy and real-time performance, and meets the high-precision operation requirements in complex clinical environments.

[0118] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0119] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0120] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned digestive tract catheterization robot navigation method based on multimodal information fusion. Figure 5 As shown in FIG, a hardware structure diagram of a digestive tract catheterization robot navigation system based on multimodal information fusion provided by an embodiment of the present invention is provided, in which any device with data processing capability is located. Figure 5 In addition to the processor, memory, and network interface shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0121] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-mentioned digestive tract catheterization robot navigation method based on multimodal information fusion. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0122] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed in this application.

[0123] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A digestive tract catheterization robot navigation system based on multimodal information fusion, characterized in that: include: Data acquisition module, data processing module, 3D model building module, path generation module and data monitoring module; among which: The data acquisition module is used to collect real-time data of human body cavities using multimodal sensors; wherein the multimodal sensors include visual sensors, force sensors, ultrasonic sensors and inertial measurement units; The data processing module is used to preprocess the collected multimodal data; wherein the preprocessing method includes noise filtering, time synchronization and coordinate alignment; The three-dimensional model building module is used to integrate multimodal data, extract cavity environment features, and build a three-dimensional model of the human cavity in real time. The module extracts time features and motion dynamic features respectively, maps the fused features into the three-dimensional geometric structure of the cavity through a three-dimensional modeling algorithm based on voxel representation, generates a cavity surface mesh, and outputs a three-dimensional model. The path generation module is used to analyze the characteristics of the human body cavity environment and generate local and global paths; The data monitoring module is used to output the path to the tube placement robot controller, drive the tube placement robot, monitor the navigation abnormality status in real time, and collect and store the path, sensor status and operation data during the navigation process.

2. The digestive tract catheterization robot navigation system based on multimodal information fusion according to claim 1, characterized in that: In real-time data collection of human cavities using multimodal sensors: The visual sensor is used to capture image information inside the cavity and identify the cavity shape, texture and dynamic changes; The force sensor is used to monitor the contact force and direction between the catheter and the cavity wall to determine whether there is abnormal pressure; The ultrasonic sensor is used to obtain depth information and physical properties of cavity tissue; The inertial measurement unit is used to record the posture and motion trajectory of the catheter in the cavity and monitor the acceleration and angular velocity.

3. The digestive tract catheterization robot navigation system based on multimodal information fusion according to claim 1, characterized in that: In analyzing the characteristics of the human cavity environment and generating local and global paths, the process of generating the local path is as follows: For each neighbor node of the current node n , calculate its temporary cost ;in is the cost of moving between nodes; Define impassable areas and dynamically adjust routes to avoid them; Determine changes in the environment ahead and make real-time adjustments.

4. The digestive tract catheterization robot navigation system based on multimodal information fusion according to claim 3, characterized in that: For each neighbor node of the current node n , calculate its temporary cost middle: Neighbor nodes appear If it is not in the open list and a better path is found, then ,renew and , and The parent node is set to .

5. The digestive tract catheterization robot navigation system based on multimodal information fusion according to claim 3, characterized in that: For each neighbor node of the current node n , calculate its temporary cost middle: If the neighbor node If it is not in the open list, add it to the open list.

6. The digestive tract catheterization robot navigation system based on multimodal information fusion according to claim 1, characterized in that: In analyzing the characteristics of the human cavity environment and generating local and global paths, the process of generating the global path is as follows: Representing the human body cavity as a weighted undirected graph ;in represents the cavity node set, Represents the edge set connecting the nodes, edge weight Represents the travel cost between two nodes; Based on the overall human body cavity, the location of the target area, the size of the navigation device and the overall passability of the cavity are determined to generate a global path.

Images