A magnetic driving double-capsule robot system and a double-capsule hand-eye cooperation method

Abstract

The application provides a magnetic driving double-capsule robot system and a double-capsule hand-eye cooperation method, and the system comprises an external magnetic driving module, a double-capsule robot module, an electromagnetic positioning module, a teleoperation module and a control unit; the external magnetic driving module comprises double mechanical arms and end driving magnets; the double-capsule robot module comprises an endoscope capsule robot and a working capsule robot, each of which is internally provided with a controlled magnet and a magnetic sensor; the electromagnetic positioning module outputs the global six-degree-of-freedom pose of the double capsule in real time; the control unit generates a control instruction through a closed-loop control algorithm to realize independent magnetic control of the double capsule; after the working capsule enters the field of view of the endoscope capsule, visual servo control is started, the pose of the endoscope capsule is dynamically adjusted, the working capsule is tracked, and a good imaging field of view is provided for the working capsule to carry out operation. The application solves the technical problems that the prior art is difficult to realize independent multi-degree-of-freedom driving of the double capsule, image acquisition and double-capsule operation cooperation, and stable operation field of view.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of magnetic drive and medical device technology, and in particular to a magnetically driven dual-capsule robot system and a dual-capsule hand-eye coordination method. Background Technology

[0002] Traditional cannulation-type endoscopy requires inserting a flexible catheter through the body's natural cavities. This procedure is prone to causing tissue damage and requires a high level of skill from the operator, limiting its widespread application in exploring target cavity environments.

[0003] In recent years, wireless capsule robot technology has made significant progress. Existing magnetically controlled capsule robots mainly use the magnetic field generated by external permanent magnets or electromagnetic coils to drive the capsule robot with embedded magnets to move within the target cavity and collect image data of the cavity wall. However, the structure of a single capsule robot is limited by the small internal space, making it difficult to simultaneously integrate high-resolution imaging components and micro-operation mechanisms. As a result, most existing products only have image acquisition functions and cannot perform preset operations such as sample acquisition while performing imaging tasks.

[0004] To address the issue of functional expansion, the research attempted to use two capsule robots to handle imaging and operational tasks respectively. However, such multi-capsule robot systems generally lack effective coordination mechanisms: without real-time image guidance, the capsule robot undertaking the operational task struggles to accurately reach the target operational area and is prone to detaching from the image's field of view during movement, leading to collaborative operation failure. Furthermore, when using multiple drive magnets to control different capsule robots, spatial coupling interference exists between the magnets, making it difficult to achieve high-precision motion control of the two capsule robots independently and without interference.

[0005] Therefore, there is an urgent need for a magnetically driven dual-capsule robot system and a dual-capsule hand-eye collaboration method that can solve the above-mentioned problems. Summary of the Invention

[0006] The purpose of this invention is to provide a magnetically driven dual-capsule robot system and a dual-capsule hand-eye collaboration method, which solves the technical problems of existing technologies in achieving independent multi-degree-of-freedom drive of dual capsules, image acquisition and dual-capsule collaborative operation, and stable operation field of view.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: This invention provides a magnetically driven dual-capsule robot system, comprising an external magnetic drive module, a dual-capsule robot module, an electromagnetic positioning module, a teleoperation module, and a control unit. The external magnetic drive module includes a first robotic arm, a second robotic arm, a first driving magnet, and a second driving magnet. The first driving magnet is mounted at the end of the first robotic arm, and the second driving magnet is mounted at the end of the second robotic arm. The dual-capsule robot module includes an endoscope capsule robot and a working capsule robot. The endoscope capsule robot has a first controlled magnet and a first magnetic sensor internally sealed and integrated. The working capsule robot has a second controlled magnet and a second magnetic sensor internally sealed and integrated. The electromagnetic positioning module includes a magnetic field emitting coil array, the first magnetic sensor, and the second magnetic sensor. The electromagnetic positioning module is configured to collect and output the global six-degree-of-freedom pose information of the endoscope capsule robot and the working capsule robot in real time. The teleoperation module is used to receive user control commands to control the movement of the endoscope capsule robot and the execution of preset actions by the working capsule robot. The control unit is communicatively connected to the external magnetic drive module, the electromagnetic positioning module, and the teleoperation module. The control unit is configured to: control the magnetic field transmitting coil array to generate a positioning magnetic field; measure the magnetic field signals at the location based on the first and second magnetic sensors, and calculate the global six-degree-of-freedom pose information of the endoscope capsule robot and the working capsule robot according to the magnetic dipole model; establish a magnetic field superposition relationship based on the global six-degree-of-freedom pose information using the magnetic dipole model; and generate control commands for the external magnetic drive module using a closed-loop control algorithm based on the magnetic field superposition relationship to achieve independent magnetic control drive of the endoscope capsule robot and the working capsule robot in the same space. Furthermore, the control unit is also configured to: activate visual servo follow-up control when the working capsule robot enters the image field of view of the endoscope capsule robot, dynamically adjusting the pose of the endoscope capsule robot to keep the working capsule robot and the target working area at the center of the image field of view during the execution of a preset operation.

[0008] Furthermore, the external magnetic drive module is configured to output a multimodal magnetic field to drive the endoscope capsule robot and the working capsule robot to move in at least one of the following modes: sliding, rolling, and oscillating; and the external magnetic drive module can achieve four-degree-of-freedom independent magnetic control of the endoscope capsule robot and the working capsule robot in the same space by regulating the multimodal magnetic field.

[0009] Furthermore, the endoscopic capsule robot also integrates a camera unit, an illumination unit, a battery, and a wireless transmission unit; the camera unit and the illumination unit work together to acquire image data within the target cavity and perform follow-up tracking imaging on the working capsule robot; the battery is used to power the endoscopic capsule robot; the first controlled magnet is used to drive the endoscopic capsule robot to move under the action of an external driving magnetic field; the wireless transmission unit sends the image data to the control unit in real time.

[0010] Furthermore, the working capsule robot also integrates an end effector, a micro motor, and a transmission mechanism; the micro motor drives the end effector through the transmission mechanism, and the end effector is used to perform at least one of sample acquisition, contact operation, or material output.

[0011] Furthermore, the establishment of the magnetic field superposition relationship through the magnetic dipole model includes: the total spatial magnetic field at the location of any capsule robot is formed by the superposition of the magnetic field vectors generated by the first driving magnet, the second driving magnet, and another controlled magnet built into the capsule robot.

[0012] Furthermore, the closed-loop control algorithm generates magnetic drive control commands based on a proportional-integral-derivative (PID) control algorithm, including: calculating the target magnetic field, magnetic force, and / or magnetic torque increment based on the deviation between the desired and actual poses of the endoscopic capsule robot and the working capsule robot, in order to construct a target control vector. Based on magnetically driven Jacobian matrix Through the inverse kinematics formula Calculate the joint motion increments of the first and second robotic arms. and The joint motion increment is then output as a control command from the external magnetic drive module.

[0013] Furthermore, it also includes an external host computer, which is communicatively connected to the control unit, and the external host computer synchronously displays the endoscopic capsule robot's perspective view and global magnetic positioning information view.

[0014] This invention also provides a dual-capsule hand-eye collaboration method for a magnetically driven dual-capsule robot system, characterized by the following steps: S1. Driving an endoscopic capsule robot to move within a target cavity via an external magnetic drive module, the endoscopic capsule robot acquires images of the intracavitary environment and determines the spatial position of the target working area based on the images; S2. Introducing a working capsule robot, acquiring the global six-degree-of-freedom pose information of the endoscopic capsule robot and the working capsule robot based on an electromagnetic positioning module, and providing global navigation for the working capsule robot in conjunction with the spatial position of the target working area; S3. Driving the working capsule robot to move based on the global navigation until the working capsule robot enters the image field of view of the endoscopic capsule robot. S4. Complete visual capture; S5. Initiate visual servo follow-up control of the endoscope capsule robot on the working capsule robot, and maintain the preset relative position of the working capsule robot in the image field of view by adjusting the pose of the endoscope capsule robot; S6. Under the dual guidance of the global six-degree-of-freedom pose information of the electromagnetic positioning module and the image field of view of the endoscope capsule robot, drive the working capsule robot to the spatial position of the target working area, and start the end effector of the working capsule robot to perform a preset operation; S7. During the execution of the preset operation, dynamically adjust the pose of the endoscope capsule robot through visual servo follow-up control, so that the working capsule robot and the target working area are located at the center of the image field of view.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: This invention adopts a dual-capsule robot split-collaboration architecture, which breaks through the technical bottleneck of limited functional integration of single capsule robots. It can simultaneously realize the visualization observation of cavity environment and mechanical operation, providing a complete and feasible technical path for operation in target cavities.

[0016] Based on the spatial magnetic field superposition model and Jacobian matrix framework, this invention overcomes the interactive magnetic interference between multiple magnetic sources, and realizes high-precision independent closed-loop control of the two capsule robots in the same space with four degrees of freedom, thereby improving the decoupling and stability of magnetic drive control. At the same time, a time-sharing excitation strategy is adopted to decouple the signals of the positioning magnetic field and the driving magnetic field, ensuring that the global six degrees of freedom pose of the two capsule robots can be tracked in real time with high frequency, stability and accuracy.

[0017] This invention, through a visual servo dynamic follow-up tracking strategy, enables the endoscopic capsule robot to autonomously adjust its observation angle and continuously lock onto the target working area, solving the technical challenge of hand-eye coordination in unconstrained cavity environments and reducing the complexity of three-dimensional teleoperation of the magnetically driven capsule robot. In addition, the multi-machine decoupling architecture adopted in this invention has high scalability and can effectively support future expansion to multi-robot cluster collaboration modes such as "one master and multiple slaves". Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the magnetically driven dual-capsule robot system provided in this embodiment; Figure 2 This is a flowchart illustrating the dual-capsule hand-eye collaboration method of the magnetically driven dual-capsule robot system provided in this embodiment; Figure 3 This is a schematic diagram of the implementation device for the magnetically driven dual-capsule robot system provided in this embodiment; Figure 4 This is a schematic diagram comparing the planned path and the actual path during actual testing of the magnetically driven dual-capsule robot system provided in this embodiment. Figure 4 (a) is a schematic diagram of the desired path. Figure 4 (b) is a schematic diagram of the actual trajectory; Figure label: 1-External magnetic drive module; 2-Endoscopic capsule robot; 3-Operating capsule robot; 4-Electromagnetic positioning module; 5-Control unit; 6-External host computer; 7-Endoscopic capsule robot view; 8-Global magnetic positioning information view. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0022] This embodiment provides a magnetically driven dual-capsule robot system, such as Figure 1 As shown, it includes an external magnetic drive module 1, a dual-capsule robot module, an electromagnetic positioning module 4, a teleoperation module, and a control unit 5.

[0023] The external magnetic drive module 1 includes a first robotic arm, a second robotic arm, a first driving magnet, and a second driving magnet. The first driving magnet is installed at the end of the first robotic arm, and the second driving magnet is installed at the end of the second robotic arm.

[0024] Specifically, the external magnetic drive module 1 consists of two multi-degree-of-freedom robotic arms and external drive magnets respectively installed at the ends of the robotic arms, which are used to generate a spatially controlled magnetic field in the target space; the first drive magnet and the second drive magnet are both permanent magnets, and their positions can be adjusted by the robotic arms. The working space of the dual capsule robot is in the space outside the drive magnets.

[0025] The dual-capsule robot module includes an endoscope capsule robot 2 and a working capsule robot 3; the endoscope capsule robot 2 has a first controlled magnet and a first magnetic sensor internally sealed and integrated; the working capsule robot 3 has a second controlled magnet and a second magnetic sensor internally sealed and integrated.

[0026] Specifically, the endoscopic capsule robot 2 and the working capsule robot 3 complement each other. The endoscopic capsule robot 2 serves as the visual perception end of the system, used for image exploration and target working area identification, and can adapt to the narrow and complex passage environment inside the gastrointestinal tract. The working capsule robot 3 serves as the physical execution end of the system, used to perform preset tasks such as sample acquisition. Both capsule robots have a controlled magnet sealed inside, which serves as the active controlled object of the system.

[0027] For example, the endoscopic capsule robot 2 and the working capsule robot 3 are medical devices, and their shells are made of biocompatible resin or other high molecular organic materials; the shape of the dual capsule robot is configured as any one of a sphere, an ellipsoid, or a cylindrical end body; the first controlled magnet and the second controlled magnet are both made of neodymium iron boron material, and their shapes are configured as any one of a sphere, an ellipsoid, a cylinder, or a ring.

[0028] like Figure 3 The schematic diagram of the system implementation device shown indicates that each neodymium iron boron magnet driving the magnet has a height of 30mm and a diameter of 30mm, and the dipole moment satisfies: Both the endoscopic capsule robot 2 and the operational capsule robot 3 have a built-in cylindrical N52-grade magnet with a diameter and height of 5mm and a dipole moment that satisfies: The magnet is aligned with the capsule's main axis. Each capsule measures 12mm in diameter and 28mm in height, with a total mass of 2.92g.

[0029] The electromagnetic positioning module 4 includes a magnetic field transmitting coil array, a first magnetic sensor, and a second magnetic sensor. The electromagnetic positioning module 4 is configured to collect and output the global six-degree-of-freedom pose information of the endoscope capsule robot 2 and the working capsule robot 3 in real time.

[0030] Specifically, the magnetic field transmitting coil array generates a positioning magnetic field signal at a preset frequency; the first magnetic sensor and the second magnetic sensor achieve high-precision real-time pose positioning by detecting changes in the external control magnetic field; the electromagnetic positioning module 4 adopts a time-division periodic excitation strategy to decouple the positioning and driving magnetic field signals, providing the system with high-frequency, stable global six-degree-of-freedom pose coordinates for the dual capsule robots. These coordinates can distinguish and independently analyze the spatial states of the two controlled capsule robots, providing pose feedback for global navigation and closed-loop control.

[0031] The remote operation module is used to receive user control commands to control the movement of the endoscope capsule robot 2 and the operation capsule robot 3 to perform preset operations.

[0032] Specifically, the teleoperation module is used to receive the target trajectory and specific operation instructions input by the user, so as to realize the macroscopic and microscopic motion control of the endoscope capsule robot 2 and the working capsule robot 3.

[0033] The control unit 5 is communicatively connected to the external magnetic drive module 1, the electromagnetic positioning module 4, and the teleoperation module, respectively. The control unit 5 is configured to: control the magnetic field transmitting coil array to generate a positioning magnetic field; measure the magnetic field signals at the location based on the first and second magnetic sensors, and calculate the global six-degree-of-freedom pose information of the endoscope capsule robot 2 and the working capsule robot 3 according to the magnetic dipole model; establish a magnetic field superposition relationship based on the global six-degree-of-freedom pose information using the magnetic dipole model; and generate control commands for the external magnetic drive module 1 using a closed-loop control algorithm based on the magnetic field superposition relationship, so as to achieve independent magnetic control drive of the endoscope capsule robot 2 and the working capsule robot 3 in the same space. Furthermore, the control unit 5 is also configured to: activate visual servo follow-up control when the working capsule robot 3 enters the image field of view of the endoscope capsule robot 2, dynamically adjusting the pose of the endoscope capsule robot 2, so that during the execution of a preset operation, the working capsule robot 3 and the target working area remain at the center of the image field of view.

[0034] Specifically, during the magnetic positioning calculation process, the control unit 5 establishes a measurement model based on the magnetic dipole model to establish the relationship between the six-degree-of-freedom pose of the capsule robot and the magnetic field signal generated by its built-in magnet. Using a Kalman filter algorithm, the kinematic model of the capsule robot is used as the state prediction equation, and the measurement model is used as the observation update equation. The motion state prediction information and the measured magnetic field information are fused together, and the six-degree-of-freedom pose of the capsule robot is obtained through iterative estimation. The control unit 5 establishes a closed-loop control algorithm based on the magnetic dipole model and generates motion commands for the first and second robotic arms in real time by solving the magnetic drive inverse kinematic model, thereby achieving precise path tracking and pose adjustment for the dual capsule robots. During system operation, the endoscope capsule robot 2 is first driven by a controllable dynamic magnetic field to complete the cavity environment inspection and target work area calibration within the target cavity. Then, relying on the electromagnetic positioning module 4, the global six-degree-of-freedom pose information of the dual capsules is obtained. Simultaneously, the environmental image data collected in real time by the endoscope capsule robot 2 is fused to build a highly reliable, low-latency multi-machine collaborative closed-loop motion control system. The working capsule robot 3 is driven smoothly to the preset target work area using an independent magnetic control method to complete various preset refined mechanical operations.

[0035] In this embodiment, the external magnetic drive module 1 is configured to output a multimodal magnetic field to drive the endoscope capsule robot 2 and the working capsule robot 3 to move in at least one of the sliding, rolling and oscillating modes, respectively; and the external magnetic drive module 1 achieves four-degree-of-freedom independent magnetic control of the endoscope capsule robot 2 and the working capsule robot 3 in the same space by regulating the multimodal magnetic field.

[0036] Specifically, the external magnetic drive system is configured to output a multimodal spatial magnetic field, drive the capsule robot to switch between different motion modes, and independently control the four degrees of freedom of each of the two capsule robots in the same space, so as to realize the collaborative operation of the two robots.

[0037] For example, the capsule robot is configured to achieve stable suspension in a liquid environment and realize multi-degree-of-freedom motion of translation and steering based on an external controlled magnetic field; or it is configured to realize multi-modal active motion of translation, rolling, swinging and steering on the inner wall surface of a cavity with stick-slip properties through controlled magnetic field force and magnetic torque.

[0038] In this embodiment, it is clarified that the endoscopic capsule robot 2 also integrates a camera unit, an illumination unit, a battery, and a wireless transmission unit; the camera unit and the illumination unit work together to acquire image data within the target cavity and perform follow-up tracking imaging on the working capsule robot 3; the battery is used to power the endoscopic capsule robot 2; the first controlled magnet is used to drive the endoscopic capsule robot 2 to move under the action of an external driving magnetic field; the wireless transmission unit sends the image data to the control unit 5 in real time.

[0039] Specifically, the camera unit integrates a high-definition camera, enabling real-time acquisition of internal environmental images during movement within the target cavity; the lighting unit ensures clear images within the cavity even in the absence of external light sources; and the battery is a button-type micro battery. Under the control of the external magnetic drive module 1, the endoscopic capsule robot 2 can independently perform image acquisition and real-time data transmission tasks within the target cavity.

[0040] In this embodiment, it is clarified that the working capsule robot 3 also integrates an end effector, a micro motor and a transmission mechanism; the micro motor drives the end effector through the transmission mechanism, and the end effector is used to perform at least one of sample acquisition, contact operation or material output.

[0041] Specifically, the working capsule robot 3 also integrates a control circuit board and a button-type micro battery; the second controlled magnet is used to drive the working capsule robot 3 to move under the action of an external driving magnetic field; the micro motor drives the end effector through the transmission mechanism, so that the working capsule robot 3 can accurately reach the target working area under the joint guidance of vision and positioning signals, and perform at least one preset operation among sample acquisition, contact operation or material output.

[0042] In this embodiment, the establishment of the magnetic field superposition relationship through the magnetic dipole model includes: the total spatial magnetic field at the location of any capsule robot is formed by the superposition of the magnetic field vectors generated by the first driving magnet, the second driving magnet, and another controlled magnet built into the capsule robot.

[0043] Specifically, the decoupling mechanism for the multi-magnetic-source spatial coupling interference problem faced by the system in driving a dual-capsule robot within the same space is as follows: Control unit 5 establishes the magnetic field superposition relationship based on the magnetic dipole model and calculates the spatial magnetic field at the location of each capsule robot in real time. In a dual-machine collaborative operation scenario, any capsule robot... The total magnetic field of the location It is not determined by a single magnetic source, but by two externally driven magnets. , And another capsule robot in the same space ( The magnetic field vectors generated by these magnetic fields are superimposed.

[0044] The formula for calculating the superposition relationship is: ,in, and They represent the driving magnets respectively. and In capsule robots The magnetic field generated at that location, Indicates another capsule robot Internal controlled magnet in The magnetic field generated at that location. This superposition relationship quantifies the interactive magnetic interference between multiple magnetic entities in space, providing a precise underlying physical mapping basis for the subsequent independent decoupled control of the dual-capsule robot.

[0045] In this embodiment, the closed-loop control algorithm generates magnetic drive control commands based on the proportional-integral-derivative (PID) control algorithm, including: calculating the target magnetic field, magnetic force, and / or magnetic torque increment based on the deviation between the desired and actual poses of the endoscope capsule robot 2 and the working capsule robot 3, in order to construct a target control vector. Based on magnetically driven Jacobian matrix Through the inverse kinematics formula Calculate the joint motion increments of the first and second robotic arms. and The joint motion increment is then output as a control command from the external magnetic drive module 1.

[0046] Specifically, based on the superposition relationship of magnetic fields, control unit 5 generates precise magnetic drive control commands through a closed-loop control algorithm. The specific steps are as follows: (1) Based on the current actual pose and the desired pose of the dual-capsule robot, calculate the spatial pose deviation, and based on this, deduce the target magnetic force increment and magnetic field deviation used to correct the deviation, and construct the target control vector in the spatial magnetic field domain. .

[0047] (2) Based on the pre-established system magnetic drive Jacobian matrix The calculation formula is obtained through inverse kinematics: The target control vector The solution is the incremental motion of the robotic arm joints corresponding to the two externally driven magnets. and .

[0048] (3) The control unit 5 outputs action commands in real time according to the calculated joint motion increment to control the external magnetic drive module 1, thereby realizing high-precision independent closed-loop control of each of the four degrees of freedom of the dual capsule robot.

[0049] Through the closed-loop architecture combining the above-mentioned magnetic field superposition relationship and inverse kinematic mapping, this embodiment realizes high-frequency dynamic feedback between the mechanical motion of the external entity and the target pose inside the capsule robot cavity, ensuring that the dual capsule robot can achieve high-precision, low-crosstalk independent control of each of the four degrees of freedom even in complex disturbance environments.

[0050] like Figure 4 As shown, this embodiment tested the system's path tracking in an external simulated cavity environment. Figure 4 (a) is a schematic diagram of the expected path, in which the planned path of the endoscope capsule robot 2 (capsule robot 1) is position point 1→2→3→4→1→5→6→4→1; the planned path of the working capsule robot 3 (capsule robot 2) is position point 1'→2'→3'→4'→1'→2'→5'→6'→1'. Figure 4 (b) is a schematic diagram of the actual trajectories. The trajectory on the left corresponds to the actual motion path of capsule robot 1, and the trajectory on the right corresponds to the actual motion path of capsule robot 2. The test results show that the actual motion trajectories of the two capsule robots match their respective planned paths well, verifying the effectiveness and accuracy of the independent closed-loop control of the two capsule robots in this system.

[0051] In this embodiment, the system is further defined as including an external host computer 6, which is communicatively connected to the control unit 5. The external host computer 6 synchronously displays the endoscopic capsule robot's perspective 7 and the global magnetic positioning information view 8.

[0052] Specifically, the external host computer 6 serves as the system's visualization terminal, allowing operators to monitor the intracavitary status in real time; the endoscopic capsule robot perspective 7 is a local image acquired in real time by the endoscopic capsule robot 2 and displayed on the external host computer 6, clearly presenting the intracavitary morphology and details of the target working area, providing intuitive local image guidance for the operation of the working capsule robot 3; the global magnetic positioning information view 8 is a virtual navigation view rendered on the external host computer 6 based on data from the electromagnetic positioning module 4, which globally maps the relative spatial relationship between the two capsules and the target working area within the same virtual coordinate system, providing three-dimensional pre-navigation for the working capsule robot 3 before entering the endoscopic field of view.

[0053] This invention also provides a method for dual-capsule hand-eye coordination in a magnetically driven dual-capsule robot system, such as... Figure 2As shown, the procedure includes the following steps: S1. Driving the endoscopic capsule robot 2 within the target cavity via an external magnetic drive module 1, the endoscopic capsule robot 2 acquires images of the intracavitary environment and determines the spatial location of the target working area based on the images; S2. Introducing the working capsule robot 3, acquiring the global pose information of the endoscopic capsule robot 2 and the working capsule robot 3 based on the electromagnetic positioning module 4, and planning a global magnetic navigation path for the working capsule robot 3 in conjunction with the spatial location of the target working area; S3. Driving the working capsule robot 3 to move based on the global magnetic navigation path until the working capsule robot 3 enters the image field of view of the endoscopic capsule robot 2, completing visual capture; S4. Start. The endoscopic capsule robot 2 uses visual servo tracking control to maintain the working capsule robot 3 at a preset relative position within the image field of view by adjusting the pose of the endoscopic capsule robot 2; S5. Under the dual guidance of the global positioning information of the electromagnetic positioning module 4 and the image field of view of the endoscopic capsule robot 2, the working capsule robot 3 is driven to reach the spatial position of the target working area, and the end effector of the working capsule robot 3 is activated to perform a preset operation; S6. During the execution of the preset operation, the pose of the endoscopic capsule robot 2 is dynamically adjusted by visual servo to ensure that the working capsule robot 3 and the target working area are located at the center of the image field of view.

[0054] Specifically, in S1, deep learning methods are used to analyze the image to locate the three-dimensional spatial position of the target work area. Steps S1 to S6 constitute a complete dual-capsule hand-eye collaboration method. Through a dual guidance mechanism of global magnetic navigation pre-positioning and visual servo precise following, the work capsule achieves autonomous navigation and precise visual operation within the target cavity.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A magnetically driven dual-capsule robot system, comprising an external magnetic drive module (1), a dual-capsule robot module, an electromagnetic positioning module (4), a teleoperation module, and a control unit (5), characterized in that: The external magnetic drive module (1) includes a first mechanical arm, a second mechanical arm, a first drive magnet, and a second drive magnet. The first drive magnet is installed at the end of the first mechanical arm, and the second drive magnet is installed at the end of the second mechanical arm. The dual-capsule robot module includes an endoscope capsule robot (2) and a working capsule robot (3); the endoscope capsule robot (2) has a first controlled magnet and a first magnetic sensor internally sealed and integrated; the working capsule robot (3) has a second controlled magnet and a second magnetic sensor internally sealed and integrated. The electromagnetic positioning module (4) includes a magnetic field transmitting coil array, a first magnetic sensor and a second magnetic sensor. The electromagnetic positioning module (4) is configured to collect and output the global six-degree-of-freedom pose information of the endoscope capsule robot (2) and the working capsule robot (3) in real time. The teleoperation module is used to receive user control commands to control the movement of the endoscope capsule robot (2) and the operation capsule robot (3) to perform preset operations; The control unit (5) is communicatively connected to the external magnetic drive module (1), the electromagnetic positioning module (4), and the teleoperation module, respectively; the control unit (5) is configured as follows: The magnetic field transmitting coil array is controlled to generate a positioning magnetic field; and the magnetic field signals at the location are measured by the first magnetic sensor and the second magnetic sensor respectively, and the global six-degree-of-freedom pose information of the endoscope capsule robot (2) and the working capsule robot (3) is calculated according to the magnetic dipole model. Based on the global six-degree-of-freedom pose information, a magnetic field superposition relationship is established using a magnetic dipole model; Based on the superposition relationship of the magnetic fields, a closed-loop control algorithm is used to generate control commands for the external magnetic drive module (1) so as to realize independent magnetic control drive of the endoscope capsule robot (2) and the working capsule robot (3) in the same space; Furthermore, the control unit (5) is also configured to: when the working capsule robot (3) enters the image field of the endoscope capsule robot (2), activate visual servo follow-up control to dynamically adjust the position and posture of the endoscope capsule robot (2) so that the working capsule robot (3) and the target working area remain in the center of the image field of view during the execution of the preset operation.

2. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, The external magnetic drive module (1) is configured to output a multimodal magnetic field to drive the endoscope capsule robot (2) and the working capsule robot (3) to move in at least one of the sliding, rolling and oscillating modes, respectively; and the external magnetic drive module (1) can achieve four-degree-of-freedom independent magnetic control of the endoscope capsule robot (2) and the working capsule robot (3) in the same space by regulating the multimodal magnetic field.

3. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, The endoscopic capsule robot (2) also integrates a camera unit, a lighting unit, a battery and a wireless transmission unit; The camera unit and the lighting unit work together to acquire image data within the target cavity and perform follow-up tracking imaging on the working capsule robot (3); The battery is used to power the endoscope capsule robot (2); The first controlled magnet is used to drive the endoscope capsule robot (2) to move under the action of an external driving magnetic field; The wireless transmission unit sends the image data to the control unit (5) in real time.

4. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, The working capsule robot (3) also integrates an end effector, a micro motor and a transmission mechanism; the micro motor drives the end effector through the transmission mechanism, and the end effector is used to perform at least one of sample acquisition, contact operation or material output.

5. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, The establishment of the magnetic field superposition relationship through the magnetic dipole model includes: the total spatial magnetic field at the location of any capsule robot is formed by the superposition of the magnetic field vectors generated by the first driving magnet, the second driving magnet, and another controlled magnet built into the capsule robot.

6. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, The closed-loop control algorithm generates magnetic drive control commands based on the proportional-integral-derivative control algorithm, including: Based on the deviation between the desired and actual poses of the endoscopic capsule robot (2) and the working capsule robot (3), the target magnetic field, magnetic force, and / or magnetic torque increments are calculated to construct the target control vector. ; Based on magnetically driven Jacobian matrix Through the inverse kinematics formula Solve for the joint motion increments of the first and second robotic arms. and ; The joint motion increment is output as a control command from the external magnetic drive module (1).

7. The magnetically driven dual-capsule robot system according to claim 1, characterized in that, It also includes an external host computer (6), which is connected to the control unit (5) in communication. The external host computer (6) synchronously displays the endoscopic capsule robot's view (7) and the global magnetic positioning information view (8).

8. A method for hand-eye coordination in a magnetically driven dual-capsule robot system, characterized in that, Includes the following steps: S1. The endoscope capsule robot (2) is driven to move in the target cavity by the external magnetic drive module (1). The endoscope capsule robot (2) collects images of the cavity environment and determines the spatial position of the target working area based on the images. S2. Introduce the working capsule robot (3), and obtain the global six-degree-of-freedom pose information of the endoscope capsule robot (2) and the working capsule robot (3) based on the electromagnetic positioning module (4). Combined with the spatial position of the target working area, provide global navigation for the working capsule robot (3). S3. Drive the operation capsule robot (3) to move based on the global navigation until the operation capsule robot (3) enters the image field of the endoscope capsule robot (2) and completes visual capture; S4. Start the visual servo follow-up control of the endoscope capsule robot (2) on the working capsule robot (3), and maintain the preset relative position of the working capsule robot (3) in the image field of view by adjusting the pose of the endoscope capsule robot (2); S5. Under the dual guidance of the global six-degree-of-freedom pose information of the electromagnetic positioning module (4) and the image field of the endoscope capsule robot (2), the working capsule robot (3) is driven to reach the spatial position of the target working area, and the end effector of the working capsule robot (3) is activated to perform the preset operation. S6. During the execution of the preset operation, the pose of the endoscope capsule robot (2) is dynamically adjusted by visual servo follow control so that the working capsule robot (3) and the target working area are located at the center of the image field of view.

Images