Magnetic micro-robot, autonomous navigation system and teaching system

Abstract

The application discloses a magnetic control micro robot, an autonomous navigation system and a teaching system, fixes an ultrasonic probe and a magnetic drive device together at the end of a multi-degree-of-freedom moving mechanism to form a teaching device, adds a controller with an autonomous navigation model based on a deep neural network to form a teaching system, monitors images of the micro robot and an environment where the micro robot is located and pose information of the magnetic drive device in real time during the teaching process, takes the images as input and the pose information as labels, constructs samples, collects multiple samples in real time to form a training set, trains the autonomous navigation model, inputs the images captured by a visual device into the trained autonomous navigation model during the autonomous navigation process, obtains a pose adjustment control instruction, adjusts the pose of the magnetic drive device, and realizes autonomous navigation. The application can realize high-precision control of the magnetic robot outside the body, has strong generality for different application scenarios, and greatly promotes the application of artificial intelligence in the field of magnetic robot control.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medical robotics and deep learning, and relates to a control technology for a magnetically controlled microrobot, specifically a magnetically controlled microrobot, an autonomous navigation system, and a teaching system. It is an autonomous navigation technology for a magnetically controlled micro-spiral robot based on imitation learning. Background Technology

[0002] Cardiovascular and cerebrovascular diseases are among the biggest threats to human life today, causing numerous disabilities and deaths every year. Currently, the most advanced and effective method for treating cardiovascular diseases is minimally invasive vascular interventional therapy, which involves inserting specialized catheters and guidewires into blood vessels under the guidance of medical imaging equipment such as X-rays and ultrasound. During this process, doctors often need to work under X-rays for extended periods, which can cause irreversible damage to the body. Furthermore, manipulating the guidewire requires extreme dexterity and sensitivity, demanding significant experience and skill from the doctor, typically requiring years of specialized training. The rise of microrobot technology offers a possible solution to this problem. Microrobots, capable of moving within complex and confined spaces, show great potential in the medical field, such as in minimally invasive surgery, drug delivery, and in vivo targeted therapy. Compared to traditional minimally invasive surgery, this technology offers advantages such as smaller incisions and faster postoperative recovery. Moreover, microrobots can be precisely controlled remotely to avoid radiation damage and operational errors. Despite numerous achievements in the academic field in recent years, few have been truly applied clinically. The control of microvascular robots has always been a significant challenge. Current control methods rely on visual feedback for information and manual operation. While this avoids radiation exposure during real-time operation, it demands a high level of remote operation skills from doctors. However, the development of artificial intelligence in recent years has offered a glimmer of hope for solving this problem. Neural networks, as a rapidly evolving AI tool, can learn from massive amounts of data, addressing the challenges of high-dimensionality, redundancy, and noise in big data—problems that traditional methods struggle with—and demonstrating enormous potential across various fields. Summary of the Invention

[0003] In view of the shortcomings of the above-mentioned technologies, the purpose of this invention is to provide an autonomous navigation method for a magnetically controlled microrobot. The method involves planning the trajectory and controlling the motion of the rotating permanent magnet actuator connected to the end effector of the robotic arm based on image data, and importing the collected posture data of the robotic arm joints and end effector, as well as the trajectory data, into the established autonomous navigation model for training, thereby realizing autonomous navigation control based on imitation learning.

[0004] Another objective of this invention is to provide a teaching system for a magnetically controlled microrobot, which teaches the magnetically controlled microrobot to perform autonomous navigation after teaching.

[0005] Another objective of this invention is to provide a magnetically controlled microrobot system with autonomous navigation capabilities. After being taught using the aforementioned teaching system, an autonomous navigation model is established, and the autonomous navigation model is used to drive the magnetically controlled microrobot to perform autonomous navigation movements.

[0006] The technical solution of this invention is as follows:

[0007] On one hand, an autonomous navigation method for a magnetically controlled microrobot employs a multi-axis robotic arm carrying a magnetic drive device to control the movement of the microrobot, and captures image information of the environment in which the microrobot is located through a vision device, including the following steps:

[0008] Establish an autonomous navigation model based on deep neural networks;

[0009] The microrobot is taught by manually moving the magnetic drive device to control its movement. During the teaching process, the image information of the microrobot and its environment and the pose information of the magnetic drive device are monitored in real time.

[0010] Using images as input and pose information as labels, samples are constructed, and multiple samples are collected in real time to form a training set for training the autonomous navigation model.

[0011] During autonomous navigation, images of the micro-robot and its environment are captured by vision devices and input into the trained autonomous navigation model to obtain the pose change information of the magnetic drive device at the next moment. Based on the pose change information, pose adjustment control commands for the magnetic drive device are generated to control the multi-axis robotic arm to adjust its pose.

[0012] This invention employs manual teaching, using experienced physicians or operators to manipulate the magnetic drive device to obtain the optimal movement mode of the magnetically controlled microrobot within the body for corresponding detection or examination. During the teaching process, images of the magnetically controlled microrobot and the pose information (or pose change information) of the magnetic drive device are acquired in real time. The pose information (or pose change information) of the magnetic drive device is used as a label to tag the images, constructing training samples to train an autonomous navigation model based on a deep neural network. After training, the trained autonomous navigation model serves as the main controller. In use, images of the microrobot's environment are captured using visual equipment and input into the trained autonomous navigation model. The autonomous navigation model outputs the pose information (or pose change information) of the magnetic drive device, which is then used to control the pose changes of the magnetic drive device to achieve autonomous navigation. During autonomous navigation, the magnetically controlled microrobot can carry detection equipment for corresponding detection or precisely release drug capsules.

[0013] Preferably, the visual device includes imaging devices such as ultrasound imaging devices, infrared imaging devices, and CT devices.

[0014] Furthermore, the vision device is an ultrasonic imaging device, and the ultrasonic probe and magnetic drive device of the ultrasonic imaging device are fixed together at the end of the multi-axis robotic arm.

[0015] Preferably, the magnetic drive device includes a housing, a permanent magnet disposed within the housing, and a rotary drive device, wherein the permanent magnet is installed at the output end of the rotary drive device; the microrobot is a helical robot, which drives the permanent magnet to rotate through the rotary drive device, thereby driving the microrobot to move forward in a helical motion.

[0016] Furthermore, the housing is made using 3D printing. To facilitate installation and assembly, it consists of an upper housing and a lower housing connected by connectors. The upper housing is provided with a connecting flange for installation at the end of a multi-axis robotic arm.

[0017] Furthermore, the rotation drive device is a motor, which is installed inside the housing through a connector. The permanent magnet is installed on the output shaft of the motor. The rotation of the motor drives the permanent magnet to rotate, so that the magnetically controlled microrobot rotates synchronously. During the rotation of the magnetically controlled microrobot, the helical surface generates propulsion in the liquid environment.

[0018] Furthermore, the ultrasonic probe is fixed to the outer shell by a bracket, specifically the upper shell. With this fixing method, the ultrasonic probe can be synchronized with the magnetic drive device, enabling timely acquisition of images of the magnetically controlled microrobot.

[0019] Furthermore, the support can be a telescopic support, preferably an elastic passive telescopic support, so that the distance between the magnetic drive device and the magnetically controlled microrobot can be adjusted to obtain the best driving efficiency while ensuring the quality of ultrasound imaging.

[0020] Furthermore, during the teaching process, the mobile environment of the microrobot can be the real environment inside a human or animal, or it can move inside a simulated body. During the teaching process, the handheld magnetic drive device is moved, and the multi-axis robotic arm adopts passive follow-up. The main purpose is to calculate the pose information of the magnetic drive device in real time. The pose information mainly includes three-dimensional coordinates and attitude information, and the attitude information is the inclination angle in three-dimensional space.

[0021] Furthermore, the microrobot can move within human organs (or organ phantoms) to perform corresponding inspections, and it can also move within complex liquid pipelines to conduct inspections.

[0022] Furthermore, a world coordinate system O-XYZ is established to record the coordinates of the magnetic drive device. The pose change information can be represented as the translation and rotation on the three coordinate axes X, Y, and Z.

[0023] Furthermore, the autonomous navigation model is a neural network designed based on a residual network. The neural network sequentially includes a first convolutional layer, a first pooling layer, a first residual block, a second residual block, a first transition layer, a third residual block, a second transition layer, a fourth residual block, a second pooling layer, a first fully connected layer, and a second fully connected layer. The four residual blocks are identical, each including two convolutional layers, one normalization layer, and one activation layer. The input to each residual block and its output after passing through that residual block are combined to form the output of that layer before entering the next layer. The two transition layers are also identical, each including one convolutional layer, one normalization layer, and one activation layer.

[0024] The image captured by the vision device is used as input. After the image is processed by the first convolutional layer, the process is executed sequentially. Finally, the pose change information is output through the second fully connected layer.

[0025] Furthermore, the multi-degree-of-freedom movement mechanism can be a multi-axis robotic arm, such as, but not limited to, a six-axis robotic arm.

[0026] Furthermore, the number of samples in the training set shall not be less than 50.

[0027] Furthermore, during the teaching process, the teaching trajectories are classified, with at least 50 samples collected for each trajectory to ensure good generalization of the final training effect, enabling it to adapt to any given trajectory. For example, for movement within blood vessels, the trajectories are divided into upper limb veins, lower limb veins, head and neck veins, etc. At least 50 samples are collected during the teaching process for each vein. Of course, the number of samples should be determined according to the complexity of the vein, which is not closely related to the technical problem solved by this invention, and therefore will not be elaborated further.

[0028] Furthermore, during the training and navigation process of the autonomous navigation model, image data can be used after data augmentation preprocessing to increase recognition accuracy. The data augmentation preprocessing includes changes to image contrast, saturation, and brightness, and can also improve feature extraction accuracy by adding some noise interference, such as Gaussian noise.

[0029] Furthermore, the navigation process of the autonomous navigation model also includes a real-time adjustment strategy. If the image acquired by the vision device does not meet the adjustment expectation of the previous step, the image will be re-input into the autonomous navigation model and new pose adjustment information will be output. The pose of the magnetic drive device will be adjusted by the new position adjustment information to drive the magnetically controlled microrobot to reach the predetermined position, forming a real-time adjustment closed-loop control to avoid the situation where the magnetically controlled microrobot does not follow the movement of the magnetic drive device.

[0030] On the other hand, the present invention provides a teaching system for a magnetically controlled microrobot, including...

[0031] Multi-degree-of-freedom movement mechanism;

[0032] Miniature robots are objects that can be driven by magnetic force;

[0033] A magnetic drive device is installed at the moving end of a multi-degree-of-freedom mobile mechanism. It can drive a micro-robot to move by magnetic force under the carrying of the multi-degree-of-freedom mobile mechanism. The pose information of the magnetic drive device can be calculated by the kinematics or coordinate system of the multi-degree-of-freedom mobile mechanism.

[0034] Vision devices, installed at the end of multi-degree-of-freedom mobile mechanisms, are used to synchronously acquire images of the microrobot and its environment to obtain the microrobot's posture and path information.

[0035] The teaching controller stores an autonomous navigation model based on a deep neural network and communicates with a multi-degree-of-freedom mobile mechanism and a vision device. During the teaching process, images of the micro-robot and its environment are captured by the vision device as input, and the pose information of the magnetic drive device is used as a label to train the autonomous navigation model.

[0036] The teaching system can teach the pose changes of the magnetic drive device, and associate the image information and pose changes of the microrobot with a neural network model to establish an autonomous navigation model based on a deep neural network. Subsequently, the magnetic drive device can be autonomously navigated and driven using this autonomous navigation model.

[0037] On the other hand, the present invention also provides a magnetically controlled microrobot system with autonomous navigation function, including...

[0038] Multi-degree-of-freedom movement mechanism;

[0039] Miniature robots are objects that can be driven by magnetic force;

[0040] A magnetic drive device is installed at the moving end of a multi-degree-of-freedom mobile mechanism. It can drive a micro-robot to move by magnetic force under the carrying of the multi-degree-of-freedom mobile mechanism. The pose information of the magnetic drive device can be calculated by the kinematics or coordinate system of the multi-degree-of-freedom mobile mechanism.

[0041] Vision devices, installed on the mobile end effector of a multi-degree-of-freedom mobile mechanism, are used to synchronously acquire images of the microrobot and its environment to obtain the microrobot's posture and path information.

[0042] The controller stores the autonomous navigation model trained by the teaching system and is connected to the multi-degree-of-freedom mobile mechanism and the vision device. During use, the images of the micro-robot and its environment captured by the vision device are input into the autonomous navigation model, and the pose change information is output. The controller calculates control commands based on the position deformation information and controls the movement of the multi-degree-of-freedom mobile mechanism to perform autonomous navigation.

[0043] It should be noted that the magnetically controlled microrobot system and the teaching system of the present invention can use the same multi-degree-of-freedom movement mechanism or different multi-degree-of-freedom movement mechanisms, because the present invention ultimately adjusts the posture of the magnetic drive device, and at least one of the multi-degree-of-freedom movement mechanisms has an automatic driving means.

[0044] Compared with corresponding technologies, the advantages of the present invention are:

[0045] By employing imitation learning, the training control system can automatically plan the optimal autonomous navigation path based on the recognized image information and expert experience in the training data. This avoids operational errors caused by insufficient experience and skill during human operation. It achieves highly precise control of the movement of a helical magnetically controlled robot inside the human body from outside the body.

[0046] The autonomous navigation model based on deep neural networks constructed in this invention can achieve high-precision recognition of the movement path of magnetically controlled microrobots in environments such as blood vessels, achieving a highly autonomous navigation level. This allows experienced physicians to widely apply artificial intelligence, greatly improving examination efficiency. Attached Figure Description

[0047] Figure 1 This is a system flowchart of the present invention.

[0048] Figure 2 This is an overview diagram of the data acquisition process of the present invention.

[0049] Figure 3 This is a schematic diagram of the teaching system of the present invention.

[0050] Figure 4 This is a schematic diagram of the teaching device of the present invention.

[0051] Figure 5 This is a schematic diagram of the internal structure of the magnetic drive device of the present invention.

[0052] Figure 6 This is a schematic diagram of the miniature helical magnetically controlled robot of the present invention.

[0053] Figure 7 This is a diagram of the neural network structure of the present invention.

[0054] Figure 8 This is a flowchart of the training set creation and data augmentation process of the present invention.

[0055] Figure 9 This is a flowchart of the autonomous navigation closed-loop control of the present invention.

[0056] 1-Six-DOF robotic arm, 2-Magnetic drive unit, 21-Upper housing, 22-Lower housing, 23-Connector, 24-Motor, 25-Permanent magnet, 26-Flange, 3-Ultrasonic probe, 4-Support, 5-Controller. Detailed Implementation

[0057] To more clearly illustrate the present invention, the following detailed description, in conjunction with the accompanying drawings and an example of a magnetically controlled micro-spiral robot moving in a blood vessel, provides a method for autonomous navigation of a magnetically controlled micro-robot. The described embodiments are for illustrative purposes only and are not intended to limit the invention. Although the present invention has been described in detail with reference to embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the present invention do not depart from the spirit and scope of the present invention and should be covered within the scope of the claims of the present invention.

[0058] like Figure 1The diagram shows a system flowchart of an autonomous navigation method for a magnetically controlled microrobot according to the present invention. In minimally invasive interventional surgery, surgeons often rely on medical imaging and specialized manipulation to control guidewires or microcapsule robots within the patient's body. This invention proposes using imitation learning to learn from the operational data of professional surgeons, resulting in an autonomous control navigation method that incorporates the techniques of professional surgeons. First, a sampling set is established. This invention mounts an ultrasound probe and a magnetic drive device together at the end effector of a six-axis robotic arm to form a teaching device. The teaching device is connected to a controller to control a built-in autonomous navigation model based on a deep neural network, thus forming a teaching system. This invention employs manual teaching, where a human directly drags the teaching device and operates the magnetically controlled microrobot based on visual feedback provided by the ultrasound probe. The entire process requires the collection of data including images captured by the ultrasound probe and all parameters during the movement of the six-axis robotic arm. After sampling, a corresponding autonomous navigation model needs to be established to learn the control methods of manual operation. First, the image data in the sampling set needs to be manually labeled, with the label being the pose change of the magnetic drive device, to obtain a training set. Then, the autonomous navigation model is established, using a deep neural network in this invention. Given that the control objective is to learn human operation and acquire an autonomous navigation model, the training set images are processed through a neural network to obtain the predicted movements of the robotic arm, which are then compared with the recorded movements of the robotic arm and represented by a loss function. The autonomous navigation model is then updated until training is complete. Once the autonomous navigation model is trained, real-time adjustment strategies can be deployed. Throughout the process, the motion state of the microrobot and information about its surrounding environment are captured by an ultrasonic probe. Control commands are then obtained using the previously trained autonomous navigation model. These commands are sent to the robotic arm controller, which in turn drives the magnetic drive device at the end effector, using changes in the magnetic field to pull the micro-magnetically controlled robot to the target position.

[0059] like Figure 2The diagram shows a flowchart of the data acquisition process in this invention. In the actual clinical application of magnetically controlled robots, technicians or doctors typically perform surgical operations by holding the magnetic drive device and relying on visual feedback from the ultrasound probe. First, a location near the target robot is found as the initial position of the magnetic drive device (or a world coordinate system can be established, utilizing the kinematics of the six-axis robotic arm itself to obtain the position and orientation information of the magnetic drive device). It is sufficient to ensure that the target robot is within the visual range of the ultrasound probe. For example, the ultrasound probe and the magnetic drive device can be fixed together. Then, the teaching system is activated, and the position of the magnetic drive device (or teaching device) is adjusted in real time based on the image information of the miniature magnetically controlled robot in the visual feedback. During this process, it is necessary to record the changes in the robotic arm's pose when manually holding the magnetic drive device (or teaching device). It is also necessary to record the image information captured by the ultrasound probe at the corresponding moment.

[0060] like Figure 3 The diagram shown illustrates the overall structure of the teaching system in this invention. It mainly includes a six-degree-of-freedom robotic arm 1, a controller 5, and a teaching device. The six-degree-of-freedom robotic arm 1 controls the pose transformation of the teaching device within a large workspace. The controller 5 is communicatively connected to the teaching device and the six-degree-of-freedom robotic arm 1, used to collect data and transmit control commands. The controller 5 has a built-in autonomous navigation model. After training the autonomous navigation model through the teaching process, the built-in autonomous navigation model in the controller 5 can control the teaching device module to perform autonomous navigation. Figure 4 The diagram shown is of the teaching device. The teaching device includes a magnetic drive unit and an ultrasonic probe mounted on the magnetic drive unit via a bracket 4. A schematic diagram of the internal structure of the magnetic drive unit is shown below. Figure 5 As shown, the magnetic drive unit has a 3D-printed outer shell. The upper shell 21 has a flange 26 that connects to the end of the robotic arm. The lower shell 22 is used to fix the motor 24 and the permanent magnet 25. The motor 24 and the permanent magnet 25 are connected by a connector 23. The permanent magnet does not contact the outer shell; starting the motor 24 causes the permanent magnet to rotate. An ultrasonic probe 27 is also fixed to the upper shell 21 via a bracket 22. During manual teaching, the hands hold the outer shell of the teaching device. For example, the bracket 4 is an elastic telescopic bracket in the axial direction of the flange 26, allowing adjustment of the distance between the permanent magnet 25 and the magnetically controlled microrobot while maintaining high-quality ultrasonic imaging to achieve optimal driving efficiency.

[0061] like Figure 6The diagram shows a schematic of the magnetically controlled microrobot of this invention. It mainly includes a helical tail and a head for housing internal magnets. In a liquid environment, the rotation of the permanent magnet 25 drives the magnetically controlled microrobot to rotate synchronously, thereby obtaining forward propulsion. Changes in the direction of the permanent magnet 25 synchronously change the direction of the driven microrobot. For the magnetic control principle, please refer to the applicant's prior application CN115429199A. Under the combined action of the magnets inside the magnetically controlled microrobot and the magnets in the magnetic drive device, the magnetically controlled microrobot can synchronously follow the rotation and attitude changes of the permanent magnet 25.

[0062] like Figure 7The diagram shows the structure of the neural network in this invention. The network structure is based on a residual network design. The neural network sequentially includes a first convolutional layer, a first pooling layer, a first residual block, a second residual block, a first transition layer, a third residual block, a second transition layer, a fourth residual block, a second pooling layer, a first fully connected layer, and a second fully connected layer. The four residual blocks are identical, each including two convolutional layers, one normalization layer, and one activation layer. The input and output of each residual block are combined to form the output of the next layer. The two transition layers are also identical, each including one convolutional layer, one normalization layer, and one activation layer. Specifically, the image recorded by the ultrasound probe is directly used as the input layer. The input image then passes through a 7x7 first convolutional layer with 64 channels and a span of 2, which performs initial feature extraction. After the first convolutional layer, a first pooling layer is applied to downsample the spatial dimensions. This first pooling layer is a 3x3 kernel with a span of 2 for max pooling. Next are four identical residual blocks, each consisting of two 3x3 convolutional layers, one normalization layer, and one activation layer. The normalization layer uses Batch Normalization, and the activation layer uses the ReLU activation function. A transition layer is placed after the second and third residual blocks (the first transition layer follows the second residual block, and the second transition layer follows the third residual block); both transition layers have the same structure, consisting of one 1x1 convolutional layer, one normalization layer, and one activation layer. The purpose of the transition layer is to adjust the size and channel size of the feature map. The core of this network is that each time the output of the previous layer is input to a residual block, it is simultaneously output and combined with the output of the next residual block. This combination involves adding the two vectors together (i.e., adding the input and output vectors of each transition layer) and using the result as the input to the next layer. This aims to prevent gradient vanishing or exploding and promote gradient flow during network training. After the final fourth residual block, an average pooling operation of a second pooling layer with a 3x3 kernel is used to aggregate spatial information. The output of the second pooling layer is flattened and connected to a first fully connected layer with 1024 units. After passing through the second fully connected layer, a 1x6 tensor is obtained, representing the changes in the x, y, and z directions (1x3) and rotation angle (1x3) in the spatial coordinate system (for simplicity, the base coordinate system (O-XYZ) of a six-axis robotic arm can be used as the world coordinate system to record the spatial coordinates of the magnetic drive device), which serve as the output of the neural network.

[0063] like Figure 8The diagram shows the flowchart of training set creation and data augmentation in this invention. After collecting the data from manual teaching, the data first needs to be manually labeled, indicating the motion state of the robotic arm's end effector relative to the previous moment for each ultrasound image, i.e., the displacement and rotation changes (or corresponding poses) in the x, y, and z directions. After obtaining the training set, data augmentation is required to increase the generalization ability of the neural network. Specific data augmentation methods include modifying the image's contrast, brightness, and color temperature, as well as adding high-speed blur, Gaussian noise, and region culling. Finally, the data-augmented input image is used for network training.

[0064] like Figure 9 The diagram shows the flowchart of the autonomous navigation closed-loop control of this invention. First, an ultrasonic probe captures the position, motion, and surrounding environment information of the magnetically controlled microrobot, and inputs the ultrasonic images into the autonomous navigation model. The model, trained with a large amount of artificial data, outputs a corresponding control command (pose information or position change information for the next moment) based on the image. This means the neural network predicts the displacement and rotation changes of the robotic arm's end effector in the x, y, and z directions based on the input ultrasonic image. This is then combined with the robotic arm pose information from the robotic arm controller. After fusing the control command and pose information, the robotic arm's motion trajectory can be planned. Under the control of the robotic arm, the teaching device performs corresponding pose changes, and the rotating magnetic field also changes accordingly. This drives the micro-spiral robot to swim towards the target direction. The entire process continues in a loop until the microrobot reaches the target position.

[0065] The above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Although the invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the invention do not depart from the spirit and scope of the invention and should be covered within the scope of the claims of the invention.

Claims

1. An autonomous navigation method for a magnetically controlled microrobot, comprising employing a multi-degree-of-freedom mobile mechanism carrying a magnetic drive device to control the movement of the microrobot, and capturing image information of the environment in which the microrobot is located through a vision device, characterized in that, Includes the following steps: Establish an autonomous navigation model based on deep neural networks; The microrobot is taught by manually moving the magnetic drive device to control its movement. During the teaching process, the image information of the microrobot and its environment and the pose information of the magnetic drive device are monitored in real time. Using images as input and pose information as labels, samples are constructed, and multiple samples are collected in real time to form a training set for training the autonomous navigation model. During autonomous navigation, images of the micro-robot and its environment are captured by a vision device and input into the trained autonomous navigation model to obtain the pose change information of the magnetic drive device at the next moment. Based on the pose change information, pose adjustment control commands for the magnetic drive device are generated to control the multi-degree-of-freedom mobile mechanism to adjust its pose. The autonomous navigation model is a neural network designed based on residual networks. The neural network sequentially includes a first convolutional layer, a first pooling layer, a first residual block, a second residual block, a first transition layer, a third residual block, a second transition layer, a fourth residual block, a second pooling layer, a first fully connected layer, and a second fully connected layer. The four residual blocks are identical, each including two convolutional layers, one normalization layer, and one activation layer. The input to each residual block and its output are combined to form the output of the next layer. The two transition layers are also identical, each including one convolutional layer, one normalization layer, and one activation layer. The image captured by the vision device is used as input. After the image is processed by the first convolutional layer, the process is executed sequentially. Finally, the pose change information is output through the second fully connected layer.

2. The autonomous navigation method for a magnetically controlled microrobot according to claim 1, characterized in that, The visual device is an ultrasound imaging device or an X-ray imaging device.

3. The autonomous navigation method for a magnetically controlled microrobot according to claim 2, characterized in that, The vision device is an ultrasonic imaging device, and the ultrasonic probe and magnetic drive of the ultrasonic imaging device are fixed together at the end of the multi-degree-of-freedom moving mechanism.

4. The autonomous navigation method for a magnetically controlled microrobot according to claim 3, characterized in that, The magnetic drive device includes a housing, a permanent magnet disposed within the housing, and a rotary drive device. The permanent magnet is installed at the output end of the rotary drive device. The microrobot is a helical robot, which drives the permanent magnet to rotate through the rotary drive device, thereby driving the microrobot to move forward in a helical motion.

5. The autonomous navigation method for a magnetically controlled microrobot according to claim 1, characterized in that, During the teaching process, the microrobot moves within the phantom body.

6. A teaching system for a magnetically controlled microrobot, characterized in that, include Multi-degree-of-freedom movement mechanism; Miniature robots are objects that can be driven by magnetic force; A magnetic drive device is installed at the moving end of a multi-degree-of-freedom mobile mechanism. It can drive a micro-robot to move by magnetic force under the carrying of the multi-degree-of-freedom mobile mechanism. The pose information of the magnetic drive device can be calculated by the kinematics or coordinate system of the multi-degree-of-freedom mobile mechanism. Vision devices, installed on the mobile end effector of a multi-degree-of-freedom mobile mechanism, are used to synchronously acquire images of the microrobot and its environment to obtain the microrobot's posture and path information. The teaching controller stores an autonomous navigation model based on a deep neural network and communicates with a multi-degree-of-freedom mobile mechanism and a vision device. During the teaching process, images of the micro-robot and its environment captured by the vision device are used as input, and the pose information of the magnetic drive device is used as a label to train the autonomous navigation model. The autonomous navigation model is a neural network designed based on residual networks. The neural network sequentially includes a first convolutional layer, a first pooling layer, a first residual block, a second residual block, a first transition layer, a third residual block, a second transition layer, a fourth residual block, a second pooling layer, a first fully connected layer, and a second fully connected layer. The four residual blocks are identical, each including two convolutional layers, one normalization layer, and one activation layer. The input to each residual block and its output are combined to form the output of the next layer. The two transition layers are also identical, each including one convolutional layer, one normalization layer, and one activation layer. The image captured by the vision device is used as input. After the image is processed by the first convolutional layer, the process is executed sequentially. Finally, the pose change information is output through the second fully connected layer.

7. A magnetically controlled microrobot system with autonomous navigation function, characterized in that, include Multi-degree-of-freedom movement mechanism; Miniature robots are objects that can be driven by magnetic force; A magnetic drive device is installed at the moving end of a multi-degree-of-freedom mobile mechanism. It can drive a micro-robot to move by magnetic force under the carrying of the multi-degree-of-freedom mobile mechanism. The pose information of the magnetic drive device can be calculated by the kinematics or coordinate system of the multi-degree-of-freedom mobile mechanism. Vision devices, installed on the mobile end effector of a multi-degree-of-freedom mobile mechanism, are used to synchronously acquire images of the microrobot and its environment to obtain the microrobot's posture and path information. The controller stores an autonomous navigation model trained by the teaching system described in claim 6 and is communicatively connected to the multi-degree-of-freedom mobile mechanism and the vision device. During use, images of the micro-robot and its environment captured by the vision device are input into the autonomous navigation model, and pose change information is output. Control commands are calculated based on the position deformation information to control the movement of the multi-degree-of-freedom mobile mechanism for autonomous navigation.

Images