Electromagnetic driving system

Abstract

The invention provides an electromagnetic driving system. In one embodiment, the electromagnetic drive system comprises: a) one or more magnetic robots located within a workspace; b) three independent mechanical arms, wherein each independent mechanical arm comprises a movable electromagnetic coil; c) a positioning system for tracking the one or more magnetic robots; the three independent mechanical arms are controlled through multi-objective optimization to ensure a coil configuration space capable of providing a magnetic field for manipulating the one or more magnetic robots in the working space.

Description

Technical Field

[0001] The present invention generally relates to electromagnetic drive systems, and more particularly to a robot-assisted reconfigurable electromagnetic drive system with a human-scale workspace. Background Art

[0002] In recent years, the development of small magnetic robots has attracted increasing attention. Due to their ability to navigate in confined spaces, they have shown revolutionary potential in biomedical research and are suitable for a variety of applications including targeted drug delivery, in vivo cell transport, and minimally invasive treatments [1]–

[10] . These magnetic robots derive their driving force from an external magnetic field generated by an electromagnetic drive system. Therefore, the magnetic robot can be regarded as the end effector of a larger robotic system that includes an external magnetic field source.

[0003] At present, the development of magnetic drive systems can be divided into two categories according to the different magnetic field sources: electromagnetic drive (EMA) systems and permanent magnet drive (PMA) systems [5],

[11] ,

[12] . This paper focuses on the EMA system with a cylindrical electromagnetic coil (hereinafter referred to as "coil"). The advantages and limitations of the above two types of systems have been studied and verified based on their characteristics

[13] –

[18] . The research results show that different magnetic field sources have their own performance indicators, and the choice of system should depend on the actual application scenario. For example, in scenarios where the magnetic field intensity needs to change rapidly, or even the magnetic field can be turned on or off freely, or there is high-frequency magnetic field modulation (such as torque-based control in spiral swimming robots), the EMA system becomes the first choice due to its relatively superior characteristics. In addition, compared with the EMA system, the PMA system is more suitable for applications with heat-sensitive or large-scale magnetic fields, although its control frequency is relatively low.

[0004] Given the advantages and disadvantages of the two types of magnetic field sources, magnetic actuation systems have been widely designed and developed over the past two decades, and can be mainly divided into fixed systems and mobile systems

[19] . Generally speaking, traditional fixed EMA systems equipped with coils are usually used to manipulate microscale devices under microscopes, such as the OctoMag system

[20] and its subsequent expansion systems

[13] ,

[21] ,

[22] . The advantages of such systems are high precision and diverse magnetic field patterns. However, these systems have certain limitations, such as relatively limited working space, poor accessibility, and rapid attenuation of magnetic field strength with increasing distance. In addition, the expansion of the coil may also lead to increased inductive impedance and resistive heating problems, the latter of which needs to be alleviated by a cooling system

[23] . Therefore, building a fixed EMA system with a human-scale working space remains a major challenge.

[0005] Recent research has focused on developing mobile magnetic drive systems with larger workspaces to absorb the advantages of fixed magnetic drive systems and overcome their disadvantages. In particular, since robotic arms can provide a larger workspace, several studies have explored mobile systems controlled by robots. Mahoney and Abbott proposed a robotically controlled single permanent magnet system that uses a six-degree-of-freedom robotic arm to control a five-degree-of-freedom capsule model, demonstrating a higher level of magnetic control capability

[24] . Pittiglio et al.

[25] proposed a PMA system with dual permanent magnets that is driven by two robotic arms to control the directional magnetic field and five independent gradients in three-dimensional space, although the system may produce unexpected transient behavior in Cartesian space and does not consider the singular configuration problem of the robotic arm. However, these mobile PMA systems may perform poorly in tasks requiring high-frequency magnetic fields or uniform magnetic fields, or in scenarios where the magnetic field needs to be turned off throughout the workspace. Another approach is to design a mobile EMA system equipped with mobile coils to achieve faster response speed and magnetic field switching capability, and to expand the workspace without increasing the coil size. In previous studies, mobile EMA systems, such as BigMag

[26] and DeltaMag

[27] , used coupled mobile coils to provide a relatively large workspace. To independently control the coil posture and avoid obstacles that may exist in the workspace, an EMA system, RoboMag, was developed. This system uses three mobile coils that are controlled in a decoupled manner by a desktop robot system

[28] . The system has an adjustable hemispherical workspace with a diameter of 203 mm and is mainly used for magnetic torque control.

[0006] The above analysis of existing magnetic drive systems reveals that a trade-off exists between torque generation, driving force generation, and workspace accessibility in the design of magnetic drive systems. The goal of the present invention is to design a magnetic drive system that not only generates a rotating magnetic field but also enables combined control of the magnetic field and magnetic force through five-degree-of-freedom control within a human-scale workspace. Furthermore, the safety of the entire magnetic manipulation process should be ensured. Based on the goals and requirements of the intended application, the following design requirements should be considered during system development: 1) The designed system should leave sufficient space around the patient to meet the potential space requirements for integration with other medical equipment in the operating room. 2) The generated magnetic field should be able to be shut down promptly when necessary to ensure operational safety. Furthermore, it is essential to be able to provide a wide range of controllable magnetic fields. 3) The feasibility of magnetic field and gradient generation should be considered for driving different types of magnetic robots, such as spiral robots that rely on high-frequency rotating magnetic fields and capsule robots based on force control.

[0007] In view of the above analysis, the mobile EMA system is the preferred solution to meet the requirements. As described in the literature

[29] , under the premise of the presence of non-magnetic restoring force, the minimum number of coils required to achieve five-degree-of-freedom heading and position control is five. In fact, designing a mobile system with five coils is already extremely challenging, not to mention that it also needs to provide a workspace that meets clinical needs. Therefore, it would be of great significance if a mobile EMA system with flexible coil configuration and the minimum number of coils can be realized in a human-scale workspace. Specifically, three linearly independent coils can work simultaneously by simply modulating the current in each coil to generate the required three-dimensional magnetic field and ensuring good magnetic field isotropy in a small target area [9]. In addition, the three-coil configuration can significantly improve the openness to the workspace, thereby better meeting the design requirements. However, two key issues still need to be further explored and resolved: At a higher level, how to construct a low-drive solution to achieve an ideal magnetic field and corresponding fine magnetic force control? At a lower level, how to design a new mobile system to achieve dexterous magnetic manipulation in a human-scale workspace?

[0008] To address the challenges encountered by existing research, this paper draws on the design concepts of the RoboMag system to propose a novel system for controlling magnetic fields and magnetic forces within a human-scale workspace. The significant advancements of this system are primarily attributed to the decoupled three-dimensional spatial motion of three coils mounted on three independent six-degree-of-freedom robotic arms, as well as the proposed magnetic manipulation control method. The main contributions of this paper can be summarized as follows: 1) This paper proposes a novel design, modeling, and control method for a robot-assisted electromagnetic actuation system. This system, under a novel control strategy, features reconfigurable coil configurations and is suitable for use in a human-scale workspace. Furthermore, this paper proposes a software framework and simulation environment based on the Robot Operating System (ROS) that significantly improves algorithm development efficiency and provides excellent visualization support for surgeons. Furthermore, this paper develops automated magnetic field calibration software for single coils to improve calibration accuracy. This open-source software is available online at: http: / / github.com / caimingxue / TRO_RARE_code. 2) This paper proposes a performance-guided optimization method for generating highly isotropic rotating magnetic fields. Unlike existing control strategies, this method optimizes coil configuration based on specific application scenarios to provide an isotropic magnetic field. 3) Unlike existing mobile EMA systems that primarily rely on rotating magnetic field control, this invention proposes a magnetic field-prioritized magnetic force control method that can manipulate a combination of magnetic field and magnetic force to achieve five-degree-of-freedom control. 4) By successfully manipulating a spiral robot and a capsule model in different scenarios, this invention demonstrates a feasible and dexterous EMA system with performance comparable to or even superior to existing mobile EMA systems.

[0009] The proposed method is expected to provide a basis for the development of a flexible, high-performance mobile EMA system that can meet the requirements of a human-scale workspace and is suitable for clinical applications in operating rooms. Summary of the Invention

[0010] The present invention provides an electromagnetic drive system. In one embodiment, the electromagnetic drive system comprises: a) one or more magnetic robots positioned within a workspace; b) three independent robotic arms, each comprising a movable electromagnetic coil; and c) a positioning system for tracking the one or more magnetic robots. The three independent robotic arms are controlled through multi-objective optimization to ensure a coil configuration space capable of providing a magnetic field for manipulating the one or more magnetic robots within the workspace.

[0011] The present invention also provides a method for using the electromagnetic drive system of the present invention. In one embodiment, the method comprises the following steps: a) inserting the one or more magnetic robots into a cavity located within the workspace; and b) providing parameters to the system to obtain a coil configuration space for manipulating the one or more magnetic robots to perform a desired motion.

[0012] The present invention and its features will be understood in more detail from the following description and the accompanying illustrative drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The overall framework of the Reconfigurable Electromagnetic Actuation System (RARE) is presented. RARE is a magnetic actuation system designed to achieve autonomous magnetic manipulation within a human-scale workspace. (a) Simulation environment developed based on the ROS architecture. (b) The actual RARE system developed by this invention. The system consists of three mobile coils, each driven by three independent six-degree-of-freedom robotic arms, thus achieving dexterous magnetic manipulation within a human-scale workspace. In addition, the system is equipped with a low-level controller STM32 and three coil drivers to control the current flowing into the coils. (c) Schematic diagram of the hardware flow. (d) Software flow architecture diagram. (e) Graphical user interface (GUI) of the magnetic field / magnetic force control unit. (f) Image processing results of the perception unit. (g) Graphical user interface (GUI) of the robotic arm motion and planning unit.

[0014] Figure 2 A visualization window displays. This window contains information about the magnetic robot's motion, the arm's posture, and the phantom used for the experiment. It also clearly displays the magnetic field distribution for researchers.

[0015] Figure 3 Demonstration of the automated acquisition of magnetic field data readings from a magnetic sensor using a robotic arm (see Video S2). (a) GUI for acquiring magnetic field data. (b) Filtering the magnetic field data readings from the magnetic sensor. (c) Verification of the linear relationship between magnetic field and current.

[0016] Figure 4 The definition of the world coordinate system and coil coordinate system used to describe the coil configuration is shown. The posture of the coil coordinate system is composed of five elements in the world coordinate system: express.

[0017] Figure 5 A brief process of the performance-guided optimization method for coil configuration design is presented.

[0018] Figure 6 Demonstrate the convergence of the optimization process by computing hypervolume analysis.

[0019] Figure 7 Comparisons of two randomly sampled coil configurations and the optimized coil configuration across various evaluation metrics are presented. Coil configurations #1 and #2 were randomly sampled from the coil configuration space. Evaluation metrics for the three coil configurations were then calculated. Current consumption was also calculated by summing the currents collected at five rotation angles of a given rotating magnetic field. Without loss of generality, this paper was implemented and evaluated for four different rotating magnetic field directions n.

[0020] Figure 8 Demonstrates the NSGA-II algorithm for solving multi-objective optimization (MOO) problems.

[0021] Figure 9 This figure demonstrates the performance of the field-priority magnetic force control method when the magnetic field and magnetic force vectors are aligned. (a-1) to (c-1) show the expected and actual forces in the x, y, and z directions for a force amplitude of 12 mN. (a-2) to (c-2) show the force percentage error for four different expected force amplitudes: 6 mN, 12 mN, 18 mN, and 24 mN.

[0022] Figure 10 This figure demonstrates the performance of the field-priority magnetic force control method when the magnetic field and magnetic force vectors are misaligned. (a-1) to (c-1) show the expected and actual forces in the x, y, and z directions for a force amplitude of 12 mN. (a-2) to (c-2) show the force percentage error for four different expected force amplitudes: 6 mN, 12 mN, 18 mN, and 24 mN.

[0023] Figure 11 The experimental setup is shown. (a) Magnetic spiral swimmer. (b) Capsule model. (c) Human body phantom. (d) PVC tube. (e) Cube container.

[0024] Figure 12Experimental results of magnetically actuating a helical swimmer using the rotating magnetic field control method proposed in this invention are presented. (a)-(c) Case 1: A helical swimmer is driven in a curved pipe under different magnetic field amplitude and frequency combinations (see Video S3). (a-1)-(c-1) show the motion of the helical swimmer in a simulated pipe environment and the isotropic unit rotating magnetic field distribution within a cubic region under the optimized coil configuration. (a-2)-(c-2) show the actual views. (a-3)-(c-3) show the real-time position of the helical swimmer from a top-down camera view. (d)-(f) Case 2: A helical swimmer is driven in a straight pipe 80 mm from the phantom's top surface (see Video S4). (d-1)-(f-1) show the re-optimized coil configuration, which differs from the configuration in Case 1. (d-2)-(f-2) show the real-time position of the helical swimmer from a top-down camera view. (d-3)-(f-3) show the isotropic magnetic field per unit rotation under the re-optimized coil configuration. (d-4)-(f-4) show the real-time position of the spiral swimmer from the side-view camera.

[0025] Figure 13 Demonstrating magnetic propulsion of a capsule through a C-shaped tube using a magnetic field-priority magnetic force control method (see Video S5). Six different combinations of magnetic fields and magnetic forces are applied to propel the capsule through the C-shaped tube. The simulation environment and actual viewports show the capsule's real-time pose and corresponding coil configuration. The red arrow in the top view indicates the capsule's heading.

[0026] Figure 14 Experimental results show magnetic propulsion of a capsule model along an N-shaped path in three-dimensional space, guided by a magnetic field-priority magnetic control method (see Video S6). The overall process is divided into three motion phases: (a)-(b) show the first motion phase, in which the capsule's heading direction remains horizontal (along the y-axis) and a magnetic force is applied along the yz plane. (b)-(c) show the second motion phase, in which the angle between the capsule's heading direction and the applied magnetic force is 180°. (c)-(d) show the third motion phase, in which the capsule's heading direction remains vertical (along the z-axis) and a magnetic force is applied along the y-axis.

[0027] Figure 15 A schematic diagram showing the overall concept of the system of the present invention after integration with a C-arm fluoroscopy device. DETAILED DESCRIPTION

[0028] The present invention provides an electromagnetic drive system. In one embodiment, the electromagnetic drive system comprises: a) one or more magnetic robots positioned within a workspace; b) three independent robotic arms, each comprising a movable electromagnetic coil; and c) a positioning system for tracking the one or more magnetic robots. The three independent robotic arms are controlled through multi-objective optimization to ensure a coil configuration space capable of providing a magnetic field for manipulating the one or more magnetic robots within the workspace.

[0029] In one embodiment, one or more of the three independent robotic arms include a mobile base.

[0030] In one embodiment, the workspace is adjusted according to the reach and position of any one of the three independent robotic arms.

[0031] In one embodiment, the workspace is a human-scale workspace.

[0032] In one embodiment, the positioning system is selected from one or more of a video camera and a C-arm fluoroscopy system.

[0033] In one embodiment, the one or more magnetic robots include a screw-type robot or a capsule-type robot.

[0034] In one embodiment, the magnetic field is controlled by a performance-guided optimization method or a magnetic field-first magnetic force control method.

[0035] In one embodiment, the magnetic field is a rotating magnetic field with high isotropy, or a magnetic field capable of providing five-degree-of-freedom control for the one or more magnetic robots.

[0036] In one embodiment, the performance-guided optimization method determines the coil configuration space based on the following model: in f3=σ3,f4=1 / κ.

[0037] In one embodiment, the magnetic field priority magnetic force control method determines the coil configuration space based on the following model: in F d represents the expected force value, and Indicates that a particular coil configuration Actual force value under the action, operability

[0038] In one embodiment, the multi-objective optimization includes using a pseudo-weight vector method to determine a single solution, which is calculated as follows:

[0039] In one embodiment, the three independent robotic arms are controlled by software, which includes: a) a magnetic field control unit for receiving parameters for implementing the expected actions of the one or more magnetic robots; b) a perception unit for collecting and processing data from the positioning system; c) a robotic arm motion and planning unit for performing coordinated motion control of the three independent robotic arms; and d) an algorithm unit for generating control instructions based on feedback of the position and environmental information of the three independent robotic arms.

[0040] The present invention also provides a method for using the electromagnetic drive system of the present invention. In one embodiment, the method comprises the following steps: a) inserting the one or more magnetic robots into a cavity located within the workspace; and b) providing parameters to the system to obtain a coil configuration space for manipulating the one or more magnetic robots to perform a desired motion.

[0041] In one embodiment, the cavity is located within a human body.

[0042] In one embodiment, the one or more magnetic robots include a screw-type robot or a capsule-type robot.

[0043] The rapid development of remote-controlled magnetic robotics in the biomedical field has significantly accelerated the development of magnetic actuation systems, as they can meet the stringent requirements of magnetic field control. However, existing magnetic actuation systems still face numerous challenges in achieving magnetic robotic actuation in human-scale scenarios, such as insufficient magnetic field dexterity and a lack of a sufficient workspace suitable for clinical use. To address these key issues, this paper proposes a robot-assisted reconfigurable electromagnetic actuation system, named RARE, which can adjust the magnetic field intensity (up to 17 mT) and magnetic field gradient (up to 120 mT / m) within a human-scale workspace. Reconfigurable coil configuration is achieved by mounting three mobile electromagnetic coils on three independent robotic arms. Furthermore, a simulation environment based on the Robot Operating System (ROS) is constructed, with clear visualization, to improve development efficiency. To achieve dexterous magnetic field generation, a performance-guided optimization method is first proposed. By optimizing a set evaluation metric, a highly isotropic rotating magnetic field is generated within a target region. Subsequently, a field-prioritized magnetic force control method is constructed to achieve coordinated management of the magnetic field and magnetic force, thereby achieving five-degree-of-freedom (5-DOF) control. When the magnetic field and the magnetic force are aligned, the average force percentage error across all sampled direction vectors is 3.87%; when the magnetic field and the magnetic force are misaligned, the average force percentage error is 9.8%. Finally, extensive experiments validated the dexterity of the proposed system within a human-scale workspace, demonstrating its potential for clinical application.

[0044] This section describes the design process and key components of the RARE system, which are key design points that enable the system to achieve dexterous magnetic manipulation within a human-scale workspace. “Dexterous” here refers to having an expandable workspace, flexible magnetic field manipulation capabilities, and the ability to manipulate a variety of magnetic robots.

[0045] A. RARE System Overview

[0046] According to the expected goals and actual needs, the present invention develops a robot-assisted reconfigurable electromagnetic drive system (RARE), such as Figure 1As shown in (see Video S1), Table I summarizes the technical specifications of the RARE system. The RARE system includes three mobile electromagnetic coils, which are mounted on three independent six-degree-of-freedom robotic arms, respectively, to generate magnetic fields and magnetic field gradients in an expandable workspace. The size of the workspace mainly depends on the extension range of the robotic arms used and the relative installation positions of the three robotic arms. By controlling the three robotic arms, a reconfigurable three-coil posture, that is, coil configuration, can be achieved to adapt to different application scenarios. The robotic arm used (SJ-603-A, Anno Robot, Shenzhen, China) has a spherical workspace with a radius of 540 mm, can carry a load of up to 3 kg, and has a repeatability accuracy of 0.1 mm. The positioning system enables the RARE system to achieve autonomous tracking of the magnetic robot and reconstruct the phantom for path planning. Since each robotic arm is mounted on a mobile base (see Figure 1 (a)), so it can be moved to the operating room and integrated into other medical equipment, such as X-ray fluoroscopy.

[0047] Table I General technical specifications of the components in the system described in the present invention

[0048] B. Hardware Design

[0049] The following describes the general hardware design process of the RARE system used in the present invention.

[0050] 1) Coil Design: Unlike other systems that tend to use coils as large as possible to generate stronger magnetic fields, the maximum load of the robotic arm used in the RARE system is 3 kg, which limits the maximum weight of the coil. In addition, for larger coils, heat dissipation must also be considered, which is a major challenge for mobile EMA systems. Therefore, in the coil design process of this system, the main considerations include weight, diameter, and its ability to generate a magnetic field, and are guided by simulation results in finite element (FE) analysis software (COMSOL Multiphysics, COMSOL, Stockholm, Sweden). Ultimately, a coil with a ferromagnetic core that can enhance magnetic field strength was prepared, weighing 1.2 kg. The detailed specifications of the coil are shown in Table I.

[0051] 2) Positioning system: The RARE system currently uses two orthogonally arranged digital cameras (MER2-160-227U3C, DaHeng) to capture the top view and side view of the workspace respectively. The frame rate of the two cameras is 50 Hz, and the downsampled image resolution is 1280 × 1024 pixels. The system uses image processing methods to reconstruct the information of the magnetic robot's environment and its spatial position for experimental result measurement and closed-loop feedback control. Although visual tracking of the magnetic robot is not a clinically relevant imaging device such as ultrasound-guided navigation

[30] ,

[31] , it is sufficient for the proof-of-concept experiment of this invention.

[0052] 3) Low-level embedded controller:

[0053] To control the current input to the coil in real time, an STM32 control board is used as a low-level embedded controller to receive the current value commanded by the high-level host computer. Subsequently, the received digital current value is encoded into a pulse width modulated signal (PWMs) and sent to the servo driver (ESCON 70 / 10, Maxon), whose maximum output current is 10A. Figure 1 (b) The system is equipped with a 1000W power supply as the energy source. All control instructions are sent from the high-level host computer to the STM32 or robotic arm control box via Socket communication (TCP / IP protocol). The overall hardware process is as follows Figure 1 (c) shown.

[0054] C. Design of software architecture and simulation environment

[0055] With the development of robotics technology, some user-friendly interactive software frameworks and robot simulation systems have gradually emerged in recent years, such as Pybullet

[32] . However, there are still relatively few research attempts related to small-scale magnetic robots. Therefore, building a good software architecture and simulation environment is necessary and critical. On the one hand, a good software architecture can provide clearer visualization effects for medical tasks and provide surgeons with intuitive interaction methods. On the other hand, the simulation environment can serve as a test platform for researchers to easily debug and optimize algorithms. In addition, another benefit of the simulation environment is that it allows researchers to verify the feasibility of related algorithms in a simulation environment with parameters closest to real conditions before the algorithms are actually applied.

[0056] 1) Software Architecture: Figure 1(d) shows a detailed view of the overall software architecture, highlighting the interaction between the functional units and the communication between the low-level controller and the high-level main control computer. All software is configured in the ROS architecture using the Python language and runs on a Linux operating system installed on a desktop computer with system specifications of Ubuntu 18.04, Intel 8GHz CPU, 32GB memory and Nvidia RTX-3080GPU. The software of the RARE system described in the present invention is mainly composed of five control units, and each unit cooperates with each other through ROS communication. Specifically, the magnetic field / magnetic force control unit receives the commanded target magnetic field / magnetic force parameters to drive the magnetic robot to complete the expected action (see Figure 1 (e)). The perception unit collects and processes real-time video images from two cameras and tracks the movement of the magnetic robot (see Figure 1 (f)). The robot motion and planning unit is mainly responsible for realizing the coordinated motion control of three robot arms based on the ROS-MoveIt interface (see Figure 1 (g)). The algorithm unit generates control instructions based on feedback from the robot's state and environmental information. The software designed by this invention integrates user-friendly control methods and optimization libraries, making it suitable for algorithm development and compatible with other real-time physics engines, thereby supporting more realistic physical interaction processes.

[0057] 2) Visualization effect: This paper builds a customized ROS visualization interface (RViz) to provide surgeons with a clear overall view of the medical task execution process, such as Figure 2 As shown in the figure, the GUI displays three robotic arms, each equipped with a coil, a phantom used in different scenarios, a magnetic field display window, the magnetic robot, and its motion information. To achieve visualization, the magnetic robot model was built in SolidWorks and imported into RViz. In addition, a visualization window based on the Visualization Toolkit (VTK) was developed in Python, in which arrows are used to represent the combined three-dimensional magnetic field generated by the three coils. The visualization interface is synchronized with the real system, thus providing a solution for remote control of the system of the present invention.

[0058] Simulation environment: In order to verify the magnetic control effect and evaluate the proposed controller, the present invention designs a simulation environment based on the actual environment characteristics, such as Figure 1(a) As shown. The simulation environment consists of three main modules, namely the manipulator kinematics module, the magnetic field modeling module and the magnetic robot dynamics module. Specifically, the manipulator kinematics module is used to achieve real-time path planning and collision detection. In order to simulate the real scene, the simulated magnetic field used is generated by the model of the actual coil. In order to evaluate the motion state of the magnetic robot and accurately simulate the actual situation, the magnetic robot dynamics model is implemented in Python, and interacts with each module through the ROS application programming interface (API). The simulation system currently supports the control of spiral robots and capsule robots. More efforts will be made in the future to handle more realistic physical interaction processes.

[0059] D. Calibration

[0060] To precisely drive the magnetic robot, it is crucial to develop an accurate calibration method for the RARE system. The present invention requires performing base visual calibration to determine the relative position relationship between the robot arm and the camera. The easy_handeye package

[33] provides a convenient software tool for base visual calibration. The present invention particularly emphasizes the magnetic field calibration of a single coil.

[0061] References

[30] and

[34] introduce relevant research on single-coil magnetic field calibration. These studies use magnetic field data collected by magnetic sensors and finite element method (FEM) simulations performed by COMSOL, and usually rely on data obtained from finite element analysis (FEA). However, FEM modeling often cannot take into account imperfections in the manufacturing process, material inhomogeneity, or unmodeled interference factors. In addition, the measured magnetic field data is obtained by manually moving the commercial magnetic sensor in a workspace marked with a large number of grid points, which requires a lot of engineering manpower. Subsequently, the calibration magnetic field is calculated using the calibration equation in reference

[34] and saved in the form of a lookup table. In addition to single-coil magnetic field calibration, for fixed EMA systems composed of multiple coils, there are also studies that propose a method of placing a static three-dimensional magnetic sensor array in the center of the workspace to calibrate the entire system

[35] ,

[36] . The present invention focuses on single coils. To solve the above problems, taking advantage of the high positioning accuracy of the robotic arm described in the present invention, a set of automatic magnetic field data acquisition software is first developed to record the position of the coil and the magnetic field data. Then, based on the recorded discrete magnetic field data, linear interpolation is used instead of the lookup table to obtain a continuous representation of the magnetic field. Figure 3 (See Video S2.) To the best of our knowledge, this is the first study to provide open-source software for automated coil magnetic field calibration.

[0062] like Figure 3As shown in (a), the software designs a 33×50 grid area on the YZ plane with a spacing of 3mm in each direction. The coil needs to be moved across all grid cells by the robotic arm to sample the magnetic field. The size of the grid cells can be flexibly adjusted according to actual conditions. Figure 3 As shown in (b), a commercial 3D magnetic field sensor (TLE493D W2B6, Infineon Technologies) is fixed at a specific location to sample magnetic field data. Note that the raw measurement data from this magnetic sensor is subject to some noise. Therefore, an extended Kalman filter (EKF) is used to reduce the impact of measurement noise. Figure 3 (b) shows the comparison results before and after filtering. In addition, in order to avoid the nonlinear effect caused by saturation of ferromagnetic materials, it is necessary to model the linear relationship between the command current applied to the coil and the generated magnetic field. Figure 3 The results in (c) show an approximately linear relationship between the magnetic field and the current. Finally, based on the regular grid interpolator and the sampled magnetic field data, a three-dimensional vector magnetic field interpolation function is constructed to calculate the magnetic field value at any point in three-dimensional space.

[0063] In order to verify the performance of the proposed magnetic field calibration process, four points in the workspace were selected to measure the actual magnetic field values. At the same time, the corresponding calibration magnetic field values ​​were calculated based on the interpolation function. In addition, the calibration method in the literature

[34] was used for comparison. This method combines FEM with measured data to generate a lookup table. Table II lists the measured magnetic field B meas , the calibration magnetic field B in

[34] FEM , and the calibration magnetic field B derived by the method of the present invention ours It should be noted that the position of the sampling point is expressed with the center of the coil tip as the coordinate origin, and the applied command current is set to 1A. Subsequently, in order to further analyze the difference in calibration performance between the method of the present invention and the method of reference

[34] , B FEM With B meas The amplitude ratio, B ours With B meas The amplitude ratio of B FEM With B meas The angle between ours With B meas The results show that the magnetic field amplitude ratio obtained by the method of the present invention is closer to 1, indicating that the calibration result is more consistent with the actual measurement value. At the same time, the angular error of the method of the present invention is relatively smaller than that of the method in reference

[34] . In general, the manufacturing tolerance of the coil and the misalignment during the system assembly process are more likely to be the main sources of error.

[0064] Table II Comparison of magnetic field calibration results between the method of the present invention and the method in reference

[34]

[0065] III. Magnetic Drive Basics

[0066] A. Explanation of symbols

[0067] In this invention, scalars are represented by standard lowercase letters (e.g. ), column vectors are represented by bold lowercase letters (e.g. ). Matrices are represented by uppercase blackboard bold letters (e.g. ). The delimiter represents a unit vector or a normalized vector direction (e.g. ). The bi-norm of a vector is expressed as The transpose of a vector or matrix is ​​represented as

[0068] A left superscript indicates that a physical quantity is expressed relative to a specific coordinate system (e.g. Represents a vector expression in coordinate system i). Represents a homogeneous transformation, describing the vector Transform from coordinate system i to coordinate system j. The gradient operator represents the partial derivative in the three basis directions of a given coordinate system, and its expression is

[0069] B. Magnetic Actuation Modeling

[0070] This section shows a magnetic robot made of magnetic material, whose magnetic dipole moment is The position in the world coordinate system w When affected by The magnetic dipole moment is The magnetic field generated by the coil When the magnetic robot is in action, it will be subjected to force and torque This modeling requires two assumptions: 1) The combined magnetic field is assumed to be the sum of the fields generated by the individual coils. This assumption is reasonable because the ferromagnetic core material never reaches saturation in this invention. 2) In most cases, it can be assumed that the magnetic robot will always attempt to align with the direction of the applied magnetic field.

[0071] In the RARE system, all generated magnetic fields and forces are calculated with reference to the world coordinate system. Therefore, it is necessary to transform the magnetic field of each coil to the world coordinate system. Since the calibrated magnetic field data is expressed in the coil coordinate system c, the following formula is used to obtain the magnetic field of the kth coil (k = 1, 2, 3) in the world frame. in, and They represent the homogeneous transformation matrix from the manipulator base coordinate system to the world coordinate system, and the homogeneous transformation matrix from the coil coordinate system to the manipulator base coordinate system, respectively. represents the linear interpolation function of the magnetic field. It should be noted that Obtained through base visual calibration.

[0072] The magnetic field generated by the three coils can then be based on the current Perform a linear approximation. in, Indicates a point in the workspace The unit current driving matrix.

[0073] When the magnetic robot is subjected to an external magnetic field When the magnetic torque The mathematical formula is as follows

[37] ,

[38] : in, and

[0074] This torque is derived from the tendency of a magnetic robot's magnetic dipole moment to align with the direction of the applied magnetic field. For magnetic robots operating in low Reynolds number environments, this torque function can be used to control the robot's heading by adjusting the direction of the applied magnetic field, as long as the magnetic field is strong enough to withstand external interference. Therefore, compared to directly controlling the magnetic torque, it is more convenient to directly specify the desired magnetic field to linearize the system. The following section will illustrate the method of magnetic torque control using magnetic field control.

[0075] Similarly, the magnetic force can be expressed as follows, highlighting the controllable parameter of the magnetic field gradient

[39] :

[0076] From formulas (3) and (4), it can be seen that if the magnetic torque and magnetic force are completely independently controlled in the isotropic magnetic manipulation workspace by adjusting the current alone, at least eight fixed coils are required. However, the RARE system can directly realize magnetic field control based on the inverse mapping of formula (2). Its expected solution is It can be expressed as: in, Indicates the location The expected magnetic field at . The analysis helps to evaluate the system's ability to control the magnetic field in the entire working space. It is worth noting that since the magnetic field generated by each coil in the present invention is not always coplanar, the matrix It is always reversible. However, compared to the OctoMag system, achieving magnetic force control with three moving coils is not easy. This issue will be discussed further below.

[0077] IV. Performance-Guided Rotating Magnetic Field Control Method

[0078] The most common use of a rotating magnetic field is to induce a helical propulsion motion in a magnetic robot. This magnetic field generates a torque independent of external forces, which causes the magnetic dipole moment of the magnetic robot to tend to align with the direction of the magnetic field. For example, a helical swimmer can move under the conditions of extremely low Reynolds numbers under the drive of a rotating magnetic field

[40] . In the RARE system, different coil configurations can achieve a variety of local magnetic field distributions. However, how to determine the appropriate coil configuration for a specific task is crucial and challenging. This section will systematically analyze the key factors for generating a highly isotropic rotating magnetic field to ensure the dexterity of the control method.

[0079] A. Problem Description

[0080] One of the motivations for designing the RARE system is to provide the necessary controllable rotating magnetic field for magnetic manipulation. Indeed, systems like OctoMag are able to maintain a small central region with an isotropic magnetic field distribution in the center of the workspace. In addition, existing studies have proposed some evaluation metrics for measuring the effectiveness of different fixed systems, including maneuverability metrics, minimum singular values, etc.

[36] ,

[41] . There is currently no literature that systematically analyzes the magnetic field distribution of mobile EMA systems based on these evaluation metrics. A more worthy point to consider is how to use the required evaluation metrics to manage the coil configuration to generate a highly isotropic rotating magnetic field. The basic idea is to set a scoring standard for each evaluation metric as a performance target for evaluating the pros and cons of a specific coil configuration. Therefore, performance-guided optimization techniques can guide users to effectively explore the design space in the process of optimizing for a given performance target.

[0081] However, due to the nonlinear relationship between the performance target and the multiple degrees of freedom of the three coils (each coil has five degrees of freedom, ignoring the rotation of its main axis), it is difficult to directly determine the ideal three-coil configuration for generating a rotating magnetic field in the target area. Multi-objective optimization (MOO) technology

[42] can provide a solution to this type of multi-objective function problem. The following will explore how to apply this technology to solve related problems.

[0082] B. Definition of evaluation metrics

[0083] To measure the ability of any coil configuration to control a rotating magnetic field, various evaluation metrics can be defined. This paper considers the characteristics of an ideal rotating magnetic field within a target region from two perspectives: the isotropy of the magnetic field and the isotropy of the magnetic field drive matrix.

[0084] Isotropy of magnetic field: For applications that require a rotating magnetic field, a magnetic field with isotropic characteristics can achieve stable and reliable motion and avoid unexpected magnetic forces on gradient-sensitive magnetic robots. The ideal coil configuration should make the sampled magnetic field conform to the expected magnetic field distribution. To achieve this goal, the isotropy index of the magnetic field obtained by sampling from multiple directions in the target area can be calculated. To simplify the calculation and without loss of generality, the expected magnetic field is set to the unit magnetic field All generated magnetic fields in the target area must be consistent with the amplitude and direction of the expected unit magnetic field. Based on this criterion, the present invention designs the following two evaluation indicators for representing magnetic field isotropy: in, Measure the similarity of the magnitude of the actual magnetic field to the expected magnetic field. The minimum value of the directional similarity between the expected magnetic field and the sampled magnetic field is measured, and the sampled magnetic field should be selected as large as possible to maximize the similarity value in the worst case.

[0085] Drive Matrix Isotropy: The ability of a system to perform a specific motion at any position and orientation without singular configuration is crucial. For the RARE system, this can be achieved by examining the magnetic field drive matrix as described in Section III. The information provided by the matrix is ​​similar to the classic Jacobian matrix in robotics and can be used to represent the control isotropy of the system.

[0086] Drive Matrix The singular value σ i (i = 1, 2, 3) is an important indicator for evaluating the mapping efficiency between coil current and magnetic field through the classical maneuverability ellipsoid. On the one hand, the larger the minimum singular value σ3, the smaller the maximum coil current required in the worst-case control scenario close to the singular configuration. This helps reduce the temperature rise of the coil caused by large current. On the other hand, making the maneuverability ellipsoid closer to a sphere can improve the controllability of the system in all directions. This can be achieved by maximizing the inverse of the condition number κ to make it close to 1, which is defined as follows: 1 / κ = σ min / σ max = σ3 / σ1 ∈ [0,1] (8)

[0087] According to the above analysis, the magnetic field isotropy index and and the drive matrix The isotropic indexes σ3 and 1 / κ will be used as the objective function for optimizing the coil configuration. It should be noted that since this system has an adjustable working space, these evaluation indicators are only targeted at specific target areas.

[0088] C. Definition of coil configuration space

[0089] In order to implement the MOO method to generate the optimized coil configuration, it is necessary to first construct a search space that includes the design constraints on the coil posture. To this end, define the world coordinate system O w-xyz With coil coordinate system O c-xyz , to describe the coil configuration, such as Figure 4 As shown. First, each coil must be kept at a reasonable distance from the patient to avoid interference. In addition, the overall configuration of the three robotic arms must also reserve enough space for the operating table and the imaging system. Therefore, the configuration adopted by the present invention is: two robotic arms are installed on one side of the operating table, and the third robotic arm is installed on the other side of the operating table. The three coils are always located above the patient, which is reasonable in practical applications. Figure 4 As shown, the generalized coordinates of each coil coordinate system can be defined by five elements relative to the world coordinate system: Moreover, in order to obtain a collision-free coil posture, some constraints need to be carefully designed for the above variables. For example, according to the actual scenario, the parameter L i Limited by boundaries The present invention uses a collision avoidance algorithm to ensure that the coil is always safely close to the patient's target area, thereby improving the magnetic field-to-current ratio.

[0090] The set of attitude parameters of the three coils is considered as the coil configuration space (CCS). The CCS for the optimization problem is defined as follows:

[0091] Should be targeted at the collection Be careful to set constraints on each element in , as these constraints may reduce the feasible region in the objective space, resulting in a suboptimal solution.

[0092] Performance-guided optimization methods

[0093] During coil configuration optimization, placing all coils as close as possible to the target area can improve the ratio of magnetic field strength to current, thereby increasing torque generation. However, this can also lead to a decrease in magnetic field isotropy or collisions. This means optimizing one performance metric can compromise another. Therefore, a compromise between these performance metrics should be made based on the specific technical requirements of the application.

[0094] Although the weighted summation method is widely used due to its simplicity, the weight values ​​should be set with caution because the weights are not always linearly proportional to the objective function values, and different weight combinations may lead to the same optimal solution on the Pareto front. The multi-objective optimization (MOO) method aims to find the optimal solution to the optimization problem with multiple objective functions that often conflict with each other. Given a coil configuration space and a set of performance evaluation functions, the MOO method can extract the Pareto solution set, that is, the coil configuration with the best trade-off. As a widely praised algorithm in the field of MOO, the non-dominated sorting genetic algorithm (NSGA-II) is widely used due to its simple structure and strong global search capability

[43] , which can provide an effective solution for accelerating the solution of MOO problems.

[0095] Based on the above analysis, combined with the four evaluation indicators and the defined CCS, the process of finding the optimal coil configuration for magnetic field control in the present invention is formalized as a multi-objective optimization problem to maximize various performance indicators. in, f3=σ3,f4=1 / k.

[0096] The following describes how to perform the MOO problem.

[0097] A brief flow chart of the performance-guided optimization method for coil configuration design is as follows: Figure 5 During the optimization process, for a sample coil configuration in CCS X First, 27 unit magnetic fields in different directions are uniformly collected from the three-dimensional space. Then, each unit magnetic field is considered as an expected magnetic field, and the expected magnetic field at a given coil configuration is calculated according to formula (5). To collect the actual magnetic field data, a cubic workspace with a side length of 10 mm (considering the millimeter-scale magnetic robot in the present invention) is established. The workspace is based on the robot position In addition, the cube is divided into a total of N = 125 (3D grid: 5 × 5 × 5) grid points To calculate the actual magnetic field After obtaining 125 actual magnetic field data, calculate and record the magnetic field evaluation indicators The above process is repeated for each unit magnetic field until the evaluation of the unit magnetic field in all 27 directions is completed. The isotropy indices σ3 and 1 / κ depend only on the current coil configuration.

[0098] After obtaining the numerical values ​​of the four objective functions, the NSGA-II algorithm is used to calculate a set of non-dominated solutions, namely the Pareto front, in a multi-threaded manner. Compared with single-threaded programming, the execution efficiency of multi-threaded programming is improved by about 50%. However, the more difficult problem is how to determine a unique solution suitable for practical applications from the Pareto solution set, that is, how to select a suitable coil configuration from the multiple coil configuration schemes obtained for subsequent control. Although the weighted summation method is widely used due to its simplicity, the weight values ​​should be set with caution, because the weights are usually not always linearly proportional to the objective function values, so it is difficult to directly determine reasonable weights. In the present invention, the pseudo-weight vector method is used to select the final solution from the optimization solution set. The pseudo-weight w of the i-th objective function i It can be calculated as follows:

[0099] This formula is used to calculate each objective function Normalized distance relative to its worst solution. It should be clear that the four objective functions have different numerical scales. Therefore, normalization is crucial in MOO.

[0100] Because the actual Pareto front is unknown, it's impossible to determine whether the algorithm has converged to the true optimal solution. Hypervolume is a widely used performance metric in multi-objective optimization problems. It reflects the volume enclosed between a preset reference point and the current solution set. In the present invention, this performance metric is maximized. Figure 6 Demonstrating the convergence of the algorithm Most of the progress has been made during the optimization process, indicating that the optimization process has achieved convergence.

[0101] E. Analysis of Optimal Coil Configuration

[0102] In order to study and compare the magnetic field isotropy and magnetic field generation capabilities of the optimized coil configuration with other coil configurations, the present invention quantitatively analyzes the four evaluation indicators proposed and the current consumption under a specific rotating magnetic field. Figure 7 Detailed results are presented. In addition to the optimized coil configuration, two other coil configurations are sampled from CCS X for comparison. Without loss of generality, the parameter L in the three configurations remains the same. As expected, the evaluation index results show that the optimized coil configuration outperforms the other two configurations in performance. At the same time, by A unit rotating magnetic field was generated under a rotating magnetic field and the current consumption of each coil configuration was evaluated. For each orientation, current values ​​were collected for one loop at a 60° offset angle, and the cumulative current for each coil was calculated. The results show that in most cases, the optimized coil configuration has lower current consumption than the other two configurations. The optimized configuration's current consumption is 13% lower than that of Coil Configuration #1 and 29.6% lower than that of Coil Configuration #2. Furthermore, the maximum current consumption difference between any two coils in the three configurations is 6.6A (Coil Configuration #1), 6.1A (Coil Configuration #2), and 2.6A (Optimized Coil Configuration). The minimum singular value and the inverse of the condition number also reflect the system's ability to amplify its current under any magnetic field orientation. Unsurprisingly, the worst performing coil configuration in terms of current consumption is Coil Configuration #2, which has the smallest σ3 and 1 / κ values.

[0103] V. Magnetic field priority magnetic force control method

[0104] As described in Section III, magnetic field control in RARE systems can be effectively achieved using equations (2) and (3). The analysis of magnetic force control is more complex than that of magnetic field control because it relies on a five-element gradient vector to determine the final three-element magnetic force vector, as shown in equation (4). For fixed EMA systems such as OctoMag, the magnetic force is typically controlled solely by modulating the current. However, in RARE systems, managing the combined control of the magnetic field and force using coil configuration and current is crucial.

[0105] A. Problem Description

[0106] For an unconstrained capsule robot, it is reasonable to assume that the magnetic dipole moment of the capsule always tends to align with the external magnetic field (i.e., the heading control of the capsule), so that its motion can be controlled by the generated magnetic force. Since it is impossible to apply magnetic force to the robot in the absence of a magnetic field, magnetic field control must be considered if a magnetic field gradient needs to be applied to the robot. As described in

[29] , a static system requires eight independent magnetic field sources to control the magnetic field and magnetic force. The difficulty lies in how to use the three mobile coils in the RARE system to achieve coordinated control of the magnetic field and magnetic force. From the perspective of magnetic task space control, it is still difficult to express the relationship between the given magnetic field, magnetic force, and the posture and current of the three coils through an explicit expression. It should be noted that each coil has a total of 6 degrees of freedom for control: 5 posture degrees of freedom (excluding the rotation of the coil along its axis) and 1 current degree of freedom. The following will detail the solution to this problem based on the magnetic field priority magnetic force control method.

[0107] B. Methods

[0108] In order to realize magnetic field and magnetic force control through three movable coils, a magnetic field priority magnetic force control method is proposed in this paper. The method is based on the magnetic dipole moment of the magnetic robot. The assumption is that the magnetic field is always aligned with the direction of the external magnetic field. It should be noted that magnetic force control depends on the external magnetic field, and at least three coils are required to achieve three-dimensional magnetic field control. Therefore, under the given magnetic field and magnetic force, the magnetic field priority magnetic force control method can ensure stable magnetic field control and optimize the actual magnetic force to be close to the given magnetic force, so that the error is maintained within an acceptable range. Obviously, the magnetic dipole moment The value of affects only the magnitude of the generated magnetic force. Therefore, in the subsequent analysis, the unit magnetic dipole moment will be used. In addition to magnetic field and force control, the driving process of the magnetic robot must ensure that each robotic arm has good maneuverability to avoid singular configurations in certain combined configurations. Therefore, the present invention also pays special attention to the maneuverability measurement of the three robotic arms.

[0109] The present invention first designs a coil configuration space (CCS) identical to that defined in Section IV. The field-priority magnetic force control method not only optimizes the error between the expected and actual magnetic forces, but also takes into account the maneuverability of the manipulator. The latter is particularly important for motion within a large workspace. The maneuverability of the manipulator is an important performance indicator in the optimization function. Therefore, the MOO problem is formulated as follows: in, F d represents the expected magnetic force value, and Indicates that in a specific coil configuration The actual magnetic force generated by the Describes the ability of the robot to achieve arbitrary speed in Cartesian space. The MOO problem is solved by the NSGA-II algorithm, and the results are as follows Figure 8 shown.

[0110] C. Feasibility Assessment

[0111] As a difficult high-dimensional problem, it is still quite complicated to verify whether the method proposed in the present invention is feasible, that is, to evaluate whether the magnetic robot in an arbitrary posture can be controlled by the expected magnetic field and magnetic force during operation. In order to simplify the quantitative analysis of the method without losing generality, the present invention defines a direction vector space for a set of expected magnetic fields and magnetic forces, and conducts detailed analysis for the following two situations: 1) Definition of direction vector space: By defining the magnetic field direction vector space b, the RARE system can easily achieve arbitrary expected magnetic field control in the entire workspace, as described in Part III. Specifically, b contains 50 direction vectors sampled from three-dimensional space. For the magnetic force direction vector space f, since the three coils are usually located above the patient in actual scenarios, the coil configuration under this constraint condition will inevitably induce a magnetic attraction between the magnetic robot and the three-coil system. Therefore, it is not realistic for the RARE system to generate a component of magnetic force along the negative z direction. Based on the above analysis, the magnetic field direction vector space S b and the magnetic force direction vector space S f The intuitive definition of is as follows:

[0112] Based on the magnetic field and magnetic force direction vector space, the present invention further sets two different situations to verify and demonstrate the feasibility and effectiveness of the method.

[0113] 2) Case 1: The direction vectors of the magnetic field and the magnetic force are consistent: To evaluate the performance of the method, we first assume that the magnetic field and the magnetic force have the same direction vector. This case is applicable to many practical application scenarios, such as manipulating a capsule robot through a pipe-like environment, where it is necessary to ensure that the direction of the magnetic force and the heading of the capsule (magnetic field direction) are aligned with the center line of the pipe. To facilitate the understanding of the evaluation process, the magnetic dipole moment Assume that the unit magnetic moment Set the external magnetic field to unit magnetic field At this time, the external magnetic force can be calculated according to formula (4).

[0114] To ensure that the applied magnetic field and the magnetic force have the same direction vector during the evaluation, Sample all direction vectors in the , and make them as a common direction vector. Once a direction vector is sampled The coil configuration and current can be determined according to the method in Algorithm 2.

[0115] Since magnetic field control is prioritized during the optimization process, the present invention focuses more on the feasibility of magnetic force generation. The present invention defines an evaluation metric for describing the difference between the expected magnetic force value and the actual magnetic force value, namely the magnetic force percentage error ε, which is expressed as follows: Among them, F d and F a represent the expected magnetic component and the actual magnetic component respectively.

[0116] The overall results are as follows Figure 9 As shown. Based on the external unit magnetic dipole moment and unit magnetic field, four expected magnetic forces F with different amplitudes are set. d , to observe the actual magnetic force obtained under 50 sampling direction vectors. Figure 9 It can be seen from (a-1) to (c-1) that when the expected magnetic force amplitude F d When the magnetic force is 12mN, the actual magnetic force F is a The components have a good correspondence with the expected values. The average magnetic percentage error ε of all 50 direction vectors is 3.87%. In addition, the feasibility evaluation is also carried out for three other cases with different amplitudes, and the results are quantified by the magnetic percentage error, as shown in Figure 2. Figure 9 (a-2) to (c-2) are shown. It can be observed that when the expected magnetic force value is far away from F d =12mN, the magnetic force percentage error ε increases. This phenomenon can be explained by the fact that once the magnetic field amplitude and magnetic dipole moment are determined, the actual magnetic force for a specific coil configuration is limited to a reasonable range, which can be estimated in advance using a rough approximation. Another possible reason is that the maneuverability index of the manipulator was considered during the optimization process, and certain pathological configurations of the manipulator were excluded, which may have a negative impact on the magnetic force percentage error. This result shows that the improvement of maneuverability comes at the expense of magnetic force control accuracy.

[0117] Case 2: The direction vectors of the magnetic field and the magnetic force are inconsistent: In order to further verify the feasibility of the method of the present invention, the situation where the direction of the magnetic field is inconsistent with the direction of the magnetic force is considered.

[0118] First from Randomly sample a magnetic field direction vector from A magnetic force direction vector is randomly sampled from . The same method as described in Case 1 is used to obtain the actual magnetic force generated. The result is as follows Figure 10 It is worth noting that when the magnetic force F d When is 12 mN, the actual magnetic force generated can still correspond well to the expected value in most direction vectors, such as Figure 10 (a-1) to (c-1). The average magnetic force percentage error of all 50 direction vectors is 9.8%. Similarly, the actual magnetic force generated under the other three expected magnetic forces is also examined, and the distribution of the magnetic force percentage error is shown as follows Figure 10 (a-2) to (c-2) are shown. The results show that when the expected magnetic force amplitude is Fd = 12, the actual magnetic force will deviate from the expected value. In addition, the overall performance of this case is worse than that of Case 1. This shows that when the magnetic field direction vector is inconsistent with the magnetic force direction vector, the accessible magnetic space of the system of the present invention will be limited.

[0119] D.Analysis

[0120] Analysis of the magnetic field-first magnetic force control method reveals how the RARE system can achieve magnetic field and magnetic force control within the workspace for magnetic manipulation. The main limiting factors hindering further reduction of the magnetic field percentage error can be attributed to two aspects. First, there are constraints on the coil configuration space, which ensure collision-free operation between the three manipulators. Second, there are maneuverability indicators related to the manipulators. As one of the optimization objectives, maneuverability may weaken the accuracy of magnetic force control. Therefore, in order to address the negative impact of these two limiting factors during the magnetic force generation process, some solutions, although suboptimal in performance, are still within an acceptable range.

[0121] VI. Experiments and Results

[0122] To verify the performance of the RARE system using the magnetic manipulation method of the present invention, a series of experiments were conducted in various application scenarios, including manipulating a spiral swimmer within a human phantom and propelling a capsule model within the same phantom and a cubic container. Using a simulator, the control method and related parameters can be debugged and optimized by conducting experiments in a simulated environment. Subsequently, experiments can be conducted in a real environment based on the proposed control method and pre-optimized parameters. This means that optimized parameters can be found with only minor adjustments, rather than tedious re-calibration from scratch. Representative visualization examples from the following experiments can be seen in the accompanying video.

[0123] A. Experimental Setup

[0124] One of the typical application scenarios of the RARE system is to drive a spiral robot guided by a rotating magnetic field. First, a magnetic spiral swimmer is designed and prepared using standard manufacturing processes to meet the purpose and requirements of subsequent demonstrations. Figure 11 As shown in (a), the head of the spiral swimmer contains an N52 grade neodymium iron boron (NdFeB) permanent magnet (2mm in diameter and 2.5mm in height), whose magnetic dipole moment is perpendicular to the main axis of the swimmer. In addition, in order to verify the magnetic field priority magnetic control method, a capsule model was selected and designed, such as Figure 11As shown in (b), it is necessary to use magnetic torque to adjust its heading and realize the magnetic force of movement. The capsule model contains a permanent magnet (diameter 4mm, height 6mm), and the direction of its magnetic dipole moment is parallel to the main axis of the capsule model. It should be pointed out that many magnetic robots in medical applications must pass through tubular tissues or the human gastrointestinal system. The experimental phantom used in the present invention is intended to simulate actual application scenarios. Therefore, a real-size human body model phantom is selected to simulate the human-scale scenario, and its key dimensions are as follows: Figure 11 To better simulate the tubular tissue in the human body, a PVC tube with an inner diameter of 10 mm was installed in an inaccessible position in the human body model (see Figure 11 (d)). The PVC tube was filled with 80% glycerol to simulate a low Reynolds number environment. Figure 11 As shown in (e), an additional cubic container, also filled with 80% glycerol, was set up to manipulate the capsule model in three dimensions. Although the above phantom does not fully represent the real medical environment, it is sufficient for proof-of-concept experiments.

[0125] B. Evaluation of Rotating Field Control

[0126] It is easy to imagine the situation where a spiral swimmer is rotated by applying a rotating magnetic field, which helps to understand the coupling between the magnetic field source and the spiral swimmer. This experiment evaluates the performance of the rotating magnetic field in two different scenarios. In the first scenario, the spiral swimmer is immersed in viscous oil in a curved tube installed on the inner wall of a human body phantom for fixation and image acquisition. Figure 12 In the second scenario, a straight pipe is set inside the phantom, which is about 80 mm away from the upper surface of the phantom (see Figure 12 (d-4)) is used to verify whether the spiral swimmer can still maintain good swimming performance under the new coil configuration. Before the experiment begins, the center line of the pipe is reconstructed as the expected path of the spiral swimmer. Then, the position of the spiral swimmer is Calculations can be performed based on images from a bird's-eye view camera. For the two scenarios described above, the corresponding coil configurations are adjusted according to Algorithm 1. Based on the generated coil configurations, a rotating magnetic field with zero magnetic amplitude is applied in a plane perpendicular to the pipeline centerline, driving the helical swimmer to rotate and propel itself within the silicone oil. At each instant, the system calculates the positions of the three coils and their corresponding currents in real time to ensure that the rotating magnetic field is always aligned with the pipeline centerline.

[0127] Figure 12(a)-(c) show the experimental results of the first scenario (see Video S3), including the simulated environment image sequence, the real experimental scene, and the magnetic field visualization effect. At the initial stage of the experiment, the magnetic field amplitude was set to 4mT and the frequency was set to 2.5Hz. Figure 12 (a). Similarly, Figure 12 As shown in (b), the amplitude and frequency of the rotating magnetic field used to drive the spiral robot through the curved part of the pipeline are set to 3mT and 2Hz respectively. Finally, a larger magnetic field parameter is applied to push the spiral robot to the target position, as shown in Figure 12 At the same time, in order to demonstrate the isotropy of the rotating magnetic field under the optimized coil configuration, a cubic space (10×10×10mm 3 ), the spiral swimmer is placed at its center to show the real-time magnetic field. Note that the actual magnetic field has been normalized to the unit magnetic field and sampled at three different rotation angles, such as Figure 12 The experimental results show that the system can generate a highly isotropic rotating magnetic field under multiple combinations of magnetic field amplitude and frequency, thereby driving the motion of the helical swimmer in a curved pipe.

[0128] Furthermore, to verify the adaptability of the method in different scenarios, new coil configurations were obtained for the second scenario through optimization, e.g. Figure 12 (d-1) is shown. The experimental results are shown in Figure 12 (d)-(f) (see Video S4). Compared with the first scenario, the difference of the new coil configuration is that there is a significant distance deviation between the coil and the spiral swimmer. This is because the possible collision between the coil and the human model phantom is taken into account during the optimization process. Despite this, the experimental results show that the spiral swimmer still exhibits good swimming performance. In addition, at the three sampled rotation angles, the new coil configuration also shows good magnetic field isotropy, such as Figure 12 As shown in (d-3) to (f-3).

[0129] In summary, the experimental results under two different scenarios fully verify the effectiveness of the method in achieving highly isotropic rotating magnetic field control.

[0130] C. Evaluation of the Magnetic Field-Priority Magnetic Control Method in a Two-Dimensional Pipeline Environment

[0131] To evaluate the performance of the magnetic field-prioritized magnetic force control method, the present invention conducted an experiment propelling a capsule model through a C-shaped PVC pipe. Specifically, the pipe was fixed to the inner wall of a human body model phantom, and the relative position between the pipe and the phantom was manually determined. The magnetic field-prioritized magnetic force control method ensures that the capsule model's heading is always along the centerline of the pipe and applies a propulsion force to propel the capsule through the entire pipe. In fact, it is expected that the magnetic field and the magnetic force have the same direction vector and are aligned with the centerline of the pipe, as described in Case 1 in Section VC.

[0132] The experimental results are as follows Figure 13 (a)-(f) (see Video S5). During the driving process, according to the current position p r The expected heading of the capsule model is calculated to control the heading of the robot, that is, to align it with the center line of the pipe. Otherwise, the capsule model may get stuck due to the limitation of the inner wall of the pipe. In addition, in order to overcome the gravity of the capsule model itself, a constant vertical magnetic force is continuously applied in the experiment. It should be noted that switching from one coil configuration to another takes a certain amount of time (sometimes more than 2 seconds), which depends on the trajectory execution time of the robot arm. For example, from Figure 13 The coil configuration shown in (a) is switched to Figure 13 When the configuration shown in (b) (actual image) is used, the robotic arm needs a certain amount of time to adjust its posture. Although the movement speed of the robotic arm can be increased, this may cause vibration and higher safety risks. If the coil configuration is adjusted in real time based on the real-time position of the capsule, the smoothness of the control process may be affected. Therefore, the present invention adopts a compromise method, which is to divide the capsule's travel path into several areas according to the curvature of the path. In each area, the corresponding coil configuration is generated according to Algorithm 2 to generate the expected magnetic field and magnetic force. It should be noted that during the configuration switching process, the magnetic field will be temporarily turned off to avoid unexpected movement of the capsule.

[0133] D. Evaluation of the Magnetic Field Priority Magnetic Control Method in Three-Dimensional Space

[0134] To further verify the effectiveness of the magnetic field-priority magnetic control method, an experiment was conducted to enable the capsule to track a planned path in three-dimensional space at different heading directions, thereby achieving five-degree-of-freedom attitude control. This experiment assumed that the capsule was immersed and suspended in silicone oil, so its heading was aligned with the applied magnetic field. This experiment used both a top-view camera and a side-view camera to track the three-dimensional positioning of the capsule. It should be noted that the side walls of the human body model have a low transparency, which to some extent affects the side-view camera's ability to capture high-quality images. Therefore, to avoid image blur, a cubic container was used instead of the human body model as the phantom in this experiment.

[0135] Figure 14Experimental results of capsule navigation under five-degree-of-freedom attitude control are presented. The capsule moves along an N-shaped path in three-dimensional space, with its heading always pointing in a given direction (see Video S6). The entire experimental process is divided into three motion phases. Specifically, in each motion phase, the expected magnetic force direction vector is inconsistent with the magnetic field direction vector, that is, the heading direction is inconsistent with the motion direction, similar to Case 2 in Section VC. Due to the limited bandwidth of the robotic arm, it is difficult to generate new coil configurations in time to respond to rapid changes in the magnetic field and magnetic force direction during rapid manipulation. In addition, if the coil configuration needs to be adjusted in real time, oscillations may occur during the capsule's movement. These factors indicate that unnecessary coil configuration switching should be minimized in actual operation.

[0136] Based on the above analysis, the present invention developed a control strategy for propelling a capsule in three-dimensional space. During each motion phase, the corresponding coil configuration is optimized according to Algorithm 2 to generate the desired magnetic field and force. The capsule's velocity depends primarily on the magnitude of the applied magnetic force, which needs to be appropriately reduced before approaching the next motion phase. Specifically, when the magnetic field direction vector and the magnetic force direction vector remain constant, proportional changes in the magnetic field and force magnitudes only affect the current value and do not alter the coil configuration. Naturally, this experiment employs a proportional-integral-derivative (PID) control method to calculate this scaling factor, further enabling synchronous regulation of the magnetic field and force magnitudes. The magnetic field magnitude should remain sufficiently strong to ensure the capsule's heading is successfully aligned in the given direction. Furthermore, the current value changes rapidly, reaching up to 20 Hz. Experimental results demonstrate that, guided by the control strategy described in this invention, the capsule can move along the planned path in all three motion phases. Leveraging the RARE system and corresponding control method, this experiment demonstrates the effectiveness of capsule control in visual inspection applications.

[0137] VII. Discussion

[0138] The design of most current EMA systems is mainly limited by limited workspace, bulky or complex structural design, and low dexterity of magnetic field generation

[44] . The above limitations significantly restrict the actual application performance of existing EMA systems in operating rooms. Therefore, the present invention establishes a new EMA system and attempts to make it suitable for human-scale workspace, thereby improving the dexterity of magnetic manipulation and making it applicable to more patients. Through a number of simulations and experiments, the feasibility of the system in achieving autonomous magnetic manipulation was verified, including using a rotating magnetic field to drive a spiral swimmer to swim in a human model phantom, and using a magnetic field priority magnetic force control method to successfully propel a capsule model. The experimental results show that the system of the present invention exhibits high dexterity in magnetic manipulation and flexibility in different workspaces, and can thus adapt to complex environments with limited accessibility and visibility. It is worth mentioning that the simulation environment developed by the present invention plays a key role in algorithm development and algorithm robustness evaluation, effectively avoiding accidental movement. The simulation environment provides a good verification platform, especially emphasizing the safety of surgical personnel, patients and robotic systems. Although the present invention has achieved substantial results, the current methods and results still have some limitations and need to be further improved.

[0139] At present, the magnetic manipulation paradigm proposed in the present invention mainly relies on visual feedback and is not applicable to in vivo operations. If the successful transformation of in vivo operations is to be achieved, full consideration must be given to cooperating with medical imaging equipment to enhance its clinically relevant application potential. In such cases where the position information of the magnetic robot is required, X-ray fluoroscopy scanners can be considered as a viable alternative for real-time tracking. Therefore, the coil configuration and layout of the three robotic arms should provide open space so that the fluoroscopy scanner can actively monitor the status of the magnetic robot during the operation. One of the dexterity properties of the system of the present invention is reflected in its feasible workspace and the design of the movable wheeled robotic arms. This dexterity is achieved by the reconfigurable coil configuration of the three robotic arms and the movable wheeled base, such as Figure 15 As shown. Thanks to its movable base, the system can be pushed directly into the operating room without changing the operating room layout, and can be easily integrated with the C-arm and fluoroscopic scanner. This becomes an important advantage of the robotic system of the present invention in the clinical application of the operating room in conjunction with the fluoroscopy instrument. However, given that the system of the present invention may be pushed in or out of the operating room as needed, the main challenge it faces is how to ensure the calibration accuracy between the three robotic arms and the fluoroscopic scanner. One possible solution is to design a mechanical positioning mechanism to ensure that the three robotic arms can be accurately fixed to the expected position before each operation. In addition, it must be acknowledged that the automatic calibration process of the three robotic arms of the present invention in conjunction with the fluoroscopy instrument still needs further research.

[0140] Experimental results using a helical swimmer demonstrate that the performance-guided optimization framework successfully generates coil configurations with favorable evaluation metrics for a variety of tasks. An important aspect of the inventive concept, not fully explored in this study, is the issue of singular configurations during the manipulator's motion. It is important to note that rotating magnetic field control relies solely on current changes, as the manipulator can be moved close to the helical swimmer and accompanies it within the workspace while maintaining the same coil configuration. Specifically, incorporating maneuverability into the optimization objective helps avoid singular configurations at certain points in the workspace. However, this results in a decrease in the isotropy of the magnetic field generation. In fact, multiple simulations have shown that even without considering maneuverability, the optimization results are satisfactory and consistently cover the specified workspace. To reduce current consumption, the optimization results favor placing the coils as close as possible to the magnetic robot. Therefore, when certain tasks require a large distance between the coils and the magnetic robot to avoid obstacles in the workspace, a new coil configuration can be optimized by adjusting the parameter L in the coil configuration space. One advantage of using a simulation environment is that it allows you to execute and verify new coil configurations in simulation and assess whether the current configuration approaches the robotic arm's kinematic singularity. The value of parameter L depends primarily on the presence of obstacles between the coils and the magnetic robot. To accommodate complex environments, a conservative value for L can be used at the expense of slightly higher current consumption.

[0141] Regarding the singular configurations of manipulator motion during magnetic field-priority magnetic control, the maneuverability of each manipulator is considered as one of the objective functions in the optimization process, aiming to maintain a large maneuverable ellipsoid volume. Unlike rotating magnetic field control, magnetic field-priority magnetic control requires switching between different coil configurations to generate the desired magnetic field and force. Therefore, maneuverability is improved at the expense of magnetic control performance, ensuring that the manipulator avoids singular configurations as much as possible. The main limitation of this method lies in the switching phase between the two coil configurations. For example, during capsule actuation, the magnetic field is turned off to avoid abnormal capsule motion during the switching phase. Simultaneously coordinating the motion of the three manipulators and the magnetic field generation to maintain the capsule's position and heading is challenging. Furthermore, low-bandwidth manipulators require a certain amount of time to complete the switching phase, during which the capsule is affected by gravity. In experiments, a silicone oil environment provides some damping, enabling the capsule to maintain position stability for a short period of time. However, this behavior fails in high-flow environments, such as pulsatile blood flow, and a high-bandwidth manipulator may be required to improve response speed. Another issue worth discussing is the closed-loop tracking of an N-shaped path in three-dimensional space. The PID method is used to calculate the scaling of the magnetic field and magnetic force with a common scaling factor, which can be achieved simply by adjusting the current. This strategy aims to control the capsule speed while reducing unnecessary coil configuration switching. However, when the expected magnetic force amplitude approaches zero, the external magnetic field will also approach zero. If there is a large disturbance at this time, the capsule's magnetic dipole moment may deviate from the direction of the external magnetic field. Therefore, the assumption that the magnetic dipole moment of the magnetic robot is always aligned with the external magnetic field no longer holds. In summary, to improve the stability of magnetic manipulation, further research is needed on more robust control algorithms and advanced coil configuration switching strategies.

[0142] On the other hand, the static glycerol environment used in the experiment still has room for improvement. For practical application scenarios, a dynamic environment is necessary and should be further studied in the future

[45]

[46] . At the same time, environmental disturbances still pose a significant challenge to existing control methods. To overcome this limitation, a feasible solution is to introduce learning-based technology to achieve automation and high-precision motion control of this system

[47]

[48] . In existing research

[39] , a reinforcement learning method was described to enable the magnetic robot to cope with unknown flow rate changes. In addition, the simulation environment designed by this invention can provide a training environment for reinforcement learning to develop robust strategies.

[0143] In summary, although the current version of this system still has certain limitations, and it is still difficult to clearly determine whether the system can achieve safe magnetic manipulation in clinically relevant scenarios. But more importantly, the system designed by the present invention and the control framework proposed have provided a novel solution for achieving high-performance magnetic manipulation. The significant improvement in the dexterity of this system is also expected to bring potential benefits to other medical applications such as drug delivery. In addition to unconstrained magnetic robots, magnetic catheters can also be manipulated under the guidance of the system described in the present invention. Some areas for improvement can be adjusted accordingly in the next generation of systems. The contents of this specification are expected to provide empirical guidance for the integration of magnetically driven robots and robotic platforms, and benefit the future development of the medical and health field.

[0144] VIII. Conclusion

[0145] In summary, this paper develops a mobile EMA system that utilizes three robotic arms to achieve autonomous magnetic manipulation within a human-scale workspace. Specifically, three mobile coils are mounted on three independent robotic arms. This system not only provides flexible magnetic field generation capabilities but also leaves ample space for the integration of medical imaging equipment. Furthermore, the development of a simulation environment facilitates the development of control algorithms. This paper proposes an automatic magnetic field calibration and interpolation method to improve the accuracy of magnetic field modeling. To ensure the flexibility of magnetic field generation, the coil configuration is first adjusted using a performance-guided optimization method to generate a rotating magnetic field with high isotropy. The optimization results are validated through comparative studies. Next, this paper proposes and analyzes a magnetic field-prioritized magnetic force control method that can drive a five-degree-of-freedom capsule robot in three-dimensional space. These verification experiments fully demonstrate the dexterity and effectiveness of the system in magnetic manipulation. In summary, the results of this study reveal the potential application of the system in clinically relevant scenarios.

[0146] Future research will focus on improving the system design and verifying the clinical feasibility of the system. Furthermore, more advanced control strategies will be developed to enhance the efficiency and safety of the system's autonomous robotic navigation in dynamic environments.

[0147] References [1]PEDupont et al., "A decade retrospective of medical robotics research from 2010to2020," Sci.Robot., vol.6, no.60, Nov.2021, Art.no.eabi8017. [2]B.J.Nelson,I.K.Kaliakatsos,and J.J.Abbott,“Microrobots forminimally invasive medicine,”Annu.Rev.Biomed.Eng.,vol.12,pp.55–85,Aug.2010. [3]M.Sitti et al.,“Biomedical applications of untethered mobilemilli / microrobots,”Proc.IEEE,vol.103,no.2,pp.205-224,Feb.2015. [4]James W.Martin et al.,“Enabling the future of colonoscopy withintelligent and autonomous magnetic manipulation,”Nature Mach.Intell.,vol.2,no.10,pp.595-606,Oct.2020. [5]J.J.Abbott,E.Diller,and A.J.Petruska,“Magnetic methods inrobotics,”Annu.Rev.Control Robot.Auton.Syst.,vol.3,no.1,pp.57–90,May 2020. [6]W.Hu,G.Z.Lum,M.Mastrangeli,and M.Sitti,“Small-scale soft-bodiedrobot with multimodal locomotion,”Nature,vol.554,no.7690,pp.81–85,Feb.2018. [7]P.Valdastri,R.J.Webster,C.Quaglia,M.Quirini,A.Menciassi,andP.Dario,“A new mechanism for mesoscale legged locomotion in compliant tubularenvironments,”IEEE Trans.Robot.,vol.25,no.5,pp.1047-1057,Oct.2009. [8]C.Bergeles,B.E.Kratochvil,and B.J.Nelson,“Visually servoingmagnetic intraocular microdevices,”IEEE Trans.Robot.,vol.28,no.4,pp.798–809,Aug.2012. [9]G.Ciuti,P.Valdastri,A.Menciassi,and P.Dario,“Robotic magneticsteering and locomotion of capsule endoscope for diagnostic and surgicalendoluminal procedures,”Robotica,vol.28,no.2,pp.199-207,2010.

[10] S.Salmanipour,O.Youssefi,and E.Diller,“Design of multi-degrees-of-freedom microrobots driven by homogeneous quasi-static magnetic fields,”IEEE Trans.Robot.,vol.37,no.1,pp.246-256,Feb.2021.

[11] Z.Yang and L.Zhang,“Magnetic actuation systems for miniaturerobots:A review,”Adv.Intell.Syst.,vol.2,no.9,2020,Art.no.2000082.

[12] J.Sikorski,C.M.Heunis,F.Franco,and S.Misra,“The ARMM system:Anoptimized mobile electromagnetic coil for non-linear actuation of flexiblesurgical instruments,”IEEE Trans.Magn.,vol.55,no.9,pp.1–9,Sept.2019.

[13] F.Niu,J.Li,W.Ma,J.Yang,and D.Sun,“Development of an enhancedelectromagnetic actuation system with enlarged workspace,”IEEE / ASME Trans. Mechatronics,vol.22,no.5,pp.2265-2276,Oct.2017.

[14] S.Zhang et al.,”Design and characteristics of 3D magneticallysteerable guidewire system for minimally invasive surgery,”IEEERobot.Automat.Lett.,vol.7,no.2,pp.4040-4046,Apr.2022.

[15] R.Chen,D.Folio,and A.Ferreira,“Mathematical approach for thedesign configuration of magnetic system with multiple electromagnets,”Robot.Auton.Syst.,vol.135,Jan.2021.

[16] P.Ryan and E.Diller,“Magnetic actuation for full dexteritymicrorobotic control using rotating permanent magnets,”IEEE Trans.Robot.,vol.33,no.6,pp.1398-1409,Dec.2017.

[17] A.Z.Taddese,P.R.Slawinski,M.Pirotta,E.De Momi,K.L.Obstein,andP.Valdastri,“Enhanced real-time pose estimation for closed-loop roboticmanipulation of magnetically actuated capsule endoscopes,”Int.J.Robot.Res.,vol.37,no.8,pp.890-911,Jun.2018.

[18] F.Ongaro,S.Pane,S.Scheggi,and S.Misra,“Design of an electro-magnetic setup for independent three-dimensional control of pairs ofidentical and nonidentical microrobots,”IEEE Trans.Robot.,vol.35,no.1,pp.174-183,Feb.2019.

[19] T.Xu,J.Yu,X.Yan,H.Choi,and L.Zhang,“Magnetic actuation basedmotion control for microrobots:An overview,”Micromachines,vol.6,no.9,pp.1346-1364,Jul.2015.

[20] M.P.Kummer et al.,“OctoMag:An electromagnetic system for 5-DOFwireless micromanipulation,”IEEE Trans.Robot.,vol.26,no.6,pp.1006–1017,Dec.2010.

[21] S.Salmanipour and E.Diller,“Eight-degrees-of-freedom remote actu-ation of small magnetic mechanisms,”in Proc.IEEE Int.Conf.Robot.Automat.,2018,pp.3608–3613.

[22] B.E.Kratochvil et al.,“MiniMag:A hemispherical electromagneticsys-tem for 5-DOF wireless micromanipulation,”in Experimental Robotics,(ser.Springer Tracts in Advanced Robotics).Berlin,Germany:Springer,2014,pp.317–329.

[23] C.Chautems and B.J.Nelson,“The tethered magnet:Force and 5-dofpose control for cardiac ablation,”in Proc.IEEE Int.Conf.Robot.Automat.,2017,pp.4837–4842.

[24] A.W.Mahoney and J.J.Abbott,“Five-degree-of-freedom manipulationof an untethered magnetic device in fluid using a single permanent magnetwith application in stomach capsule endoscopy,”Int.J.Robot.Res.,vol.35,pp.129–147,2016.

[25] G.Pittiglio,et al.,“Collaborative magnetic manipulation via tworobotically actuated permanent magnets,”IEEE Trans.Robot.,2022,doi:10.1109 / TRO.2022.3209038.

[26] J.Sikorski,I.Dawson,A.Denasi,E.E.Hekman,and S.Misra,“IntroducingBigMag-a novel system for 3D magnetic actuation of flexible surgicalmanipulators,”in Proc.IEEE Int.Conf.Robot.Automat.,2017,pp.3594–3599.

[27] L.Yang,X.Du,E.Yu,D.Jin,and L.Zhang,“Deltamag:an electromagneticmanipulation system with parallel mobile coils,”in Proc.IEEEInt.Conf.Robot.Automat.,2019,pp.9814–9820.

[28] X.Du,M.Zhang,J.Yu,L.Yang,P.W.Y.Chiu,and L.Zhang,“Design and real-time optimization for a magnetic actuation system with enhanced flexibility,”IEEE / ASME Trans.Mechatron.,vol.26,no.3,pp.1524-1535,Jun.2021.

[29] A.J.Petruska and B.J.Nelson,“Minimum bounds on the number ofelectromagnets required for remote magnetic manipulation,”IEEE Trans.Robot.,vol.31,no.3,pp.714–722,Jun.2015.

[30] J.Sikorski,C.M.Heunis,R.Obeid,V.K.Venkiteswaran,and S.Misra,“Aflexible catheter system for ultrasound-guided magnetic projectile delivery,”IEEE Trans.Robot.,vol.38,no.3,pp.1959-1972,Jun.2022.

[31] S.Pane,G.Faoro,E.Sinibaldi,V.Iacovacci,and A.Menciassi,“Ultrasound acoustic phase analysis enables robotic visual-servoing ofmagnetic microrobots,”IEEE Trans.Robot.,vol.38,no.3,pp.1571-1582,Jun.2022.

[32] E.Coumans and Y.Bai.(2017).Pybullet,a Python module for physicssimulation in robotics,games and machine learning.[Online].Available:https: / / pybullet.org.

[33] S.Sarabandi,J.M.Porta,and F.Thomas,“Hand-eye calibration madeeasy through a closed-form two-stage method,”IEEE Robot.Automat.Lett.,vol.7,no.2,pp.3679-3686,Apr.2022.

[34] X.Du,L.Yang,J.Yu,K.F.Chan,P.W.Y.Chiu,and L.Zhang,“Robo-mag:Amagnetic actuation system based on mobile electromagnetic coils with tunableworking space,”in Proc.5th IEEE Int.Conf.Adv.Robot.Mechatronics,2020,pp.125-131.

[35] R.Yu et al.,“Modeling electromagnetic navigation systems formedical applications using random forests and artificial neural networks,”inProc.IEEE Int.Conf.Robot.Automat.,2020,pp.9251-9256.

[36] S.L.Charreyron,Q.Boehler,B.Kim,C.Weibel,C.Chautems,and B. J.Nelson,“Modeling electromagnetic navigation systems,”IEEETrans.Robot.,vol.37,no.4,pp.1009-1021,Aug.2021.

[37] Q.Wang,X.Du,D.Jin,and L.Zhang,“Real-time ultrasound dopplertracking and autonomous navigation of a miniature helical robot foraccelerating thrombolysis in dynamic blood flow,”ACS Nano,p.ac-snano.1c07830,Jan.2022,doi:10.1021 / acsnano.1c07830.

[38] T.Xu,C.Huang,Z.Lai,and X.Wu,“Independent control strategy ofmultiple magnetic flexible millirobots for position control and pathfollowing,”IEEE Trans.Robot.,vol.38,no.5,pp.2875-2887,Oct.2022.

[39] M.Cai et al.,“Deep reinforcement learning framework-based flowrate rejection control of soft magnetic miniature Robots,”IEEE Trans.Cybern.,2022,doi:10.1109 / TCYB.2022.3199213.

[40] T.Xu,G.Hwang,N.Andreff,and S.Re′gnier,“Planar path following of3-D steering scaled-up helical microswimmers,”IEEE Trans.Robot.,vol.31,no.1,pp.117-127,Feb.2015

[41] A.Pourkand and J.J.Abbott,“A critical analysis of eight-electromagnet manipulation systems:The role of electromagnet configuration onstrength,isotropy,and access,”IEEE Robot.Automat.Lett.,vol.3,no.4,pp.2957–2962,Oct.2018.

[42] N.Gunantara,“A review of multi-objective optimization:methods andits applications,”Cogent Eng.,vol.5,no.1,Jul.2018.

[43] K.Deb,A.Pratap,S.Agarwal,and T.Meyarivan,“A fast and elitist mul-tiobjective genetic algorithm:NSGA-II,”IEEE Trans.Evol.Comput.,vol.6,no.2,pp.182-197,Apr.2002.

[44] B.J.Nelson et al.,“Magnetically actuated medical robots:an invivo perspective,”Proc.IEEE,vol.110,no.7,pp.1028-1037,Jul.2022.

[45] C.Pawashe,S.Floyd,E.Diller,and M.Sitti,“Two-dimensional au-tonomous microparticle manipulation strategies for magnetic micro-robots influidic environments,”IEEE Trans.Robot.,vol.28,no.2,pp.467-477,Apr.2012.

[46] I.S.Khalil et al.,“The control of self-propelled microjets insidea microchannel with time-varying flow rates,”IEEE Trans.Robot.,vol.30,no.1,pp.49-58,Feb.2014.

[47] S.Xu,J.Liu,C.Yang,X.Wu,and T.Xu,“A learning-based stable servocontrol strategy using broad learning system applied for microroboticcontrol,”IEEE Trans.Cybern.,Nov.2021,doi:10.1109 / TCYB.2021.3121080.b.

[48] M.E.Tiryaki,S.O.Demir,and M.Sitti,“Deep learning-based 3Dmagnetic microrobot tracking using 2D MR images,”IEEE Robot.Automat.Lett.,vol.7,no.3,pp.6982-6989,Jul.2022.

Claims

1. An electromagnetic drive system comprising: a. One or more magnetic robots located within the workspace; b. three independent robotic arms, each of which comprises a movable electromagnetic coil; c. a positioning system for tracking said one or more magnetic robots; Its characteristics are: The three independent robotic arms are controlled through multi-objective optimization to ensure a coil configuration space that can provide a magnetic field for manipulating the one or more magnetic robots within the workspace.

2. The electromagnetic drive system according to claim 1, characterized in that: One or more of the three independent robotic arms include a mobile base.

3. The electromagnetic drive system according to claim 1, characterized in that: The workspace is adjusted according to the reach and position of any one of the three independent robotic arms.

4. The electromagnetic drive system according to claim 1, characterized in that: The workspace is a human-scale workspace.

5. The electromagnetic drive system according to claim 1, characterized in that: The positioning system is selected from one or more of a video camera and a C-arm fluoroscopy device.

6. The electromagnetic drive system according to claim 1, characterized in that: The one or more magnetic robots include a spiral-type robot or a capsule-type robot.

7. The electromagnetic drive system according to claim 1, characterized in that: The magnetic field is controlled by a performance-guided optimization method or a magnetic field-priority magnetic force control method.

8. The electromagnetic drive system according to claim 7, characterized in that: The magnetic field is a rotating magnetic field with high isotropy, or a magnetic field capable of providing five-degree-of-freedom control for the one or more magnetic robots.

9. The electromagnetic drive system according to claim 7, characterized in that: The performance-guided optimization method determines the coil configuration space based on the following model: where f1 = ‖b‖ iso ,f2=(b cosine ) min ,f3=σ3,f4=1 / k.

10. The electromagnetic drive system according to claim 7, characterized in that: The magnetic field priority magnetic force control method determines the coil configuration space based on the following model: Among them E F =-‖F d -F(x)‖,F d represents the expected force value, and F(x) represents the actual force value under a specific coil configuration x, operability 11. The electromagnetic drive system according to claim 1, characterized in that: The multi-objective optimization involves using a pseudo-weight vector method to determine a single solution, which is calculated as:

12. The electromagnetic drive system according to claim 1, characterized in that: The three independent robotic arms are controlled by a software comprising: a. a magnetic field control unit for receiving parameters for implementing the desired action of one or more magnetic robots; b. a sensing unit for collecting and processing data from the positioning system; c. a robotic arm motion and planning unit for performing coordinated motion control of the three independent robotic arms; and d. An algorithm unit for generating control instructions based on feedback of the positions and environmental information of the three independent robotic arms.

13. A method of using the electromagnetic drive system according to claim 1, characterized in that: The method comprises the following steps: a. inserting the one or more magnetic robots into a cavity located within the workspace; and b. providing parameters to the system to obtain a coil configuration space for manipulating the one or more magnetic robots to perform desired actions.

14. The method according to claim 13, wherein: The cavity is located in the human body.

15. The method according to claim 13, wherein: The one or more magnetic robots include a spiral-type robot or a capsule-type robot.

Images