Micro-nano robot cluster multimodal behavior control method and system based on magnetic field

By adopting a multimodal behavior regulation method based on magnetic field in the field of micro-nano robots, combining magnetic field and fluid resistance field, a multi-dimensional magnetic induction expression is established, which solves the shortcomings of multi-magnetic field dimensions and type design in the existing technology, and achieves high precision micro-nanobot cluster motion and task execution.

CN115502973BActive Publication Date: 2024-11-26TONGJI UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202211164823.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-23
Publication Date
2024-11-26
Estimated Expiration
2042-09-23

AI Technical Summary

Technical Problem

The prior art is difficult to implement multi-magnetic field dimensions and types of method design in the field of micro-nano robots, and cannot meet the needs of high-precision regulation, especially in complex tasks and small-size application scenarios.

Method used

Using a multimodal behavior regulation method of micro-nano robot clusters based on magnetic fields, multi-dimensional magnetic induction intensity expression is established by determining the task type and action sequence, combining external drive magnetic field, fluid resistance field and boundary conditions to realize the multi-dimensional magnetic induction intensity expression of micro-nano robot clusters. Modal behavior regulation and introduce graph attention mechanism for real-time identification and tracking.

Benefits of technology

It realizes the diverse movement of micro-nano robot clusters, adapts to complex environments, improves the robustness and accuracy of task execution, is suitable for multiple application scenarios, and provides high-precision regulation capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115502973B_ABST
    Figure CN115502973B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and system for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field, wherein the method comprises the following steps: S1, judging whether the micro-nano robot cluster is performing a single-step simple task or a multi-step complex task, if it is a single-step simple task, entering S3, if it is a multi-step complex task, entering S2; S2, determining a timing combination scheme for control according to the sequence of actions and modal behaviors performed by the micro-nano robot cluster; S3, determining the magnetic field dimension and type required for realizing the action and modality of each step of the cluster; S4, establishing a multi-dimensional magnetic induction intensity expression to realize multimodal behavior control of the micro-nano robot cluster; S5, obtaining the position and state of the micro-nano robot cluster in real time and realizing effective tracking and information feedback of the cluster. Compared with the prior art, the method of the present invention has the advantages of generalization, strong robustness, adaptability to multi-type scenarios and high-precision control requirements.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of micro-nano robots, and in particular to a method and system for regulating multimodal behavior of a micro-nano robot cluster based on a magnetic field. Background Art

[0002] Micro-nano robots are miniaturized robots with a scale of micro-nano (several nanometers to hundreds of micrometers), integrating advanced micro-nano manufacturing technology, new materials and intelligent robot control strategies. Its miniaturization is a double-edged sword. On the one hand, it shows great potential for the execution of complex tasks, which can trigger great changes in the fields of medicine, environment and chemistry. On the other hand, it is different from the mechanical equilibrium conditions and extremely low imaging resolution of macro robots, which brings challenges to the effective movement and perception of micro-nano robots. Inspired by the collective intelligence of bird flocks and fish schools in nature, scientists have invented and constructed various micro-nano robot clusters. With the increase in the number of individuals in the cluster and the increase in the complexity of the tasks, micro-nano robot clusters will face multi-dimensional and multi-factor limitations such as space and time, such as how to adapt to complex and tortuous three-dimensional environments and how to quickly and effectively avoid dynamic obstacles. Therefore, giving micro-nano robot clusters the potential for multi-modal robust motion and designing programmable trigger strategies for multi-modal behavior regulation of micro-nano robot clusters are key issues in promoting the application of micro-nano robots.

[0003] Since the micro-nano robot cluster is in a low Reynolds number environment during operation, that is, the viscosity force is dominant and the inertia force is negligible, the reciprocating symmetrical motion cannot achieve effective displacement. Therefore, if you want to achieve long-range motion and the execution of complex tasks, you need to provide it with power continuously. Consider that the micro-nano robot cluster can be used as a transducer, that is, it can obtain other forms of external energy and convert it into kinetic energy to achieve effective motion. Common external driving energy sources include magnetic field, ultrasonic field, light field and electric field. Among them, the magnetic field has the advantages of convenient programming compared with other methods due to its own controllable direction, intensity and frequency, and has become an important external driving energy in the field of micro-nano robots. The Helmholtz coil is a common magnetic field generating device in the field of micro-nano robot driving. It is composed of a pair of parallel and coaxial circular coils. The distance between the two coils is the same as the radius of the circular coil, and the current of the same direction and magnitude is passed through the two coils. A pair of Helmholtz coils can provide a uniform magnetic field in the region of interest (ROI), and multiple pairs of Helmholtz coils can achieve various types of magnetic fields through orthogonal combination. The prior art with the document publication number CN111508681A discloses a three-dimensional equal-diameter magnetic field generating device and provides a structural design of a three-dimensional coil. The prior art with the document authorization announcement number CN102820118B provides a design of a magnetic field generating device using three groups of mutually orthogonal Helmholtz coils and a method for realizing a spatial rotating magnetic field. The above existing technologies focus on the hardware design and implementation of the magnetic field generating device, and lack generalized technology and method design for multiple magnetic field dimensions and types. The prior art with the document authorization announcement number CN105662318B discloses a magnetic field system that integrates visual perception, feedback interaction and a magnetic field generating device. By regulating the axis of the universal uniform rotating magnetic field in space, the posture control and friendly human-computer interaction of the controlled object, i.e., the capsule robot, are realized, thereby enhancing the applicability of the magnetic field system. However, this technology lacks the expansion of the functions of the magnetic field system, and is difficult to be applied to the field of micro-nano robots with higher regulation requirements and smaller sizes.

[0004] Therefore, how to achieve the design of methods with multiple magnetic field dimensions and types to meet the various application scenarios and high-precision control requirements in the field of micro-nano robots has become a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a magnetic field strategy and system for multimodal behavior regulation of micro-nano robot clusters.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field, the method comprising the following steps:

[0008] S1. According to the pre-set deterministic task, determine whether the micro-nano robot cluster is performing a single-step simple task or a multi-step complex task. If it is a single-step simple task, proceed to step S3; if it is a multi-step complex task, proceed to step S2;

[0009] S2, determining a timing combination scheme for regulation according to the sequence of actions and modal behaviors of the micro-nano robot cluster required for the deterministic task, and executing step S3;

[0010] S3. According to the dynamic model revealed by the interaction between the micro-nano robot cluster and the external driving magnetic field, fluid resistance field, boundary conditions and monomers, determine the magnetic field dimension and type required for the cluster's action and mode realization at each step;

[0011] S4. Establish a multi-dimensional magnetic induction intensity expression according to the required magnetic field dimension and type to achieve multi-modal behavior control of micro-nano robot clusters;

[0012] S5. According to the cluster multimodal recognition and tracking method, the position and state of the micro-nano robot cluster are obtained in real time, and effective tracking and information feedback of the cluster are achieved.

[0013] Furthermore, the magnetic field is a multi-dimensional magnetic field, including a one-dimensional magnetic field, a two-dimensional magnetic field and a three-dimensional magnetic field; the one-dimensional magnetic field includes a static magnetic field and a pulsed alternating magnetic field; the two-dimensional magnetic field includes a polarization rotation field and an oscillating field; the three-dimensional magnetic field includes a conical magnetic field; the magnetic induction intensity expression contains the amplitude, frequency and angular direction information of the multi-dimensional magnetic field.

[0014] Furthermore, the establishment of the one-dimensional magnetic field magnetic induction intensity expression comprises the following steps:

[0015] S4011: Determine the type of one-dimensional magnetic field. If it is a static magnetic field, the expression of magnetic induction intensity is:

[0016] B s =B s e c

[0017] Among them, B s is the magnetic induction intensity amplitude, e c is the direction of the static magnetic field in three-dimensional space; if it is a pulse alternating magnetic field, go to S4012;

[0018] S4012: If it is a pulse alternating magnetic field, the expression of magnetic induction intensity is:

[0019] B p =B p u(t)e c

[0020] Among them, Bp is the magnetic induction intensity amplitude, u(t) is the pulse magnetic field function, e c is the direction of the pulse alternating magnetic field in three-dimensional space; u(t) is expressed as:

[0021]

[0022] Wherein, k∈{0, 1, 2...}, DC=1%~99%, f is the alternating frequency of the magnetic field;

[0023] The establishment of the expression of the two-dimensional magnetic field magnetic induction intensity comprises the following steps:

[0024] S4021: Determine the type of the two-dimensional magnetic field. If it is a polarization rotating field, go to S402; if it is an oscillating field, go to S4023;

[0025] S4022: Establishment of the expression of the magnetic induction intensity of the two-dimensional polarized rotating field. First, the spatial coordinates of the rotation axis vector of the polarized rotating field need to be established, and the rotation plane needs to be defined. Then, based on the relationship between the orthogonal vectors, a two-dimensional vector expression rotating around the rotation axis is established. The amplitude and frequency of the magnetic induction intensity are further set. Finally, the magnetic induction intensity vector is transformed into a rectangular coordinate system and decomposed into a three-dimensional space coordinate system.

[0026] S4023: Establishment of the expression of the magnetic induction intensity of the two-dimensional oscillating field. First, the spatial coordinates of the normal line of the oscillation plane need to be established, and the baseline vector and the two-dimensional swing angle need to be defined to determine the oscillation boundary. Then, the baseline is used as the vertical axis, and the vector perpendicular to the baseline is used as the horizontal axis. The relationship between the vectors in the oscillation plane is analyzed, and a two-dimensional oscillation vector expression with the baseline as the central axis is established. The magnetic induction intensity amplitude and the oscillation frequency are further set. Finally, the magnetic induction intensity vector is transformed into a rectangular coordinate system and decomposed into a three-dimensional space coordinate system.

[0027] The establishment of the three-dimensional magnetic field magnetic induction intensity expression first analyzes the direction of the magnetic field. The direction of the conical magnetic field rotating around an arbitrary axis with the coordinate origin as the vertex in the three-dimensional space can be determined by the normal of the cone base, that is, the rotation axis vector and the angle between the cone surface vector and the rotation axis. The rotation axis vector space coordinates are established, the cone surface vector and the angle between the cone surface vector and the rotation axis are defined, and then the cone surface vector is decomposed into two component vectors along the rotation axis and perpendicular to the rotation axis. The component vector parallel to the rotation axis does not rotate, and the perpendicular component vector rotates around the rotation axis vector on the cone base. According to the relationship between the space vectors, the expression of the three-dimensional conical vector rotating around the rotation axis is established, and the magnetic induction intensity amplitude and the rotation frequency are further set. Finally, the magnetic induction intensity vector is transformed into rectangular coordinates and decomposed into a three-dimensional space coordinate system.

[0028] Furthermore, the cluster multimodal recognition and tracking method specifically includes the following steps:

[0029] S501: Based on the single modality of the micro-nano robot cluster and the image sequence of potential application scenarios, manually add annotations to extract the basic features of the cluster modality;

[0030] S502: Expand data based on the acquired basic features to construct a feature data set;

[0031] S503: Based on the target-aware twin graph attention network, real-time dynamic tracking of the micro-nano robot cluster during task execution is realized, and the position and status information of the micro-nano robot cluster during task execution is obtained.

[0032] Furthermore, the target-aware twin graph attention network includes

[0033] Feature extraction twin network, used to extract feature templates and search image features to implement the target perception area selection mechanism;

[0034] The target information is embedded in the graph attention subnetwork. Based on the graph attention mechanism, the aggregated expression of the graph nodes is calculated, and the aggregated features are combined with the original features to form an enhanced feature expression, generating a response graph for classification and regression.

[0035] The target localization classification and regression subnetwork includes a classification branch for predicting the category information of each position cluster mode and a regression branch for calculating the target bounding box of the position; the classification branch and the regression branch share the response map output by the target information embedding map attention subnetwork.

[0036] A system adopting the magnetic field-based multimodal behavior control method of a micro-nano robot cluster, the system comprising a control board, a power amplifier, a three-dimensional AC Helmholtz coil, a three-dimensional mobile platform, a microscopic observation platform, a base, a gantry support frame, a microscopic observation system, an image acquisition system, an anti-vibration platform and a host computer control device; the control board is connected to the power amplifier; the power amplifier is connected to the three-dimensional AC Helmholtz coil; the three-dimensional AC Helmholtz coil, the three-dimensional mobile platform and the microscopic observation platform are arranged on the base; the microscopic observation system and the image acquisition system are carried on the gantry support frame; the anti-vibration platform is equipped with a base and a gantry support frame; the host computer control device is connected to the control board and sends a control signal to the control board via a local area network.

[0037] Furthermore, there are three power amplifiers; the control board is connected to the three power amplifiers respectively, and is used to convert the received digital control signal into an analog voltage signal and output it to the power amplifier; the three power amplifiers are all constant current power amplifiers, which are used to convert the received voltage signal output by the control board into a current signal and amplify it and output it to the three-dimensional AC Helmholtz coil.

[0038] Furthermore, the three-dimensional AC Helmholtz coil includes three groups of Helmholtz coils arranged perpendicularly to each other in pairs, and are nested and installed in sequence according to size; the three-dimensional AC Helmholtz coil also includes an inner support plate, a fixed connector and an outer support plate; the fixed connector connects the inner support plate and the outer support plate to form a U-shaped groove between the two; the U-shaped groove is used to place the coil winding of the three-dimensional AC Helmholtz coil; the coil winding adopts flat copper wire winding; the three groups of Helmholtz coils arranged perpendicularly to each other in pairs are all input with driving current amplified by the corresponding power amplifier, and generate a uniform magnetic field with a corresponding magnetic induction intensity amplitude at the center of the coil axis.

[0039] Furthermore, the three-dimensional mobile platform is used to control the three-dimensional position of the working area of ​​the micro-nano robot cluster, so as to realize the adjustment and focusing of the microscopic observation platform area; the microscopic observation platform is placed above the three-dimensional mobile platform, and is used to place the micro-nano robot cluster samples to be observed; the base is a base made of high-strength non-magnetic material; and the gantry support frame is an external support structure.

[0040] Furthermore, the microscopic observation system includes a microscope stand, a spindle accessory, an observation tube, an eyepiece, an objective lens, an objective lens converter, a halogen light source, a color filter, a fluorescent light source, a fluorescent excitation block and an optical path extension structure; the optical path extension structure connects the objective lens converter and the objective lens, and is used for long-distance optical path extension under a gantry support frame; the image acquisition system includes a high-sensitivity camera for image acquisition and microscopic imaging software for image display and analysis; the host computer control device has manual debugging, automatic control, handle automatic control, sequence parameter table import control and external command control functions.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The magnetic field-based multimodal behavior control method of micro-nano robot clusters provided by the present invention adopts a top-down approach to efficiently and orderly decompose application-oriented strategy design and strategy implementation tasks, which can be easily combined and expanded, providing technical support for the diversity and innovative applications of micro-nano robot cluster modes.

[0043] Second, the multimodal identification and tracking method of micro-nano robot clusters provided by the present invention introduces a graph attention mechanism to propagate the modal information of the target micro-nano robot cluster from the template feature to the search feature, and establishes an enhanced feature expression, which is conducive to the rapid identification of the target micro-nano robot cluster. The target perception area selection mechanism is introduced to adapt to the size and aspect ratio changes of different micro-nano robot clusters, solve the problem of target loss caused by target gradient in the target tracking process, and provide robustness for target tracking in the multimodal evolution process of micro-nano robot clusters.

[0044] 3. The present invention provides a generalized method for constructing a magnetic field expression in two steps. First, a simple basic vector is found and the relationship between the dimensional vectors is constructed to obtain the characteristic direction of the magnetic field. Then, the amplitude and change law of the magnetic induction intensity in the characteristic direction of the magnetic field are designed, and finally the magnetic field expression can be easily obtained.

[0045] 4. The multimodal behavior control magnetic field system for micro-nano robot clusters provided by the present invention provides a magnetic field system that integrates multi-type perception, scalable programming, friendly human-computer interaction and closed-loop control algorithm, which is suitable for various application scenarios and high-precision control requirements in the field of micro-nano robots.

[0046] 5. The multi-dimensional magnetic field design provided by the present invention enhances the movement diversity of the micro-nano robot cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a structural block diagram of the method for controlling multimodal behavior of a micro-nano robot cluster based on magnetic field of the present invention;

[0048] Figure 2 Design a flow chart for the control method of the present invention;

[0049] Figure 3 It is a schematic diagram of the magnetic induction intensity vector of a one-dimensional magnetic field of the present invention;

[0050] Figure 4 It is a schematic diagram of the magnetic induction intensity vector of the two-dimensional polarization rotating field of the present invention;

[0051] Figure 5 It is a schematic diagram of the two-dimensional oscillating field magnetic induction intensity vector of the present invention;

[0052] Figure 6 is the two-dimensional constant amplitude oscillation field of the present invention, δ∈(0, 2π] when B co_|| and B co_⊥ Amplitude variation law diagram of ;

[0053] Figure 7 It is a schematic diagram of the magnetic induction intensity vector of the three-dimensional conical magnetic field of the present invention;

[0054] Figure 8 It is a structural diagram of the magnetic field system of the present invention;

[0055] Fig. 9 It is the principle block diagram of the constant current power amplifier of the present invention;

[0056] Fig.10 It is a schematic diagram of the structure of a three-dimensional AC Helmholtz coil of the present invention;

[0057] Fig.11 This is a simulation diagram of the magnetic field distribution and uniformity of the z-axis coil of the present invention;

[0058] Fig.12 It is a simulation diagram of the magnetic field distribution and uniformity of the Y-axis coil of the present invention;

[0059] Fig.13 It is a simulation diagram of the magnetic field distribution and uniformity of the Z-axis coil of the present invention.

[0060] The numbers in the figure are as follows:

[0061] 1. Control board, 2. Power amplifier, 3. Three-dimensional AC Helmholtz coil, 4. Three-dimensional mobile platform, 5. Microscopic observation platform, 6. Base, 7. Gantry support frame, 8. Microscopic observation system, 9. Image acquisition system, 10. Anti-vibration platform, 11. Host computer control equipment. DETAILED DESCRIPTION

[0062] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0063] Example

[0064] Magnetic micro-nano robot clusters are generally composed of Fe3O4 nanoparticle monomers with good magnetic properties. The size, morphology and magnetic moment of the particles can be changed by controlling the reaction conditions of the preparation system, and the particles can be further given good biocompatibility and monodispersity through surface modification. Under the influence of external driving magnetic fields, fluid resistance fields, and boundary conditions, robot monomers exhibit different kinematic behaviors, such as rotation, rolling, and oscillation. Due to the coupling between robot monomers, clusters composed of multiple robots emerge with multimodal behaviors. By modulating the external magnetic field, the action switching and mode generation of the robot cluster can be conveniently realized to achieve the execution of complex tasks in unstructured constrained environments.

[0065] like Figure 1 and 2 As shown, a method for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field comprises the following steps:

[0066] S1. According to the pre-set deterministic task, determine whether the micro-nano robot cluster is performing a single-step simple task or a multi-step complex task. If it is a single-step simple task, proceed to step S3; if it is a multi-step complex task, proceed to step S2;

[0067] S2, determining a timing combination scheme for regulation according to the sequence of actions and modal behaviors of the micro-nano robot cluster required for the deterministic task, and executing step S3;

[0068] S3. According to the dynamic model revealed by the interaction between the micro-nano robot cluster and the external driving magnetic field, fluid resistance field, boundary conditions and monomers, determine the magnetic field dimension and type required for the cluster's action and mode realization at each step;

[0069] S4. Establish a multi-dimensional magnetic induction intensity expression according to the required magnetic field dimension and type to achieve multi-modal behavior control of micro-nano robot clusters;

[0070] S5. According to the cluster multimodal recognition and tracking method, the position and state of the micro-nano robot cluster are obtained in real time, and effective tracking and information feedback of the cluster are achieved.

[0071] The key to regulating the multimodal motion of a micro-nano robot cluster is to establish a magnetic field expression corresponding to the cluster mode, namely, a magnetic induction intensity expression, and construct a mapping relationship between the two from the dimensions of time and space. Optionally, the magnetic field includes a one-dimensional magnetic field, a two-dimensional magnetic field, and a three-dimensional magnetic field.

[0072] The one-dimensional magnetic field includes a static magnetic field and a pulsed alternating magnetic field. The static magnetic field can be used for the initial magnetization of micro-nano robot clusters and the assembly of chain-shaped, ribbon-shaped and other clusters. The pulsed alternating magnetic field can be used for the reshaping of the mode of micro-nano robot clusters, that is, to restore the existing mode to the initial diffuse mode.

[0073] The two-dimensional magnetic field includes a circular polarization rotating field, an elliptical polarization rotating field, a variable amplitude oscillation field and a constant amplitude oscillation field, wherein the circular polarization rotating field can be used for a micro-nano robot cluster to form a vortex-shaped mode, and the two-dimensional plane motion of the cluster can be realized by changing the polarization plane axis, and the elliptical polarization rotating field has the above-mentioned function of the circular polarization rotating field, and further, because its polarization plane has a long and short axis, it can realize the extension and shortening of the cluster dimension, construct an ellipsoidal cluster, and enhance the environmental adaptability of the micro-nano robot cluster; the variable amplitude oscillation field can be used for a micro-nano robot cluster to form a disk-shaped mode, and the climbing motion of the cluster in a three-dimensional environment can be realized by alternating the oscillation plane, and the constant amplitude oscillation field has the above-mentioned function of the variable amplitude oscillation field, and because the amplitude of the magnetic induction intensity in the oscillation plane remains unchanged, it is beneficial to maintain the shape of the cluster, and enhances the motion robustness of the cluster during application.

[0074] The three-dimensional magnetic field includes a conical magnetic field, which can drive the micro-nano robot cluster to achieve three-dimensional movement, thereby enhancing the movement diversity of the cluster.

[0075] The magnetic induction intensity expression of the above magnetic field should include information such as the amplitude, frequency and (direction) angle of the magnetic field. The method of establishing it is as follows:

[0076] 1) The establishment of the expression of one-dimensional magnetic field magnetic induction intensity includes the following steps:

[0077] S4011: Determine the type of one-dimensional magnetic field. If it is a static magnetic field, the expression of magnetic induction intensity is:

[0078] B s =B s e c

[0079] Among them, B s is the magnetic induction intensity amplitude, e c is the direction of the static magnetic field in three-dimensional space, and the magnetic induction intensity B s The angles θ and Confirm. If it is a pulse alternating magnetic field, go to S4012;

[0080] S4012: If it is a pulse alternating magnetic field, the expression of magnetic induction intensity is:

[0081] B p =B p u(t)e c

[0082] Among them, B p is the magnetic induction intensity amplitude, u(t) is the pulse magnetic field function, e c is the direction of the pulse alternating magnetic field in three-dimensional space, and the magnetic induction intensity B p The angles θ and Determine; the u(t) is expressed as:

[0083]

[0084] Wherein, k∈{0, 1, 2...}, DC=1%~99%, and f is the magnetic field alternating frequency.

[0085] like Figure 3 As shown, it is the magnetic induction intensity vector B of the static magnetic field and the pulse alternating magnetic field s , B p Schematic diagram of the changes.

[0086] 2) The establishment of the expression of magnetic induction intensity of two-dimensional magnetic field includes the following steps:

[0087] S4021: Determine the type of the two-dimensional magnetic field. If it is a polarization rotating field, go to S4022; if it is an oscillating field, go to S4023;

[0088] S4022: To establish the expression of the magnetic induction intensity of the two-dimensional polarization rotating field, we first need to establish the spatial coordinates of the rotation axis vector of the polarization rotating field. The direction of the two-dimensional polarization rotating field can be uniquely determined by the normal direction of the rotating surface, that is, the direction of the rotation axis. Figure 4 The spatial coordinates of the rotation axis unit vector n shown in a. In the spatial coordinate system, the center of the polarization rotation field is located at the origin of the coordinate system, and the angle between the rotation axis vector n and the z axis is Projection n on the xoy plane p The angle with the x-axis is θ.

[0089] In the Cartesian three-dimensional space coordinate system, n can be expressed by θ and Defined as:

[0090]

[0091] Then define the rotation plane with n as the rotation axis, that is, find a pair of vectors on the plane that are perpendicular to n and orthogonal to each other.

[0092] Find the unit vector l1 in the xoy plane and the projection n of n in the xoy plane p Vertical, which ensures n⊥l1:

[0093] l1=(sinθ,-cosθ,0) T

[0094] Solve for the unit vector l2=(x, y, z) that is perpendicular to n and l1 T , that is, the inner product is 0:

[0095]

[0096] The coordinates of l2 are solved as follows:

[0097] like Figure 4 As shown in b, l1 and l2 are taken as the directions of the two orthogonal components of the polarization rotation field, and the direction of the l1+l2 synthetic vector is the magnetic induction intensity vector B. c In the direction of the magnetic induction intensity on each component, when the magnitude changes sinusoidally, and the amplitude is equal and the phase difference is 90°, a circular polarization rotating field rotating around n can be formed.

[0098]

[0099] Set the combined magnetic field B c =B1+B2 has a rotation frequency of f, and the magnetic induction intensity vector B cPerform a rectangular coordinate transformation and decompose it into a three-dimensional space coordinate system, as shown in the following formula:

[0100]

[0101] The spatial coordinates of the rotation axis vector of the elliptical polarization rotating field are the same as those of the circular polarization rotating field ( Figure 4 (a)), that is, the direction of the l1+l2 synthetic vector is the magnetic induction intensity vector B ep direction. Figure 4 c, the magnitude of the magnetic induction intensity of each vector component on the rotating plane changes sinusoidally, and the amplitude has a coefficient k (k∈N * , k = 1 is a multiple of the circular polarization rotating field), respectively constituting the major axis vector B of the elliptical polarization rotating field ep_max and the minor axis vector B ep_min , the phase difference is still 90°, which can form an elliptical polarization rotating field rotating around n:

[0102]

[0103] Combined magnetic field B ep =B1+B2 has a rotation frequency of f, and the magnetic induction intensity vector B ep Perform a rectangular coordinate transformation and decompose it into a three-dimensional space coordinate system, as shown in the following formula:

[0104]

[0105] S403: Figure 5 As shown in a(1), to establish the expression of the magnetic induction intensity of the two-dimensional oscillating field, we first need to establish the spatial coordinates of the normal line of the oscillation plane and select the unit vector As the normal direction, the vertex of the oscillation plane is located at the origin of the coordinate system. Then set the baseline vector l (for convenience, select it in the xoy plane, the coordinate is l = (sinθ, -cosθ, 0) T ) and the two-dimensional swing angle δ, the boundaries of the oscillation plane are defined by the two. Figure 5 As shown in a(2), the angular velocity of the baseline vector is ω = 2πf. After a period of time t, it swings to the position of vector l'. Now take the initial baseline as the vertical axis and the component l' perpendicular to it ⊥ As the horizontal axis, analyze the relationship between the vectors in the oscillation plane, where:

[0106] l′=l+l′ ⊥

[0107] The vector l′ ⊥ Move the baseline vector l in the opposite direction by a distance |l| until its starting point coincides with the starting point of the normal vector n, that is, the coordinate origin o, and Figure 5 a(2) Rotate a certain angle to get Figure 5 a(3), that is, the direction of n×l is l′ ⊥ direction. Figure 5 As shown in b, l and l′ ⊥ As the directions of the two orthogonal components of the oscillation plane, l is the direction of the baseline magnetic induction intensity, l′ ⊥ is the direction of the magnetic induction intensity perpendicular to the baseline, then l'=l+l' ⊥ The direction of the synthetic vector is the direction of the magnetic induction intensity vector of the oscillating field. c unchanged, l′ ⊥ Magnetic induction intensity vector B v When a sinusoidal change is made (ω=2πf), a two-dimensional variable amplitude oscillation field with l as the baseline can be formed, where:

[0108] B vo =B c l+B v_max sin(ωt)·(n×l)

[0109] Assume the oscillation frequency of the magnetic field is f, and the magnetic field intensity vector B vo Perform a rectangular coordinate transformation and decompose it into a three-dimensional space coordinate system, as shown in the following formula:

[0110]

[0111] like Figure 5 As shown in c(1), the normal, baseline and two-dimensional swing angle of the two-dimensional fixed amplitude oscillation field are consistent with the settings of the two-dimensional variable amplitude oscillation field. Figure 5 As shown in c(2), since the amplitude of the constant amplitude oscillation field remains unchanged over time, that is, |l′|=|l|, for the convenience of analysis, the vector l′ is decomposed into l′=l′ || +l′ ⊥ . The vector l′ ⊥ Move distance |l′ in the opposite direction of baseline vector l || | until its starting point coincides with the starting point of the normal vector n, that is, the coordinate origin o, and Figure 5 c(2) rotates a certain angle to get Figure 5 c(3), that is, the direction of n×l is l′ ⊥ direction. Figure 5 d, l′ || and l′ ⊥ As the direction of the two orthogonal components of the constant amplitude oscillation plane, l′ || is the direction of the baseline magnetic induction intensity, l′ ⊥ is the direction of the magnetic induction intensity perpendicular to the baseline, then l'=l' || +l′ ⊥The direction of the synthetic vector is the constant amplitude oscillation field magnetic induction intensity vector B co direction, respectively set l′ || and l′ ⊥ The magnetic flux density vector B in the L direction co_|| and B co_⊥ , we can get the constant amplitude magnetic induction intensity vector.

[0112] In particular, when δ is in different phases, B co_|| and B co_⊥ The amplitude changes as Figure 6 shown.

[0113] 3) The expression of three-dimensional magnetic field magnetic induction intensity is established as follows:

[0114] like Figure 7 As shown in a, the expression of the magnetic induction intensity of the three-dimensional magnetic field, i.e. the conical magnetic field, is established by first analyzing the direction of the magnetic field. The direction of the conical magnetic field rotating around an arbitrary axis with the coordinate origin as the vertex in three-dimensional space can be determined by the normal of the cone base, i.e. the rotation axis vector n, and the angle α between the cone surface vector l and the rotation axis, as shown in Figure 7 a. In the Cartesian three-dimensional space coordinate system, n can be expressed by θ and Defined as:

[0115]

[0116] In the vectors n and n p Find the cone vector l in the plane formed, then the coordinates of l are:

[0117]

[0118] It is easy to imagine that l can form a cone by rotating around n once. Specifically, l is decomposed into two vector components along the rotation axis and perpendicular to the rotation axis. The vector component l parallel to the rotation axis is || Does not rotate, perpendicular vector l ⊥ It rotates around n in the base of the cone, with an angular velocity of ω=2πf. After a period of time t, the rotation angle is ωt, and the vector after rotation is l' ⊥ .

[0119] like Figure 7 As shown in b(1), the starting point of the rotation axis n is the origin, n is the vertical axis, l ⊥ is the horizontal axis, and l ⊥ , n perpendicular vector j is another horizontal axis to build the spatial relationship of each vector:

[0120] l || =|l|cos(α)·n=(l·n)·n

[0121] l⊥ =ll || =l-(l·n)·n

[0122] The cone vector l′ after rotation angle ωt = l || +l′ ⊥ , now || From the known vector, we can get l' ⊥ , using a bird's-eye view Figure 7 b(2) is analyzed, l′ ⊥ Can be decomposed into l ⊥ Direction|l ⊥ |cos(ωt) and n×l ⊥ Direction|l ⊥ |sin(ωt). Figure 7 b(2) is rotated by a certain angle to obtain Figure 7 b(3), we know that n×l ⊥ The direction is the direction of j. That is, l' ⊥ It can be expressed as:

[0123]

[0124] like Figure 7 c, and because |l ⊥ |=|j|,j=n×l ⊥ =n×l, so l′ ⊥ = l ⊥ cos(ωt)+(n×l)sin(ωt). will l ⊥ and (n×l) as the directions of the two orthogonal components of the rotation base, l || is the direction of the magnetic induction intensity along the rotation axis, then l'=l || +l′ ⊥ The direction of the resultant vector is the magnetic induction intensity vector B of the conical magnetic field. conical direction, when l || The magnetic induction intensity in the direction remains unchanged, l ⊥ When the magnitude of the magnetic induction intensity on the (n×l) component changes sinusoidally, and the amplitudes are equal and the phase difference is 90°, a conical rotating magnetic field rotating around n can be formed.

[0125] B conical =B n (l·n)·n+B0cos(ωt)·l ⊥ +B0sin(ωt)·(n×l)

[0126] Assume the rotation frequency of the magnetic field is f, and the magnetic field intensity vector B conical Perform a rectangular coordinate transformation and decompose it into a three-dimensional space coordinate system, as shown in the following formula:

[0127]

[0128] Where ω=2πf,

[0129] During the task execution process, the micro-nano robot cluster needs to monitor the status of the cluster in real time, including the completion of task execution and action switching, the dynamic characteristics of the cluster modal evolution, and the series of positions during the cluster movement.

[0130] Traditional manual monitoring methods have defects. On the one hand, manual observation and recording are delayed, and effective information cannot be recorded in real time. On the other hand, the human-led judgment method will introduce subjective errors, and the characteristic parameters of the cluster mode cannot be objectively and effectively identified, and the monitoring accuracy is lacking. The monitoring method that uses image sensors combined with target recognition and tracking algorithms can effectively solve the above problems based on high frame rate cameras, feature data sets, and deep neural network training models, and realize the recognition of cluster modes and dynamic target tracking in mission scene sequences.

[0131] The cluster multimodal recognition and tracking method comprises the following steps:

[0132] S501: Based on the single modality of the micro-nano robot cluster and the image sequence of potential application scenarios, manually add annotations (boxes) to extract the basic features of the cluster modality;

[0133] The network training for target recognition and tracking requires a feature dataset, which contains the foreground and background information of the target features. Therefore, image sequences of micro-nano robot clusters in different application scenarios are collected, which can optionally include scenes such as vascular targeting, cargo transportation, and unstructured mazes. The size and resolution of the image are determined by the acquisition camera. In order to facilitate batch processing and increase the image processing speed, the collected images are grayed and the image resolution is adjusted to 127×127. The open source image annotation software labelme on Github is used to manually calibrate the modality of the micro-nano robot cluster, so that the size of the annotation box contains all the information of the modality as much as possible without other background information.

[0134] S502: Expand data based on the acquired basic features to construct a feature data set;

[0135] S503: Realize real-time dynamic tracking of micro-nano robot cluster tasks based on target-aware twin graph attention network (SiamGAT);

[0136] The model training in this embodiment is based on Pytorch 1.2.0, CUDA 9.0 parallel development framework, using Python language, and the specific experimental environment is: Ubuntu 16.04+Python 3.5+Pytorch 1.2.0+CUDA 9.0. The model-target perception twin graph attention network structure adopted is based on the open source target tracker of Zhejiang University of Technology (Guo, D., et al. "GraphAttention Tracking." 2020.), and the network structure mainly includes three modules: feature extraction twin sub-network, target information embedding graph attention sub-network and target positioning classification regression sub-network.

[0137] The feature extraction twin network is used to extract the feature template and search the image feature F t and F s Each 1×1×c tensor of the feature map is considered as a node (part), where c represents the number of feature channels, V t and V s F t and F s In particular, for F t In order to solve the problem of a large amount of background information and lost foreground information introduced by using a fixed-size area as a template feature, only the micro-nano robot cluster features within the annotation box are used as the area of ​​interest, and the values ​​of other nodes outside the annotation box to complete batch processing are set to 0, that is:

[0138]

[0139] The target perception area selection mechanism is implemented to adapt to the changes in size and aspect ratio of different micro-nano robot clusters, and to solve the problem of target loss caused by target gradual change during target tracking. It is suitable for the multimodal evolution process of micro-nano robot clusters and provides robustness for cluster target tracking.

[0140] The target information embedding graph attention subnetwork is based on the aforementioned feature extraction node set V t and V s Construct a bipartite graph G = (V, E), where V = V s ∪V t , The sub-graphs of feature template and search image are and Then for each pair of graph nodes (i, j)∈E, we create a description node i∈V s and node j∈V t Evaluation of the correlation scores between:

[0141]

[0142] in and represents the feature vector of nodes i and j, W s and W y is a linear transformation matrix. In order to balance the amount of information, e ij Perform softmax normalization:

[0143]

[0144] Among them, a ij It can be intuitively understood as giving attention to the search graph node i based on the information of the template node j. Then, using G t All nodes in G are passed to s The attention of the i-th node in , that is, the aggregate expression of node i can be calculated: Finally, the aggregated features are combined with the original features to form an enhanced feature expression: Produce response plots for categorical regression.

[0145] The target localization classification and regression subnetwork includes a classification branch for predicting the category information of each location cluster modality and a target bounding box regression branch for calculating the location. These two branches share the response map output by the aforementioned image attention module.

[0146] The image of the search area is adjusted to a grayscale image and resized to 287×287, and input into the above-mentioned target perception twin graph attention network together with the feature template. Through model training, state monitoring and real-time dynamic tracking of micro-nano robot cluster tasks can be achieved.

[0147] S504: Based on the output of the classification regression subnetwork, the position and state of the micro-nano robot cluster during the task execution process are obtained.

[0148] The position and state of the micro-nano robot cluster are expressed as:

[0149] P=[(x s ,y s , z s ), (x1, y1, z1), ..., (x e ,y e , z e )] T

[0150]

[0151] Among them, P∈R represents the center position of the micro-nano robot cluster, that is, the center position of the target bounding box, and S∈R represents the mode of the micro-nano robot cluster and the corresponding characteristic parameters under the mode. Optionally, the modes of the micro-nano robot cluster include diffuse, disc, vortex, ellipsoid, chain, ribbon, etc. The number of characteristic parameters between different modes is inconsistent. When storing data, the array should be established with the modal dimension with the most dimensional parameters. When the characteristic parameters of other modes are insufficient, 0 parameters should be added. The subscripts s and e represent the initial and terminal states during the task execution, respectively.

[0152] like Figure 8 As shown, a system for realizing a multimodal behavior control method of a micro-nano robot cluster based on a magnetic field comprises a control board 1, a power amplifier 2, a three-dimensional AC Helmholtz coil 3, a three-dimensional mobile platform 4, a microscopic observation platform 5, a base 6, a gantry support frame 7, a microscopic observation system 8, an image acquisition system 9, an anti-vibration platform 10 and an upper computer control device 11; the control board 1, the power amplifier 2 and the three-dimensional AC Helmholtz coil 3 are connected in sequence; the three-dimensional AC Helmholtz coil 3, the three-dimensional mobile platform 4 and the microscopic observation platform 5 are carried on the base 6; the microscopic observation system 8 and the image acquisition system 9 are carried on the gantry support frame 7; the base 6 and the gantry support frame 7 are arranged on the anti-vibration platform 10; the upper computer control device 11 is connected to the control board 1 and sends a control signal to the control board 1 through a local area network.

[0153] The control board 1 is used to convert the received digital control signal into an analog voltage signal and output it to the power amplifier 2, and the output 3 channels of the control board 1 are respectively connected to the three power amplifiers 2;

[0154] like Fig. 9 As shown, the power amplifier 2 is a constant current power amplifier, which is used to convert the voltage signal received from the control board 1 into a current signal and amplify and output it to the three-dimensional AC Helmholtz coil 3 for coil driving; there are three power amplifiers 2, and the three power amplifiers 2 are power amplifiers of different specifications, which are respectively used to drive Helmholtz coils of different dimensions. The specific parameters are shown in Table 1.

[0155] Table 1 Power amplifier parameters

[0156]

[0157] like Fig.10As shown, it is the front view, side view and three-dimensional stereogram of the structure of the three-dimensional AC Helmholtz coil 3. The three-dimensional AC Helmholtz coil 3 includes three groups of Helmholtz coils arranged perpendicular to each other in pairs, and nested and installed in sequence according to size. The axial directions of the three groups of Helmholtz coils correspond to the X, Y, and Z axes of the magnetic field system respectively, and the axial centers of the three groups of Helmholtz coils intersect at the coordinate origin o. The support structure of the three-dimensional AC Helmholtz coil 3 includes an inner support plate, a fixed connector and an outer support plate. A plurality of fixed connectors connect the inner support plate and the outer support plate and form a U-shaped groove between the two. The U-shaped groove is used to place the Helmholtz coil winding. Optionally, the inner support plate and the outer support plate are high-strength epoxy resins, and the fixed connectors use non-magnetic aviation plugs and oxygen-free copper screws. The coil winding is made of oxygen-free yarn wrapped wire, which has good corona resistance, strong overload capacity, high temperature resistance, high insulation level, and relatively low flow rate per unit area relative to the maximum current of 30A, 40A, and 50A output by power amplifier 2, which can reduce current density, reduce heat generation, and ensure long-term use. Furthermore, the use of flat copper wire winding, the wiring is regular and tight, and the component magnetic field is effectively reduced.

[0158] Each set of coils is input with a driving current amplified by the corresponding power amplifier 2, and the relationship between the current and the magnetic induction intensity is defined according to the simplified Biot-Savart law:

[0159]

[0160] A uniform magnetic field of corresponding magnetic induction intensity amplitude is generated at the center of the coil axis, where μ0 is the vacuum magnetic permeability, I is the current passed through the coil, and R is the average diameter of the coil. Optionally, the coil is engineered according to the set technical indicators such as the equivalent diameter of the three-axis coil, the volume and distribution uniformity of the magnetic field uniform area, the magnetic field intensity, and the working time as shown in Table 2.

[0161] Table 2 Engineering Technical Specifications

[0162]

[0163] The specific parameters of the three-dimensional AC Helmholtz coil 3 are shown in Table 3.

[0164] Table 3 Specific parameters

[0165]

[0166]

[0167] The evaluation method for the uniformity of magnetic field distribution is

[0168]

[0169] Among them, B0 is the magnetic induction intensity at the center of the axis, which is the reference point for the magnetic field generated by the Helmholtz coil. The coil constant, whether theoretical or measured, is based on this reference point. P B is the magnetic induction intensity in a spherical area with a radius r and a certain scale, with the axis center as the origin. P -B0 is the magnetic induction intensity B P The difference between B0 and B0 is normalized by B0 to obtain the magnetic field distribution uniformity s. Optionally, the magnetic field distribution and uniformity of the Z-axis, Y-axis and X-axis coils are simulated using Maxwell 16.0 software, as shown in Fig.11 , 12 , as shown in 13.

[0170] The three-dimensional mobile platform 4 is used to control the three-dimensional position of the working area of ​​the micro-nano robot cluster, and realize the adjustment and focusing of the microscopic observation platform 5 area.

[0171] The microscopic observation platform 5 is placed above the three-dimensional moving platform 4 for placing the micro-nano robot cluster sample to be observed.

[0172] The base 6 carries a three-dimensional AC Helmholtz coil 3, a three-dimensional moving platform 4 and a microscopic observation platform 5. The base 6 is selected from high-strength non-magnetic materials to ensure a stable structure. Optionally, high-strength epoxy resin is used.

[0173] The gantry support frame 7 carries a microscopic observation system 8 and an image acquisition system 9. The external support structure is light, easy to assemble and expandable, which is beneficial to simplifying the structure of the magnetic field system, reducing the weight and space limitations brought by the original microscopic observation system 8 support structure, and facilitating the integration of the microscopic observation system 8 and the image acquisition system 9.

[0174] The microscopic observation system 9 includes a microscope stand, a spindle accessory, an observation tube, an eyepiece, an objective lens, an objective lens converter, a halogen light source, a color filter, a fluorescent light source, a fluorescent excitation block and an optical path extension structure; the microscope stand (including a focusing device) is selected as BX3-MFB, the spindle accessory is selected as BX3-URA, with an 8-hole turntable structure, the observation tube is selected as a trinocular observation tube U-TR30-2, and the eyepiece is selected as 10X WHN10X and 10X diopter adjustable eyepieces WHN10X-H, objective lenses with magnifications of 5X-MPLN5X, 10X-MPLFLN10X and 20X-MPLFLN20X are selected for observation of micro-nano robot clusters of different sizes, the objective lens converter selects the 5-position objective lens converter U-5RE-2, and the halogen light source (halogen lamp bulb-JC12V100WHAL-L, halogen lamp housing-U-LH100-3, halogen lamp housing extension cable-U-RMT, 100W halogen lamp power supply-TH4-200) is used for micro-nano machines For bright-field imaging of the robot cluster, the color filters U-25LBD and U-25IF are selected for distortion correction during halogen light source imaging. The fluorescent light source is the wide-spectrum white light LED fluorescent light source FC904, and the fluorescence excitation blocks are FS302 and U-FF. The optical path extension structure (sleeve SM1 (inside & outside) L=102.4mm-SM1L40, RMS (inside) to SM1 (outside) adapter ring-SM1A3TS, RMS external thread and SM1 internal thread adapter ring-SM1BC) is used to connect the objective lens converter and the objective lens to realize long-distance imaging of the magnetic field system under the gantry support frame 7.

[0175] The image acquisition system 9 includes a back-illuminated high-sensitivity black-and-white camera BioHD-EPS, a camera global shutter of 2.3 million pixels, a full-frame camera speed of 50fps, a 1x C-mount, and professional microscopic imaging software VisiView, which supports multi-channel bright field fluorescence acquisition and overlay, high-speed acquisition of time series and image streams, and automatic adjustment of dynamic display range. The software supports image measurement functions including statistics such as ruler, length, area, brightness, and brightness change curve over time.

[0176] The host computer control device 11 sends a control signal to the control board 1 through the local area network. The host computer control device 11 integrates various debugging and control methods such as manual debugging, automatic control, handle automatic control, sequence parameter table import control and external command control to meet various task requirements during the experiment.

[0177] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field, characterized in that: The method comprises the following steps: S1. According to the pre-set deterministic task, determine whether the micro-nano robot cluster is performing a single-step simple task or a multi-step complex task. If it is a single-step simple task, proceed to step S3; if it is a multi-step complex task, proceed to step S2; S2, determining a timing combination scheme for regulation according to the sequence of actions and modal behaviors of the micro-nano robot cluster required for the deterministic task, and executing step S3; S3. According to the dynamic model revealed by the interaction between the micro-nano robot cluster and the external driving magnetic field, fluid resistance field, boundary conditions and monomers, determine the magnetic field dimension and type required for the cluster's action and mode realization at each step; S4. Establish a multi-dimensional magnetic induction intensity expression according to the required magnetic field dimension and type to achieve multi-modal behavior control of micro-nano robot clusters; S5. According to the cluster multimodal recognition and tracking method, the position and state of the micro-nano robot cluster are obtained in real time, and effective tracking and information feedback of the cluster are achieved; The magnetic field is a multi-dimensional magnetic field, including a one-dimensional magnetic field, a two-dimensional magnetic field and a three-dimensional magnetic field; the one-dimensional magnetic field includes a static magnetic field and a pulsed alternating magnetic field; the two-dimensional magnetic field includes a polarization rotation field and an oscillating field; the three-dimensional magnetic field includes a conical magnetic field; the magnetic induction intensity expression includes the amplitude, frequency and angular direction information of the multi-dimensional magnetic field; The establishment of the one-dimensional magnetic field magnetic induction intensity expression comprises the following steps: S4011: Determine the type of one-dimensional magnetic field. If it is a static magnetic field, the expression of magnetic induction intensity is: B s =B s e c Among them, B s is the magnetic induction intensity amplitude, e c is the direction of the static magnetic field in three-dimensional space; if it is a pulse alternating magnetic field, go to S4012; S4012: If it is a pulse alternating magnetic field, the expression of magnetic induction intensity is: B p =B p u(t)e c Among them, B p is the magnetic induction intensity amplitude, u(t) is the pulse magnetic field function, e c is the direction of the pulse alternating magnetic field in three-dimensional space; the u(t) is expressed as: Where, k∈{0,1,2…}, DC=1%~99%, f is the magnetic field alternating frequency; The establishment of the expression of the two-dimensional magnetic field magnetic induction intensity comprises the following steps: S4021: Determine the type of the two-dimensional magnetic field. If it is a polarization rotating field, go to S402; if it is an oscillating field, go to S4023; S4022: Establishment of the expression of the magnetic induction intensity of the two-dimensional polarized rotating field. First, the spatial coordinates of the rotation axis vector of the polarized rotating field need to be established, and the rotation plane needs to be defined. Then, based on the relationship between the orthogonal vectors, a two-dimensional vector expression rotating around the rotation axis is established. The amplitude and frequency of the magnetic induction intensity are further set. Finally, the magnetic induction intensity vector is transformed into a rectangular coordinate system and decomposed into a three-dimensional space coordinate system. S4023: Establishment of the expression of the magnetic induction intensity of the two-dimensional oscillating field. First, the spatial coordinates of the normal line of the oscillation plane need to be established, and the baseline vector and the two-dimensional swing angle need to be defined to determine the oscillation boundary. Then, the baseline is used as the vertical axis, and the vector perpendicular to the baseline is used as the horizontal axis. The relationship between the vectors in the oscillation plane is analyzed, and a two-dimensional oscillation vector expression with the baseline as the central axis is established. The magnetic induction intensity amplitude and the oscillation frequency are further set. Finally, the magnetic induction intensity vector is transformed into a rectangular coordinate system and decomposed into a three-dimensional space coordinate system. The establishment of the three-dimensional magnetic field magnetic induction intensity expression first analyzes the direction of the magnetic field. The direction of the conical magnetic field rotating around an arbitrary axis with the coordinate origin as the vertex in the three-dimensional space can be determined by the normal of the cone base, that is, the rotation axis vector and the angle between the cone surface vector and the rotation axis. The rotation axis vector space coordinates are established, the cone surface vector and the angle between the cone surface vector and the rotation axis are defined, and then the cone surface vector is decomposed into two component vectors along the rotation axis and perpendicular to the rotation axis. The component vector parallel to the rotation axis does not rotate, and the perpendicular component vector rotates around the rotation axis vector on the cone base. According to the relationship between the space vectors, the expression of the three-dimensional conical vector rotating around the rotation axis is established, and the magnetic induction intensity amplitude and the rotation frequency are further set. Finally, the magnetic induction intensity vector is transformed into rectangular coordinates and decomposed into a three-dimensional space coordinate system.

2. The method for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field according to claim 1, characterized in that: The cluster multimodal recognition and tracking method specifically includes the following steps: S501: Based on the single modality of the micro-nano robot cluster and the image sequence of potential application scenarios, manually add annotations to extract the basic features of the cluster modality; S502: Expand data based on the acquired basic features to construct a feature data set; S503: Based on the target-aware twin graph attention network, real-time dynamic tracking of the micro-nano robot cluster during task execution is realized, and the position and status information of the micro-nano robot cluster during task execution is obtained.

3. The method for controlling multimodal behavior of a micro-nano robot cluster based on a magnetic field according to claim 2, characterized in that: The object-aware twin graph attention network includes Feature extraction twin network, used to extract feature templates and search image features to implement the target perception area selection mechanism; The target information is embedded in the graph attention subnetwork. Based on the graph attention mechanism, the aggregated expression of the graph nodes is calculated, and the aggregated features are combined with the original features to form an enhanced feature expression, generating a response graph for classification and regression. The target localization classification and regression subnetwork includes a classification branch for predicting the category information of the cluster modality at each location and a regression branch for calculating the target bounding box at that location; The classification branch and regression branch share the response map output by the target information embedding map attention sub-network.

4. A system using the magnetic field-based multimodal behavior control method of micro-nano robot clusters according to claim 1, characterized in that: The system comprises a control board (1), a power amplifier (2), a three-dimensional alternating current Helmholtz coil (3), a three-dimensional mobile platform (4), a microscopic observation platform (5), a base (6), a gantry support frame (7), a microscopic observation system (8), an image acquisition system (9), an anti-vibration platform (10) and a host computer control device (11); the control board (1) is connected to the power amplifier (2); the power amplifier (2) is connected to the three-dimensional alternating current Helmholtz coil (3); the three-dimensional alternating current Helmholtz coil (3), the three-dimensional mobile platform (4) and the microscopic observation platform (5) are arranged on the base (6); the microscopic observation system (8) and the image acquisition system (9) are carried on the gantry support frame (7); the anti-vibration platform (10) is carried on the base (6) and the gantry support frame (7); the host computer control device (11) is connected to the control board (1) and sends a control signal to the control board (1) through a local area network.

5. The system according to claim 4, characterized in that There are three power amplifiers (2); the control board (1) is respectively connected to the three power amplifiers (2) and is used to convert the received digital control signal into an analog voltage signal and output it to the power amplifier (2); the three power amplifiers (2) are all constant current power amplifiers and are used to convert the received voltage signal output by the control board (1) into a current signal and amplify and output it to the three-dimensional AC Helmholtz coil (3).

6. The system according to claim 4, characterized in that The three-dimensional AC Helmholtz coil (3) comprises three groups of Helmholtz coils arranged perpendicularly to each other in pairs, and are nested and installed in sequence according to their sizes; the three-dimensional AC Helmholtz coil (3) also comprises an inner support plate, a fixed connection piece and an outer support plate; the fixed connection piece connects and fixes the inner support plate and the outer support plate to form a U-shaped groove therebetween; the U-shaped groove is used to place the coil winding of the three-dimensional AC Helmholtz coil (3); the coil winding adopts a flat copper wire winding; the three groups of Helmholtz coils arranged perpendicularly to each other in pairs are all input with a driving current amplified by a corresponding power amplifier (2), and a uniform magnetic field with a corresponding magnetic induction intensity amplitude is generated at the center of the coil axis.

7. The system according to claim 4, characterized in that The three-dimensional mobile platform (4) is used to control the three-dimensional position of the micro-nano robot cluster working area, and realize the adjustment and focusing of the microscopic observation platform (5) area; the microscopic observation platform (5) is placed above the three-dimensional mobile platform (4) and is used to place the micro-nano robot cluster sample to be observed; the base (6) is a base made of high-strength non-magnetic material; and the gantry support frame (7) is an external support structure.

8. The system according to claim 4, characterized in that The microscopic observation system (8) comprises a microscope stand, a spindle accessory, an observation tube, an eyepiece, an objective lens, an objective lens converter, a halogen light source, a color filter, a fluorescent light source, a fluorescent excitation block and an optical path extension structure; the optical path extension structure connects the objective lens converter and the objective lens and is used for long-distance optical path extension under a gantry support frame; the image acquisition system (9) comprises a high-sensitivity camera for image acquisition and microscopic imaging software for image display and analysis; the host computer control device (11) has the functions of manual debugging, automatic control, handle automatic control, sequence parameter table import control and external command control.

Citation Information

Patent Citations

  • Rotating magnetic field generation system and rotating magnetic field implementation method thereof

    CN102820118B

  • A human-computer interaction control method for a space universal rotating magnetic field

    CN105662318B

  • Three-dimensional equal-diameter Helmholtz coil

    CN111508681A

  • Heterogeneous integration method of soft robot based on viscous transfer printing

    CN112476486A

  • Autonomous navigation and control method of magnetic micro-nano robot

    CN113671960A