Flow and manipulation method and device based on spherical modular self-reconfigurable robot

Abstract

The present invention discloses a flow and manipulation method, device, medium and terminal based on a spherical modular self-reconfigurable robot. The method comprises the following steps: obtaining environmental position information of each module, synthesizing all environmental position information obtained to obtain an overall environmental model or the shape and position of a manipulated object; inputting the overall environmental model or the shape and position of a manipulated object into a trained neural network for calculation to obtain a sequence of changes in the external contour of a modular self-reconfigurable robot cluster; planning the motion trajectory of each module based on the sequence of changes in the external contour of the modular self-reconfigurable robot cluster to obtain motion trajectory information of each module, and each module moves based on the motion trajectory information; the present invention can be applied to spherical modular self-reconfigurable robots, can simulate the flow and manipulation capabilities of water and multicellular tissues, and the method has better adaptability, versatility, expansibility and flexibility.

Description

Technical Field

[0001] The present invention relates to the field of robot control, and in particular to a flow and manipulation method and device based on a spherical modular self-reconfigurable robot. Background Art

[0002] Modular Self-Reconfiguration Robot (MSRR) can change the connection relationship between modules according to the requirements of the environment and tasks. The configuration of MSRR can be quadruped, snake, lizard, etc. The action sequence that converts the current configuration of the modular robot to the target configuration is called self-reconfiguration process (SR). The unique movement mode of MSRR is flow, which is composed of multiple continuous self-reconfiguration processes, that is, different modules are frequently disconnected or connected. Flow simulates the adaptability of water to different surfaces and reflects the unique advantages of MSRR in exploration and other aspects. Secondly, the effective interaction of MSRR with objects in real environments requires a strong manipulator arm. The manipulation method based on self-reconfiguration imitates the arm composed of cells that are constantly moving, and its manipulation of objects is smoother and more controllable. The collectively driven self-reconfiguration process can overcome the problem of reduced module function due to the reduction of module size, and synergistically obtain greater force, torque and manipulation capabilities than the sum of the capabilities of each module.

[0003] Existing modular self-reconfigurable robots lack corresponding flow and manipulation methods to control their operation. Existing modular robots mainly move by simulating animal muscles, bones and joints and are unable to simulate flow. Therefore, existing control methods are not applicable to modular self-reconfigurable robots, and existing manipulation methods have low adaptability, versatility, extensibility and flexibility. When surmounting obstacles, their obstacle surmounting stability and success rate are low. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a flow and manipulation method, device, medium and terminal based on a spherical modular self-reconfigurable robot, aiming to solve the flow and manipulation problem of how to self-reconfigure the modular self-reconfigurable robot.

[0005] In order to solve the above technical problems, the first aspect of the embodiment of the present application provides a flow and manipulation method based on a spherical modular self-reconfigurable robot, the method comprising:

[0006] Acquire the environmental position information of each module, synthesize all the acquired environmental position information, and obtain the overall environmental model or the shape and position of the manipulated object;

[0007] Inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster;

[0008] The motion trajectory of each module is planned based on the external contour change sequence of the modular self-reconfigurable robot cluster, and the motion trajectory information of each module is obtained. Each module moves based on the motion trajectory information.

[0009] As a further improved technical solution, the environmental position information of each module is obtained, and all the environmental position information obtained is synthesized to obtain the overall environmental model or the shape and position of the manipulated object, including:

[0010] Control each module to collect local environment information and module position information according to preset instructions, process each local environment information and module position information, obtain a local mesh model of each module and module 3D position information, and use the local mesh model of each module and module 3D position information as each environment position information;

[0011] Based on the 3D position information of the self, all the local mesh models are spliced ​​to obtain an overall mesh model;

[0012] The overall mesh model is detected to obtain the shape and 3D position of the manipulated object.

[0013] As a further improved technical solution, the processing of each of the local environment information and the module's own position information to obtain the local mesh model of each module and the module's own 3D position information includes:

[0014] Processing each of the local environment information through a depth estimation network to obtain a local mesh model of each of the modules;

[0015] The position information of each module is converted into global world coordinates relative to the position of the same main module, and the global world coordinates are used as the 3D position information of the module itself.

[0016] As a further improved technical solution, the overall environment model or the shape and position of the manipulated object is input into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster, including:

[0017] Using imitation learning and reinforcement learning methods to train the initial neural network to obtain the trained neural network;

[0018] The overall environment model or the shape and position of the manipulated object are input into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster.

[0019] As a further improved technical solution, the initial neural network is trained by using imitation learning and reinforcement learning methods, and the trained neural network includes:

[0020] Acquire a fluid surface profile change sequence when the magnetron fluid runs in the environment model, and train an initial teacher network based on the fluid surface profile change sequence and privileged information to obtain a target teacher network, wherein the privileged information includes the environment model and magnetron fluid properties;

[0021] Acquire actual observation information, and train a student network based on the actual observation information and a target teacher network to obtain a preliminary neural network, wherein the teacher network performs knowledge distillation on the student network based on a behavior loss function;

[0022] Perform reinforcement learning training on the preliminary neural network to obtain the trained neural network.

[0023] As a further improved technical solution, the motion trajectory of each module is planned based on the external contour change sequence of the modular self-reconfigurable robot cluster to obtain the motion trajectory information of each module, and each module moves based on the motion trajectory information, including:

[0024] Making a movement judgment according to the position of each module in the sequence of changes in the external contour of the modular self-reconfigurable robot cluster;

[0025] If the current position of the module is consistent with the target empty position, it is determined that the module does not need to be moved, wherein the module that does not need to be moved is a fixed module;

[0026] If the current position of the module is inconsistent with the target empty position, it is determined that the module needs to be moved, wherein the module that needs to be moved is a free module;

[0027] The priority of the free module is set and the trajectory of the free module is planned, and the free module moves based on the priority and the trajectory.

[0028] As a further improved technical solution, the setting of the priority of the free module and planning of the trajectory of the free module, wherein the free module moves based on the priority and the trajectory, comprises:

[0029] Update the target slot ID corresponding to the free module as the priority of the free module;

[0030] The running direction and revolution and translation change of the free module are calculated, and the trajectory of the free module is planned based on the revolution and translation change and the running direction. The free module moves based on the priority and the trajectory.

[0031] A second aspect of the embodiment of the present application provides a flow and manipulation device based on a spherical modular self-reconfigurable robot, comprising:

[0032] A local perception synthesis module is used to obtain the environmental position information of each module, synthesize all the environmental position information obtained, and obtain an overall environmental model or the shape and position of the manipulated object;

[0033] A contour sequence generation module is used to input the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contours of the modular self-reconfigurable robot cluster;

[0034] The module motion control module is used to plan the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, obtain the motion trajectory information of each module, and each module moves based on the motion trajectory information.

[0035] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flow and manipulation method based on a spherical modular self-reconfigurable robot as described above.

[0036] A fourth aspect of the embodiments of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0037] The communication bus realizes the connection and communication between the processor and the memory;

[0038] When the processor executes the computer-readable program, the processor implements the steps in any of the above-mentioned flow and manipulation methods based on a spherical modular self-reconfigurable robot.

[0039] Beneficial effect: Compared with the prior art, the flow and manipulation method based on the spherical modular self-reconfigurable robot of the present invention includes: obtaining the environmental position information of each module, synthesizing all the environmental position information obtained, and obtaining the overall environmental model or the shape and position of the manipulated object; inputting the overall environmental model or the shape and position of the manipulated object into the trained neural network for calculation, and obtaining the external contour change sequence of the modular self-reconfigurable robot cluster; planning the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, and obtaining the motion trajectory information of each module, and each module moves based on the motion trajectory information; after adopting the above method, the present invention can be applied to the spherical modular self-reconfigurable robot, and the flow and manipulation ability of water and multicellular tissues are simulated by self-reconstructing each module. Compared with the legged robot and the mechanical arm with fixed connection relationship, the present method has better adaptability, versatility, expansibility and flexibility, and the present method can flexibly adapt to the shape of the grasped object and can manipulate heavy loads in the opposite direction of gravity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the flow and manipulation method based on the spherical modular self-reconfigurable robot of the present invention.

[0041] Figure 2 It is a structural principle diagram of the terminal device provided by the present invention.

[0042] Figure 3 It is a structural block diagram of the device provided by the present invention.

[0043] Figure 4 The figure is a schematic structural diagram of the FreeBOT spherical robot provided by the present invention.

[0044] Figure 5 It is a FreeBOT simulation and two directional vector schematic diagrams provided by the present invention.

[0045] Figure 6 It is a schematic diagram of the module 3D position and local mesh model provided by the present invention.

[0046] Figure 7 It is a schematic diagram of the teacher network structure and training method provided by the present invention.

[0047] Figure 8 It is a schematic diagram of the student network structure and training method provided by the present invention.

[0048] Fig. 9 It is a schematic diagram of a method for fitting obstacles or fixed modules when calculating the 3D position of a passing point provided by the present invention.

[0049] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present application are given in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thoroughly and comprehensively understood.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0052] The inventor has found through research that the prior art has the following problems:

[0053] (1) Both flow and manipulation are realized based on self-reconfiguration algorithms. The existing research directions of self-reconfiguration algorithms can be divided into search-based methods, agent-based methods, and control-based methods. Search-based methods estimate the path distance between the initial, current, and final configurations through some metrics, such as the minimum number of steps to search for the shortest path. Therefore, search-based methods have the disadvantage of consuming a lot of computing time to traverse the exponentially growing configuration space. In agent-based methods, each module in the MSRR is regarded as an agent that can observe the local environment and act independently. For example, the Million Module March algorithm inspired by reinforcement learning can control a large-scale configuration composed of cube modules to flow distributedly over obstacles composed of cubes. Therefore, agent-based methods have a certain degree of randomness and are difficult to debug on hardware.

[0054] (2) Manipulation methods based on self-reconfiguration have rarely been studied. Existing studies mainly use the principle of mechanical advantage or draw inspiration from the anatomical structure of animals in the design of modular robots. For example, Polypod uses the near-singularity condition in the Jacobian matrix. Its actuator arm is nearly perpendicular to the load force when fully extended, and can move heavy loads within a small distance. For example, the "Deformatron" module can play one of the three roles of muscle, tendon and bone in the configuration of a modular robot: the driving power of the muscle module is proportional to the number of parallel module chains in the muscle, the tendon module uses a stretchable non-actuated connector to convert the translational motion of the muscle into rotational motion, and the skeletal module is responsible for transmitting the movement of the muscle. Similarly, "ATRON-anatomy" simulates muscles, bones and joints to generate modular robots that can expand their functional diversity with the number of modules. The self-reconfiguration of the Morpho module borrows the "inversion" behavior of a chlorophyll genus to produce self-deformation within the configuration. All of the above studies have failed to achieve distributed smooth manipulation.

[0055] In order to solve the above problems, various non-limiting implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0056] like Figure 1 As shown, the flow and manipulation method based on a spherical modular self-reconfigurable robot provided in an embodiment of the present application includes the following steps:

[0057] S1, obtaining the environmental position information of each module, synthesizing all the environmental position information obtained, and obtaining an overall environmental model or the shape and position of the manipulated object;

[0058] Specifically, the spherical modular self-reconfigurable robot is usually composed of several spherical modules. The connection of the spherical modules can better adapt to the uneven surface of the object. The spherical module can adopt the FreeBOT spherical robot. FreeBOT consists of a rough metal spherical shell and an internal car containing an electromagnet and a differential wheel. The internal car has two driving forces, namely yaw and roll. With the help of the yaw force, the internal car can choose the forward direction of the differential wheel (vector W s ), with the help of rolling force, the spherical module can perform two basic actions, namely rotation and revolution. Among them, rotation refers to the internal car rotating around the center of the module itself along the inner shell, and revolution refers to the entire module revolving in a convex shape around the center of another adsorbed module. Rotation only changes the position of the internal car relative to the spherical shell, while the 3D position of the spherical shell remains unchanged. Revolution directly changes the 3D position of the spherical shell, and the position of the internal car relative to the spherical shell will also change. Each spherical module can be equipped with an RGB camera and a UWB positioning chip to obtain environmental position information;

[0059] First, it is necessary to obtain the environmental position information of each module in the spherical modular self-reconfigurable robot, synthesize all the environmental position information obtained, and obtain the overall environmental model or the shape and position of the manipulated object.

[0060] The step of obtaining the environmental position information of each module and synthesizing all the environmental position information obtained to obtain the overall environmental model or the shape and position of the manipulated object includes the following steps:

[0061] S101, according to the preset instruction, control each module to collect local environment information and module's own position information, process each of the local environment information and the module's own position information, obtain a local mesh model of each module and the module's own 3D position information, and use the local mesh model of each module and the module's own 3D position information as each of the environment position information;

[0062] S102, splicing all the obtained local mesh models based on the own 3D position information to obtain an overall mesh model;

[0063] S103, detecting the entire mesh model to obtain the shape and 3D position of the manipulated object.

[0064] The processing of each of the local environment information and the module's own position information to obtain the local mesh model of each module and the module's own 3D position information includes the following steps:

[0065] Processing each of the local environment information through a depth estimation network to obtain a local mesh model of each of the modules;

[0066] The position information of each module is converted into global world coordinates relative to the position of the same main module, and the global world coordinates are used as the 3D position information of the module itself.

[0067] Specifically, each module in the spherical modular self-reconfigurable robot is controlled according to preset instructions to collect information about the local environment and the module's own position. The RGB camera module can collect information about the surrounding local environment, such as surrounding pictures or video information, and the UWB positioning chip can obtain the module's own position information, which is the local relative position P. u -P v, the surrounding local environment information is input into the depth estimation network for processing, and the local mesh model is output. The local relative position is converted into the global world coordinate P relative to the position of the same main module. Based on the global world coordinate, the local mesh model of each module is spliced ​​in the main module, that is, the vertex coordinates in each local mesh model are adjusted to the world coordinate system, the overlapping parts are deleted and the noise is removed to obtain the overall mesh model. Then, the 3D position and shape of the manipulated object are detected from the overall mesh model. The detection method adopts Voxel R-CNN. Finally, the overall mesh model, the 3D positions of all modules and the 3D position of the manipulated object are output [Mesh,P,P obj ], if the spherical modular self-reconfigurable robot is only used to climb over obstacles, the position and shape of the manipulated object may not be output.

[0068] S2, inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster;

[0069] Specifically, the overall environment model or the shape and position of the manipulated object is input into a neural network trained by imitation learning and lifelong reinforcement learning for calculation. The trained neural network outputs a sequence of changes in the external contour of the modular self-reconfigurable robot cluster. The sequence of changes in the external contour of the modular self-reconfigurable robot cluster does not specify how the modules inside the modular self-reconfigurable robot cluster move, so that the trained neural network can greatly reduce the number of output channels, reduce the amount of calculation of the neural network, and improve the calculation speed of the neural network, making the convergence of the neural network possible. Among them, imitation learning uses fluids that can collect complete environmental and self-information to generate teaching data, and preliminary training obtains a usable preliminary neural network. Reinforcement learning further explores diverse tasks and uses reward information to expand the functions of the neural network.

[0070] The step of inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster includes the following steps:

[0071] S201, training the initial neural network using imitation learning and reinforcement learning methods to obtain the trained neural network;

[0072] S202, inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster.

[0073] The method of using imitation learning and reinforcement learning to train the initial neural network to obtain the trained neural network includes the following steps:

[0074] S2011, obtaining a fluid surface profile change sequence when the magnetron fluid runs in the environment model, and training an initial teacher network based on the fluid surface profile change sequence and privileged information to obtain a target teacher network, wherein the privileged information includes the environment model and magnetron fluid properties;

[0075] S2012, obtaining actual observation information, and training a student network based on the actual observation information and a target teacher network to obtain a preliminary neural network, wherein the teacher network performs knowledge distillation on the student network based on a behavior loss function;

[0076] S2013, performing reinforcement learning training on the preliminary neural network to obtain the trained neural network.

[0077] Specifically, Blender is used to draw a mesh model and 3D print the mesh model. The mesh model can be a small environment such as a staircase, so that the magnetron fluid can flow and manipulate in the small environment. A depth camera is used to capture the contour change sequence of the magnetron fluid surface for training the teacher network. The training of the teacher network also uses privileged information, which is an accurate environment model and magnetron fluid properties. However, the control of the magnetron fluid cannot achieve precise manipulation actions. Therefore, the teacher strategy also needs to be explored and learned in a simulation environment to generate a student strategy. The teacher strategy is in the form of the real state of the environment. Full access to privileged information, this privileged learning approach enables the teacher policy to discover the best behavior under full observation, and then train a student policy that only has access to actual observations of the physical robot in the field The actual observation information on site is incomplete and noisy. The student strategy predicts the teacher's optimal behavior based on partial and noisy observation information and is trained through imitation learning. The student network can perform knowledge distillation under the guidance of the teacher network, that is, the teacher network performs knowledge distillation on the student network based on the behavior loss function to achieve an effect similar to that of the teacher network. The behavior loss function is pre-set by the user. After training, the student network obtains a preliminary neural network.

[0078] Then, the preliminary neural network is trained by reinforcement learning to obtain the trained neural network. The action space and observation space of reinforcement learning are defined as follows: When all modules are connected together, a tree-like configuration is formed. Each branch l = 1, 2, ..., n in the tree maintains a phase variable φ lA nominal trajectory is defined according to the phase. The nominal trajectory is the stepping motion of the branch end (i.e., the leaf node). Inverse kinematics is used to calculate the nominal joint target q of each joint actuator i=1,…,n i (φ l ), so the action of reinforcement learning is defined as the phase difference Δφ l and the remaining joint position target Δq i The observation information is defined as in Refers to ontological observation, Refers to external observation, Refers to a privileged status. Contains the configuration's movement speed, direction, joint position and velocity history, and action history, is the environment model near each leaf node, These include contact force, contact normal, friction coefficient, applied external forces and torques, and swing phase duration.

[0079] S3, planning the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, obtaining the motion trajectory information of each module, and each module moves based on the motion trajectory information.

[0080] Specifically, each module receives the external contour change sequence of the modular self-reconfigurable robot cluster sent by the main module, and plans the motion trajectory within a period of time in a distributed manner according to the internal position of the module itself. Each module moves based on the motion trajectory information. At the end of the external contour change sequence of the modular self-reconfigurable robot cluster within a period of time, each module can re-perceive the new environment and calculate the new external contour change sequence and module trajectory.

[0081] Wherein, planning the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster to obtain the motion trajectory information of each module, and each module moving based on the motion trajectory information includes the following steps:

[0082] S301, performing movement judgment according to the position of each module in the external contour change sequence of the modular self-reconfigurable robot cluster;

[0083] S302, if the current position of the module is consistent with the target empty position, it is determined that the module does not need to be moved, wherein the module that does not need to be moved is a fixed module;

[0084] S303, if the current position of the module is inconsistent with the target empty position, it is determined that the module needs to be moved, wherein the module to be moved is a free module;

[0085] S304, setting the priority of the free module and planning the trajectory of the free module, and the free module moves based on the priority and the trajectory.

[0086] The step of setting the priority of the free module and planning the trajectory of the free module, wherein the free module moves based on the priority and the trajectory, comprises the following steps:

[0087] S3041, updating the target slot ID corresponding to the free module as the priority of the free module;

[0088] S3042, calculating the running direction and revolution and translation change of the free module, planning the trajectory of the free module based on the revolution and translation change and the running direction, and the free module moves based on the priority and the trajectory.

[0089] Specifically, each module determines whether to be fixed or moving according to its position in the external contour change sequence of the modular self-reconfigurable robot cluster, and obtains the target position of each module by comparing the current configuration with the next frame contour in the external contour change sequence of the modular self-reconfigurable robot cluster. Each configuration contains two types of modules, namely fixed modules and free modules. Free modules refer to modules that have not yet reached the target position, and modules that have reached the target position are fixed modules. A free module may have two types of collisions during movement. The first type is a collision with other free modules, and the second type is a collision with fixed modules or obstacles. The first type of conflict is avoided by using the update ID that matches the target position as the priority. Free modules with lower priorities will stop at appropriate steps when a collision is foreseen until free modules with higher priorities leave the collision range.

[0090] The second type of collision is avoided by setting the trajectory's pass-through points. Fig. 9 shows how to calculate the pass-through points that come into contact with obstacles or fixed modules without colliding. Fig. 9 In the top view shown, M 1 is a free module, M 2 It is M 1 Connection module, M 3 It is M 1 The next module to be connected according to the connection planning result, M 4 is a fixed module that has reached its target slot, first, in and We uniformly select 10 points on the 3D line segment between and record them as P i via ,i=1,…,10, and P i via intersect and The plane with the normal vector is called the through plane. The through plane is used to cut the outline of obstacles and fixed modules, and then the mesh model and fixed modules (such as M 4 ) is offset outward by a radius R. Finally, the offset contour from the mesh model or fixed module is i via Select a passing point P from the intersection of the center and the circle with radius 2R i pass is the radius, and the trajectory between two consecutive passing points consists of three basic actions, M u The running direction is adjusted according to the following equation:

[0091] ΔR=from_rotvec(Axis,ΔΘ)

[0092] R s+1 =ΔR×R s

[0093] Among them, the from_rotvec function calculates the incremental rotation matrix according to the rotation axis Axis and the rotation angle ΔΘ. For the yaw action, Axis is M u M s , the MSRR cluster contains n modules M u ,u∈[1,n], ΔΘ is a 3D vector The angle between u W s and M s The plane where the vector M is located s is the direction of the N pole of the electromagnet inside the spherical shell and the vector W s is the forward direction of the differential wheel, and the role of yaw is to adjust M u W s Pointing to the next passing point, for the rotation action, Axis and ΔΘ change according to the collision angle and position of the next module in the simplified path output by the connection planning. The role of rotation is to adjust M u M s Points to the next module to be connected. For the revolution action, Axis is M u The two direction vectors M s and W s The cross product of , ΔΘ = 1° is user defined. In addition to the change in direction, the translation change of the revolution is calculated according to the following equation:

[0094]

[0095] in Indicates the 3D position of the current connection module.

[0096] When a module moves along the trajectory controlled by the points calculated above, it can just touch the mesh model or other fixed modules without collision. This feature of the trajectory enables the module to maintain contact with obstacles or fixed modules. The more contact points there are, the larger the support polygon of the configuration.

[0097] The external contour change sequence of the modular self-reconfigurable robot cluster only contains a few frame contours t=1, 2, …, T. Therefore, when the time or configuration conversion step reaches T, the modular self-reconfigurable robot cluster re-collects the environmental position information and regenerates a new contour sequence. This cycle continues. The trained neural network used in the contour sequence generation stage has a long calculation time. Therefore, generating contour predictions within the future T time at one time can avoid frequent calls to large-scale neural networks.

[0098] Based on the above-mentioned flow and manipulation method based on a spherical modular self-reconfigurable robot, this embodiment provides a flow and manipulation device based on a spherical modular self-reconfigurable robot, including:

[0099] The local perception synthesis module 1 is used to obtain the environmental position information of each module, synthesize all the environmental position information obtained, and obtain the overall environmental model or the shape and position of the manipulated object;

[0100] A contour sequence generation module 2 is used to input the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contours of the modular self-reconfigurable robot cluster;

[0101] The module motion control module 3 is used to plan the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, obtain the motion trajectory information of each module, and each module moves based on the motion trajectory information.

[0102] In addition, it is worth explaining that the working process of the flow and manipulation device based on the spherical modular self-reconfigurable robot provided in this embodiment is the same as the working process of the flow and manipulation method based on the spherical modular self-reconfigurable robot mentioned above. For details, please refer to the working process of the flow and manipulation method based on the spherical modular self-reconfigurable robot, which will not be repeated here.

[0103] Based on the above-mentioned flow and manipulation method based on a spherical modular self-reconfigurable robot, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the flow and manipulation method based on a spherical modular self-reconfigurable robot as described in the above-mentioned embodiment.

[0104] like Figure 2 As shown, based on the above-mentioned flow and manipulation method based on the spherical modular self-reconfigurable robot, the present application also provides a terminal device, which includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communication interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display the user guide interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0105] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0106] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0107] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0108] Compared with the prior art, the flow and manipulation method based on a spherical modular self-reconfigurable robot of the present invention includes: obtaining the environmental position information of each module, synthesizing all the environmental position information obtained to obtain an overall environmental model or the shape and position of the manipulated object; inputting the overall environmental model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster; planning the motion trajectory of each module based on the sequence of changes in the external contour of the modular self-reconfigurable robot cluster to obtain the motion trajectory information of each module, and each module moves based on the motion trajectory information; after adopting the above method, the present invention can be applied to a spherical modular self-reconfigurable robot, and by allowing each module to self-reconstruct, the flow and manipulation capabilities of water and multicellular tissues are simulated. Compared with a legged robot and a robotic arm with a fixed connection relationship, the present method has better adaptability, versatility, expansibility and flexibility, and the method can flexibly adapt to the shape of the grasped object and can manipulate heavy loads in the opposite direction of gravity.

[0109] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

[0110] Of course, the description of the above-mentioned embodiments of the present invention is relatively detailed, but it cannot be understood as limiting the scope of protection of the present invention. The present invention may also have many other implementation methods. Based on this implementation method, other implementation methods obtained by ordinary technicians in this field without any creative work all belong to the scope of protection of the present invention. The scope of protection of the present invention shall be based on the attached claims.

Claims

1. A flow and manipulation method based on a spherical modular self-reconfigurable robot, characterized in that: include: Acquire the environmental position information of each module, synthesize all the acquired environmental position information, and obtain the overall environmental model or the shape and position of the manipulated object; Inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster; Planning the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, obtaining the motion trajectory information of each module, and each module moves based on the motion trajectory information; The step of obtaining the environmental position information of each module and synthesizing all the environmental position information obtained to obtain the overall environmental model or the shape and position of the manipulated object includes: Control each module to collect local environment information and module position information according to preset instructions, process each local environment information and module position information, obtain a local mesh model of each module and module 3D position information, and use the local mesh model of each module and module 3D position information as each environment position information; Based on the 3D position information of the self, all the local mesh models are spliced ​​to obtain an overall mesh model; The overall mesh model is detected to obtain the shape and 3D position of the manipulated object.

2. The flow and manipulation method based on a spherical modular self-reconfigurable robot according to claim 1, characterized in that: The processing of each of the local environment information and the module's own position information to obtain the local mesh model of each module and the module's own 3D position information includes: Processing each of the local environment information through a depth estimation network to obtain a local mesh model of each of the modules; The position information of each module is converted into global world coordinates relative to the position of the same main module, and the global world coordinates are used as the 3D position information of the module itself.

3. The flow and manipulation method based on a spherical modular self-reconfigurable robot according to claim 2, characterized in that: The step of inputting the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster includes: Using imitation learning and reinforcement learning methods to train the initial neural network to obtain the trained neural network; The overall environment model or the shape and position of the manipulated object are input into the trained neural network for calculation to obtain a sequence of changes in the external contour of the modular self-reconfigurable robot cluster.

4. The flow and manipulation method based on a spherical modular self-reconfigurable robot according to claim 3, characterized in that: The imitation learning and reinforcement learning methods are used to train the initial neural network, and the trained neural network is obtained, which includes: Acquire a fluid surface profile change sequence when the magnetron fluid runs in an environment model, and train an initial teacher network based on the fluid surface profile change sequence and privileged information to obtain a target teacher network, wherein the privileged information includes the environment model and magnetron fluid properties; Acquire actual observation information, and train a student network based on the actual observation information and a target teacher network to obtain a preliminary neural network, wherein the teacher network performs knowledge distillation on the student network based on a behavior loss function; Perform reinforcement learning training on the preliminary neural network to obtain the trained neural network.

5. The flow and manipulation method based on a spherical modular self-reconfigurable robot according to claim 4, characterized in that: The step of planning the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster to obtain motion trajectory information of each module, and each module moving based on the motion trajectory information comprises: Making a movement judgment according to the position of each module in the sequence of changes in the external contour of the modular self-reconfigurable robot cluster; If the current position of the module is consistent with the target empty position, it is determined that the module does not need to be moved, wherein the module that does not need to be moved is a fixed module; If the current position of the module is inconsistent with the target empty position, it is determined that the module needs to be moved, wherein the module that needs to be moved is a free module; The priority of the free module is set and the trajectory of the free module is planned, and the free module moves based on the priority and the trajectory.

6. The flow and manipulation method based on a spherical modular self-reconfigurable robot according to claim 5, characterized in that: The step of setting the priority of the free module and planning the trajectory of the free module, wherein the free module moves based on the priority and the trajectory, comprises: Update the target slot ID corresponding to the free module as the priority of the free module; The running direction and revolution and translation change of the free module are calculated, and the trajectory of the free module is planned based on the revolution and translation change and the running direction. The free module moves based on the priority and the trajectory.

7. A flow and manipulation device based on a spherical modular self-reconfigurable robot, used to implement the flow and manipulation method based on a spherical modular self-reconfigurable robot as described in any one of claims 1 to 6, characterized in that: include: A local perception synthesis module is used to obtain the environmental position information of each module, synthesize all the environmental position information obtained, and obtain an overall environmental model or the shape and position of the manipulated object; A contour sequence generation module is used to input the overall environment model or the shape and position of the manipulated object into the trained neural network for calculation to obtain a sequence of changes in the external contours of the modular self-reconfigurable robot cluster; The module motion control module is used to plan the motion trajectory of each module based on the external contour change sequence of the modular self-reconfigurable robot cluster, obtain the motion trajectory information of each module, and each module moves based on the motion trajectory information.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flow and manipulation method based on a spherical modular self-reconfigurable robot as described in any one of claims 1-6.

9. A terminal device, characterized in that: include: A processor, a memory and a communication bus; the memory stores a computer-readable program that can be executed by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the flow and manipulation method based on the spherical modular self-reconfigurable robot as described in any one of claims 1-6 are implemented.

Images