Drosophila larva-imitating full-software robot multi-task control method

Abstract

The invention discloses a multi-task control method for a full-software robot simulating fruit fly larvae. The design of the method is based on a fruit fly larva sensing-motion-control neural circuit and a motion adaptation neural circuit. The method is applied to a soft robot simulating fruit fly larvae, so that the soft robot can have multiple motion modes such as continuous wriggling, steering and obstacle avoidance, and meanwhile control tasks such as target arrival and navigation are efficiently completed. In addition, according to the control method, the motion rhythm of the soft robot can be adaptively optimized through environment feedback information, and the decision uncertainty of the current state is measured according to historical tracks. In an area needing to be accurately sensed, the soft robot can automatically slow down the motion rhythm to improve the positioning and detection capacity, the method is suitable for the field of navigation exploration and microoperation in a complex environment, the problems that a traditional control algorithm for driving the soft robot is low in efficiency, poor in stability, weak in generalization capacity and the like are solved, and the application prospect is wide. And an important reference is provided for the design of a control method of a full-software robot in reality.

Description

Technical Field

[0001] The invention belongs to the technical field of robot control, and in particular relates to a multi-task control method of a fully soft robot imitating a fruit fly larva. Background Art

[0002] Existing bionic soft robots include leg-type robots, inchworm-type robots, snake-like robots, caterpillar robots, etc.; the motion modes of the above-mentioned bionic robots are relatively single, and it is usually difficult to simultaneously achieve multi-mode motions such as forward, turning, twisting, or their combined motions, and they are easily restricted by narrow spaces.

[0003] As a typical model organism, the fruit fly larvae have a compact and light body structure, and have significant advantages such as strong environmental perception, flexible soft movement, and strong movement adaptability. Although its neural structure is relatively simple, it has a variety of movement modes such as forward and backward creeping, head shaking, turning, flipping, raising head, and tail flicking. The movement is smooth and varied, and the movement mode switching is smooth. The soft robot imitating the fruit fly larvae is designed with multiple groups of lateral, longitudinal and oblique control muscles, giving the robot multi-mode movement ability, thereby realizing the multi-task control of the soft robot.

[0004] The perception-movement-control neural circuit mechanism of Drosophila larvae has been fully revealed. According to relevant research, the creeping movement of Drosophila larvae is controlled by a layered neural circuit, which mainly relies on the AcNs neural population in the ventral nerve cord. This type of neuron has some characteristics of a central pattern generator (CPG), which can generate autonomous oscillation signals and regulate the rhythm of movement. AcNs neurons control the direction and speed of movement by connecting A01j and A02j neurons. A01j neurons show a gradient distribution from back to front, which determines the forward direction of movement, while the rapid propagation activity in A02j neurons leads to rapid creeping propulsion. During the movement adaptation process of Drosophila larvae, the response time difference of the mechanosensory neurons vch in adjacent body segments of the larvae will occur. This change will be detected by Basin-1 neurons, resulting in the enhancement of its synapses with Wave neurons, strengthening the creeping CPG and thus improving movement adaptability. By imitating the perception-motor-control neural circuits and motor adaptation neural circuits of fruit fly larvae, we can design a highly efficient control method to guide soft robots to complete control tasks such as navigation, obstacle avoidance, and trajectory tracking, effectively alleviating the technical difficulties of weak driving capabilities and poor generalization capabilities of soft robot systems, thereby improving the practicality of soft robot systems in various scenarios (industrial and civilian). Summary of the invention

[0005] The purpose of the present invention is to provide a multi-task control method for a fully soft robot that imitates fruit fly larvae, which senses the dynamic change information of the environment to learn and then autonomously adjusts the control strategy to complete control tasks in multiple environments.

[0006] The method of the present invention is achieved through the following technical solutions: (1) Based on the perception-motor-control neural circuit of the larvae, we mimic the connection relationship between high-level neurons AcNs and low-level neurons A01j and A02j to design a hierarchical drive control system for the soft robot; (2) Based on the larvae’s motion adaptation neural circuit, we designed a soft robot multi-task learning adaptation system by imitating the connection between the sensory neuron Basin-1 and the Wave neuron, as well as the feedback effect on the peristaltic CPG.

[0007] The step (1) is based on the perception-movement-control neural circuit of the larvae, including a mechanical perception module, a bottom-level driving module and a central decision-making module. The central decision-making module generates a control signal based on deep reinforcement learning according to the current proprioceptive state and control target, and sends it to the mechanical perception module and the bottom-level driving module. The mechanical perception module based on the world model receives the control signal, processes the surrounding environment information, predicts the change of the future state, and sends the change to the bottom-level driving module. The bottom-level driving module receives the control signal and the predicted environmental change, activates the actuator of the soft robot, and prompts it to move. According to the current state s t , run the hierarchical drive control system, and obtain the functional relationship of the next moment state: s t+1 =F(s t ,A(s t ,S(s t ,C(s t ,g t )),C(s t ,g t ))), where F(·,·) is the forward dynamics simulation, A(·,·,·) is the underlying driving module function, S(·,·) is the mechanical perception module function, and C(·,·) is the central decision-making module function.

[0008] The step (2) is based on the neural circuit structure of larval movement adaptation, including a meta-reinforcement learning module based on short-term memory and an active learning module for uncertainty measurement. The meta-reinforcement learning module based on short-term memory uses a model-independent gradient meta-learning algorithm to encode task structure information into network parameters. The active learning module for uncertainty measurement uses an active learning algorithm based on maximum entropy to quantify the uncertainty of the soft robot in the decision-making process, select the decision state with the largest amount of information, build an experience sharing cache, and achieve precise control.

[0009] In the present invention, driving a soft robot to a specified position is defined as a microtask. These microtasks constitute macrotasks based on different structural features. Macrotasks include target arrival, anti-interference arrival and navigation.

[0010] The present invention provides a multi-task control method for a fully soft robot imitating a fruit fly larva, which specifically comprises the following steps:

[0011] 1. Design a hierarchical drive control system for a soft robot consisting of a mechanical perception module, a bottom-level drive module, and a central decision-making module, including the following steps: Step 1.1: At time step t, the central decision module determines the proprioceptive state s of the current soft robot. t And the control target g t , output control signal π H (u t |s t ,g t ) = argmax u Q(s t ,u,g t ;ω). Among them, Q(s t ,a t )=E π [G t |S t =s t ,A t =a t ; ω] is the action value function, is the reward after time step t, and γ is the discount factor. The present invention uses a neural network for parameterization, and ω represents the parameters of the neural network; Step 1.2: Imitating the gated neurons in the larval nervous system, the control signal output by the neural network needs to be sent to the gated neural network for modulation. The gated neural network h(·) measures the similarity between the samples stored in the experience sharing cache and the control signal output by the reinforcement learning network in the last H time steps. When the output of the gated neural network is greater than the threshold When The model predictive control (MPC) submodule will be enabled to achieve precise control with small data samples; Step 1.3: The mechanosensory module receives proprioceptive sensations t And the control signal u t , based on the world model, generates the state prediction value s at the next moment t ′ +1 , the relationship is s t ′ +1 =g(s t ,u t)+ε. Among them, ε is Gaussian noise, that is, ε~N(0,I), and g(·) is the function after neural network parameterization; Step 1.4: The underlying drive module receives proprioception s t , control signal u t And the state prediction value s t ′ +1 , bottom-level driver module π L (a t |s t ,u t ,s t ′ +1 ) is the specific execution module of the control signal, where a t represents the action performed at time step t; Step 1.5: Change action a t Input into the forward dynamics simulation to obtain the state s at the next moment t+1 =F(s t ,a t ). Repeat the above tasks until the control task is completed.

[0012] 2. Design a multi-task learning adaptation algorithm for soft robots consisting of a meta-reinforcement learning module based on short-term memory and an active learning module for uncertainty measurement, including the following steps: Step 2.1: In the virtual simulation environment, select tasks from the same distribution, that is, For each task Using the Bellman equation, we can calculate Among them, g t It's a task At each moment, the sub-goal, ω - are the parameters of the target network. By using the gradient descent method to optimize the objective function Ability to encode shared structural knowledge of the control task into the network parameters; Step 2.2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum amount of information: Where: π Q (u) = SoftMax[π H (u t |s t ,g t )]; Step 2.3: Set the decision status And related data are stored in the experience sharing cache.

[0013] Furthermore, the simulated fruit fly larvae movement scene is used as the algorithm interaction environment, and the fruit fly larvae movement data and environmental data are used to run the soft robot multi-task learning adaptation system to train the underlying driving module, mechanical perception module and central decision-making module.

[0014] Furthermore, the multi-task learning adaptation algorithm of the soft robot is run until each module converges, and then the hierarchical drive control system of the soft robot is run to control the soft robot to complete the macro task in the specified environment.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention combines the principles of bionics with soft robot control technology to realize a multi-task control method for a fully soft robot that mimics fruit fly larvae. The method is based on the neural circuit structure of fruit fly larvae in nature and provides a verification platform for neuroscience theory. The method is based on the perception-motion-control neural circuit of fruit fly larvae, and constructs a mechanical perception module, a bottom-level driving module, and a central decision-making module, so that the robot can achieve multiple modes of motion, and can achieve continuous peristalsis, steering, and obstacle avoidance functions without complex path planning or control algorithms. Based on the fruit fly larvae motion adaptation loop, a meta-reinforcement learning module based on short-term memory and an active learning module for uncertainty measurement are constructed, so that the robot can use environmental feedback to adaptively optimize its own motion rhythm. In areas that require precise perception, the soft robot can automatically slow down the rhythm to improve positioning and detection capabilities, thereby achieving efficient coverage of the target area and accurate completion of micro-environmental tasks, and is widely applicable to navigation, exploration, and micro-manipulation in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0017] Figure 1 A schematic diagram comparing the Drosophila larvae neural circuit principle and system modules provided by the present invention.

[0018] Figure 2 This is an information flow diagram of the soft robot control algorithm provided by the present invention.

[0019] Figure 3 This is the navigation trajectory diagram of the soft robot provided by the present invention.

[0020] Figure 4 This is a simulation result diagram provided by the present invention.

[0021] Figure 5 A comparison diagram of the soft robot provided by the present invention and the motion model of a real fruit fly larva. DETAILED DESCRIPTION

[0022] In order to better understand the technical solution of the present invention, the following further introduces the implementation of the present invention in combination with the drawings and specific examples. Note that the aspects described below in combination with the drawings and specific examples are only exemplary and should not be understood as limiting the scope of protection of the present invention.

[0023] Example 1

[0024] 1. Design a hierarchical drive control system for a soft robot consisting of a mechanical perception module, a bottom-level drive module, and a central decision-making module, including the following steps:

[0025] Step 1: At time step t, the central decision module determines the proprioceptive state s of the current soft robot. t And the control target g t , output control signal π H (u t |s t ,g t ) = argmax u Q(s t ,u,g t ;ω). Among them, Q(s t ,a t )=E π [G t |S t =s t ,A t =a t ; ω] is the action value function, is the reward after time step t, γ is the discount factor. The present invention uses a neural network for parameterization, and ω represents the parameters of the neural network.

[0026] Step 2: Imitating the gated neurons in the larval nervous system, the control signal output by the neural network needs to be sent to the gated neural network for modulation. The gated neural network h(·) measures the similarity between the samples stored in the experience sharing cache and the control signals output by the reinforcement learning network in the last H time steps. When the output of the gated neural network is greater than the threshold When The model predictive control (MPC) submodule will be enabled to achieve precise control with small data samples.

[0027] Step 3: The mechanosensory module receives proprioceptive sensations t And the control signal u t , based on the world model, generates the state prediction value s at the next moment t′ +1 , the relationship is s t ′ +1 =g(s t ,u t )+ε. Among them, ε is Gaussian noise, that is, ε~N(0,I), and g(·) is the function after parameterization of the neural network.

[0028] Step 4: The underlying drive module receives proprioception s t , control signal u t And the state prediction value s t ′ +1 , bottom-level driver module π L (a t |s t ,u t ,s t ′ +1 ) is the specific execution module of the control signal, where a t represents the action performed at time step t.

[0029] Step 5: Change action a t Input into the forward dynamics simulation to obtain the state s at the next moment t+1 =F(s t ,a t ). Repeat the above tasks until the control task is completed.

[0030] 2. Design a multi-task learning adaptation algorithm for soft robots consisting of a meta-reinforcement learning module based on short-term memory and an active learning module for uncertainty measurement, including the following steps:

[0031] Step 1: In the virtual simulation environment, select tasks from the same distribution, that is, For each task Using the Bellman equation, we can calculate Among them, g t It's a task At each moment, the sub-goal, ω - are the parameters of the target network. By using the gradient descent method to optimize the objective function Ability to encode shared structural knowledge of the control task into the network parameters.

[0032] Step 2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum amount of information: Where: π Q (u) = SoftMax[π H (ut |s t ,g t )], in this decision state Provide guidance to soft robots.

[0033] Step 3: Set the decision status And related data are stored in the experience sharing cache.

[0034] Collect and process the motion data of fruit fly larvae, simulate the motion scene of fruit fly larvae as the algorithm interaction environment, use the motion data and environmental data to run the multi-task learning adaptation algorithm of the soft robot, and train the underlying driving module, mechanical perception module and central decision-making module, including the following steps: Step 1: Build a virtual simulation environment and design the environmental dynamics equation; Step 2: Build a bionic fruit fly larva soft robot model and import it into the virtual simulation environment; Step 3: Train the underlying driving module. Collect and preprocess the crawling data of fruit fly larvae as the training data set for the underlying driving module. Define the optimization goal of the driving module as minimizing the difference between the state access distribution of the soft robot and the state access distribution of the real fruit fly larvae. Use relative entropy to quantify the difference between the two distributions, that is, where d π represents the state access distribution of the soft robot, and is the true state distribution. is a fixed state access distribution, which can obtain the final optimization goal: Step 4: Train the mechanical perception module. A neural network is trained using a semi-supervised learning method to approximate the transfer function g(·). That is, a proxy model is trained using existing real data. The proxy model is modeled as a Gaussian process GP, which generates pseudo labels for some unlabeled data. These model-generated data are combined with existing real data to form an augmented dataset, which will be used to train the world model. Step 5: Initialize the playback cache and experience sharing cache. The capacity is Use the replacement algorithm that has not been used for the longest time to update; Step 6: Use the DDQN algorithm to train the central decision module. Define sparse rewards to capture the structural invariance between tasks and environments, namely: Step 7: Randomly select a virtual simulation environment and place the soft robot in the environment; Step 8: Randomly select a control target, run a forward dynamics simulation, and collect sequence data, where u t = argmaxu Q(s t ,u,g t ; ω), and the quintuple (s t ,u t ,r t ,s t+1 ,g t ) into the playback buffer; Step 9: Using the Bellman equation, calculate Step 10: Define the objective function to be minimized Step 11: Optimize the objective function using a gradient-based meta-learning algorithm in Step 12: Run steps 8 to 11 in a single virtual environment until convergence; Step 13: Switch to the virtual simulation environment, randomly select the control target, and let the soft robot explore in the unfamiliar environment; Step 14: Based on the maximum entropy active learning algorithm, select the decision state with the maximum amount of information: Step 15: In State Let the soft robot perform exploratory actions, that is, sample multiple different instructions Use forward dynamics simulation to get the corresponding next time step state Step 16: Setup And use the gradient descent method to optimize the objective function Step 17: Change the status Store in experience sharing cache; Step 18: Run steps 13 to 17 until convergence.

[0035] Run the multi-task learning adaptation algorithm of the soft robot until each module converges, and then run the hierarchical drive control algorithm of the soft robot to control the soft robot to complete the macro task in the specified environment. It includes the following steps: Step 1: Specify a virtual environment, given a series of target points g1, g2, …, g N ; Step 2: Initialize counter cnt=1 and time step t=0; Step 3: According to the current entity state s t , and g cnt , obtain the current control signal u from the central decision module t ; Step 4: According to the control signal ut and the entity state s t , obtain the next moment state prediction value s from the mechanical perception module t ′ +1 ; Step 5: According to the control signal u t , entity state s t And the state prediction value s t ′ +1 , use the underlying driver module to obtain action a t ; Step 6: Change action a t Input the forward dynamics simulation to obtain the next moment's body state s t+1 , and check whether the current control target g has been completed cnt , if completed, the counter cnt is incremented by 1, otherwise no operation is performed; Step 7: If cnt>N, the task ends, otherwise jump back to step 3.

[0036] The present invention simulates three types of environments: rough plate (friction coefficient 0.9, small viscosity), wet plate (friction coefficient 0.3, large viscosity), smooth plate (friction coefficient 0.3, small viscosity), and selects two advanced reinforcement learning control algorithms: TD3 and DDPG for comparison. The following table shows the target reaching experiment success rate of the method without external disturbance and with external disturbance.

[0037] Table 1 Experimental success rate of the algorithm under undisturbed conditions

[0038] Table 2 Experimental success rate of the algorithm under disturbance

[0039] illustrate: Figure 1 The comparison diagram between the real Drosophila larvae neural circuit and the various modules designed by the present invention clearly shows the location and function of each module. Figure 2 for Figure 1 The further refinement of the model explains the information flow between modules. Figure 3 A schematic diagram of a soft robot completing a navigation task in a virtual environment; Figure 4 It is the loss function curve in the simulation iteration process; Figure 5 t-SNE visualization of preprocessed behavioral data for Drosophila larvae and output motions of the soft robot.

[0040] It can be seen from Table 1 and Table 2 that the control method proposed in the present invention can excellently complete the control task both in normal conditions and in the presence of external disturbances, and its performance far exceeds that of TD3 and PPO. Figure 1 It can be seen that the two systems designed by the present invention correspond one-to-one with the neural circuits responsible for movement control and learning adaptation in real fruit fly larvae, have good biological interpretability, low computational cost and high efficiency. Figure 2 It can be seen that in the hierarchical drive control system, the flow of information has a jumping residual structure, that is, the central decision module will send the control signal to the mechanical perception module and the underlying drive module respectively, which improves the accuracy of control. Figure 3 It can be seen that the soft robot imitating the fruit fly larvae can successfully complete the navigation task in the virtual environment, and basically moves along the shortest path between the two target points during the navigation process. Figure 4 It can be seen that when the soft robot learns different instructions (forward, turn left, turn right) in the virtual environment, the model loss decreases steadily. Figure 5 It can be seen that after training, the motion pattern output of the soft robot has a high degree of overlap with the label data, proving that after training, the soft robot can mine implicit manifold features from the label data and absorb them as its own implicit motion mode. It can be seen from the above result graph that, with the support of the control method proposed in the present invention, the soft robot imitating the fruit fly larvae can continuously learn and optimize the motion pattern in a virtual environment, thereby quickly and stably improving the motion control performance. When faced with structured control tasks with complex dynamics, the soft robot can efficiently explore the environment, autonomously generate adaptive motion patterns, and then complete the control task, showing excellent environmental adaptability and task execution efficiency.

[0041] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A multi-task control method for a fully soft robot imitating a fruit fly larva, characterized in that: It includes a hierarchical drive control system for soft robots and a multi-task learning adaptation system for soft robots. The hierarchical drive control system for soft robots is based on the perception-motion-control neural circuit of larvae, and is designed to imitate the connection relationship between high-level neurons AcNs and low-level neurons A01j and A02j. It is composed of a mechanical perception module, a low-level drive module, and a central decision-making module. The multi-task learning adaptation system for soft robots is based on the motion adaptation neural circuit of larvae, and is designed to imitate the connection relationship between perception neurons Basin-1 and Wave neurons, as well as the feedback effect on peristalsis CPG. It is composed of a meta-reinforcement learning module based on short-term memory and an active learning module for uncertainty measurement.

2. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: The central decision-making module generates a control signal based on deep reinforcement learning according to the proprioceptive state and the control target, and sends it to the mechanical perception module and the underlying driving module; the mechanical perception module based on the world model receives the control signal, processes the surrounding environment information, predicts the change of the future state, and sends the predicted value to the underlying driving module; the underlying driving module receives the control signal and the predicted environmental change, activates the actuator of the soft robot, and prompts it to move; according to the current state s t , run the hierarchical drive control system, and obtain the functional relationship of the next moment state: s t+1 =F(s t ,A(s t ,S(s t ,C(s t ,g t )),C(s t ,g t ))), where F(·,·) is the forward dynamics simulation, A(·,·,·) is the underlying driving module function, S(·,·) is the mechanical perception module function, and C(·,·) is the central decision-making module function.

3. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: The short-term memory-based meta-reinforcement learning module uses a model-independent gradient meta-learning algorithm to encode task structure information into network parameters. The active learning module for uncertainty measurement uses a maximum entropy active learning algorithm to quantify the uncertainty of the soft robot in the decision-making process, selects the decision state with the largest amount of information, builds an experience sharing cache, and achieves precise control.

4. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: The information transmission relationship between the central decision-making module, the mechanical perception module, and the underlying driving module is as follows: Step 4.1: At time step t, the central decision module determines the proprioceptive state s of the current soft robot. t And the control target g t , output control signal π H (u t |s t ,g t )=argmax u Q(s t ,u,g t ;ω), where Q(s t ,a t )=E π [G t |S t =s t ,A t =a t ; ω] is the action value function, is the return after time step t, γ is the discount factor, parameterized using a neural network, and ω represents the parameters of the neural network; Step 4.2: Imitating the gated neurons in the larval cognitive system, the control signal output by the neural network needs to be sent to the gated neural network for modulation. The gated neural network h(·) measures the similarity between the samples stored in the experience sharing cache and the control signals output by the reinforcement learning network in the last H time steps. When the output of the gated neural network is greater than the threshold N, that is, when h(s t-H,...,t-1 )>N, the model predictive control (MPC) submodule will be enabled to achieve precise control under small data samples; Step 4.3: The mechanosensory module receives proprioceptive sensations t And the control signal u t , based on the world model, generate the state prediction value s′ at the next moment t+1 , the relation is s′ t+1 =g(s t ,u t )+ε, where ε is Gaussian noise, i.e., ε~N(0,I), and g(·) is a function parameterized by a neural network; Step 4.4: The underlying drive module receives proprioception s t , control signal u t And the state prediction value s′ t+1 , bottom-level driver module π L (a t |s t ,u t ,s′ t+1 ) is the specific execution module of the control signal, where a t represents the action performed at time step t; Step 4.5: Change action a t Input into the forward dynamics simulation to obtain the state s at the next moment t+1 =F(s t ,a t ), and repeat the above tasks until the control task is completed.

5. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: The meta-reinforcement learning module based on short-term memory and the active learning module for uncertainty measurement include the following steps: Step 5.1: In the virtual simulation environment, select tasks from the same distribution, that is, For each task Using the Bellman equation, we can calculate Among them, g t It's a task At each moment, the sub-goal, ω - are the parameters of the target network, and the objective function is optimized by using the gradient descent method Ability to encode shared structural knowledge of the control task into the network parameters; Step 5.2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum amount of information: Where: π Q (u) = SoftMax[π H (u t |s t ,g t )], in this decision state Provide guidance to soft robots; Step 5.3: Set the decision status And related data are stored in the experience sharing cache.

6. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: Run the multi-task learning adaptation system of the soft robot until each module converges, then run the hierarchical drive control system of the soft robot to control the soft robot to complete the macro task in the specified environment, including the following steps: Step 6.1: Specify a virtual environment, given a series of target points g1, g2, …, g N ; Step 6.2: Initialize counter cnt=1 and time step t=0; Step 6.3: According to the current entity state s t , and g cnt , obtain the current control signal u from the central decision module t ; Step 6.4: According to the control signal u t And the entity state s t , obtain the next moment state prediction value s′ from the mechanical perception module t+1 ; Step 6.5: According to the control signal u t , entity state s t And the state prediction value s′ t+1 , use the underlying driver module to obtain action a t ; Step 6.6: Change action a t Input the forward dynamics simulation to obtain the next moment's body state s t+1 , and check whether the current control target g has been completed cnt , if completed, the counter cnt is incremented by 1, otherwise no operation is performed; Step 6.7: If cnt>N, the task ends, otherwise jump back to step 6.

3.

7. The multi-task control method of a fully soft robot imitating a fruit fly larva according to claim 1, characterized in that: The simulated fruit fly larvae motion scene is used as the algorithm interaction environment. The fruit fly larvae motion data and environmental data are used to run the soft robot multi-task learning adaptation system to train the underlying driving module, mechanical perception module and central decision-making module, including the following steps: Step 7.1: Train the underlying driving module. Collect and preprocess the crawling data of fruit fly larvae as the training data set of the underlying driving module. Define the optimization goal of the driving module as minimizing the difference between the state access distribution of the soft robot and the state access distribution of the real fruit fly larvae. Use relative entropy to quantify the difference between the two distributions, that is, where d π represents the state access distribution of the soft robot, and is the true state distribution, since is a fixed state access distribution, which can obtain the final optimization goal: Step 7.2: Train the mechanical perception module. Use a semi-supervised learning method to train a neural network to approximate the transfer function g(·). That is, train a proxy model through existing real data. The proxy model is modeled as a Gaussian process GP. The Gaussian process model will generate pseudo labels for some unlabeled data. These model-generated data are combined with the existing real data to form an augmented data set, which will be used to train the world model. Step 7.3: Initialize the replay cache and experience sharing cache with a capacity of Use the replacement algorithm that has not been used for the longest time to update; Step 7.4: Use the DDQN algorithm to train the central decision module and define sparse rewards to capture the structural invariance between tasks and environments, namely: Step 7.5: Randomly select a virtual simulation environment and place the soft robot in the environment; Step 7.6: Randomly select a control target, run a forward dynamics simulation, and collect sequence data, where u t = argmax u Q(s t ,u,g t ; ω), and the quintuple (s t ,u t ,r t ,s t+1 ,g t ) into the playback buffer; Step 7.7: Using the Bellman equation, calculate Step 7.8: Define the objective function to be minimized Step 7.9: Optimize the objective function using a gradient-based meta-learning algorithm in Step 7.10: Run steps 7.6 to 7.9 in a single virtual environment until convergence; Step 7.11: Switch to the virtual simulation environment, randomly select the control target, and let the soft robot explore in the unfamiliar environment; Step 7.12: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum amount of information: Where: π Q (u) = SoftMax[π H (u t |s t ,g t )]; Step 7.13: In the state Let the soft robot perform exploratory actions, that is, sample multiple different instructions Use forward dynamics simulation to get the corresponding next time step state Step 7.14: Setup And use the gradient descent method to optimize the objective function Step 7.15: Change the status Store in experience sharing cache; Step 7.16: Run steps 7.11 to 7.15 until convergence.

Images