A drosophila larva-imitating soft robot multi-task control method

Abstract

This invention discloses a multi-task control method for a fully soft robot inspired by a fruit fly larva. The method is designed based on the perception-motor-control neural circuit and the motion adaptation neural circuit of a fruit fly larva. Applied to a soft robot inspired by a fruit fly larva, this method enables the robot to possess multiple motion modes such as continuous peristalsis, turning, and obstacle avoidance, while efficiently completing control tasks such as target arrival and navigation. Furthermore, the control method can adaptively optimize the robot's motion rhythm using environmental feedback information and measure the decision uncertainty of the current state based on historical trajectories. In areas requiring precise perception, the soft robot can automatically slow down its motion rhythm to improve localization and detection capabilities, making it suitable for navigation exploration and micro-manipulation in complex environments. This method solves the problems of low efficiency, poor stability, and weak generalization ability of traditional control algorithms driving soft robots, providing an important reference for the design of control methods for fully soft robots in reality.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically relating to a multi-task control method for a fully soft robot that mimics a fruit fly larva. Background Technology

[0002] Existing biomimetic soft robots include legged robots, inchworm-like robots, snake-like robots, caterpillar robots, etc. The motion modes of the above-mentioned biomimetic robots are relatively simple, and they usually cannot simultaneously achieve multiple modes of motion such as forward movement, turning, twisting, or their combination, and are easily limited by narrow spaces.

[0003] As a typical model organism, the fruit fly larva has a compact and lightweight body structure, possessing significant advantages such as strong environmental perception, flexible soft movement, and strong adaptability. Although its neural structure is relatively simple, it exhibits a rich variety of movement patterns, including forward and backward crawling, head shaking, turning, side-flipping, head lifting, and tail flicking. Its movements are smooth and varied, with seamless switching between action patterns. The soft robot inspired by the fruit fly larva is designed with multiple sets of lateral, longitudinal, and oblique control muscles, endowing the robot with multi-modal movement capabilities, thereby enabling multi-task control of the soft robot.

[0004] The sensory-motor-control neural circuit mechanism of Drosophila larvae has been fully elucidated. According to relevant research, the peristaltic crawling of Drosophila larvae is controlled by a hierarchical neural circuit, mainly relying on the AcNs neuronal population in the ventral nerve cord. These neurons possess some characteristics of central pattern generators (CPGs), capable of generating autonomous oscillatory signals to regulate the rhythm of movement. AcNs neurons control the direction and speed of movement by connecting A01j and A02j neurons. A01j neurons exhibit a gradient distribution from back to front, determining the direction of movement, while the rapid propagation activity in A02j neurons leads to rapid peristaltic propulsion. During the movement adaptation process of Drosophila larvae, a time difference in the response of vch neurons in adjacent somatic segments occurs. This change is detected by Basin-1 neurons, leading to synaptic enhancement between Basin-1 neurons and Wave neurons, strengthening the peristaltic CPG, and thus improving movement adaptation. By mimicking the sensory-motor-control neural circuit and the motor adaptation neural circuit of fruit fly larvae, a highly efficient control method can be designed to guide soft robots to complete control tasks such as navigation, obstacle avoidance, and trajectory tracking. This effectively alleviates the technical difficulties of weak driving ability and poor generalization ability of soft robot systems, thereby improving the practicality of soft robot systems in various scenarios (industrial and civilian). Summary of the Invention

[0005] The purpose of this invention is to provide a multi-task control method for a fully soft robot that mimics a fruit fly larva. By sensing dynamic changes in the environment, the robot learns and then autonomously adjusts its control strategy to complete control tasks in multiple environments.

[0006] The method of this invention is achieved through the following technical solution:

[0007] (1) Based on the larval perception-motor-control neural circuit, the connection relationship between high-level neurons AcNs and low-level neurons A01j and A02j is imitated to design a hierarchical drive control system for soft robots.

[0008] (2) Based on the motor adaptation neural circuit of larvae, the connection between the sensory neuron Basin-1 and the Wave neuron, as well as the feedback effect on the peristaltic CPG, is used to design a multi-task learning and adaptation system for soft robots.

[0009] Step (1) is based on the larval perception-motor-control neural circuit, including a mechanosensory module, a low-level driving module, and a central decision-making module. The central decision-making module, based on deep reinforcement learning, generates control signals according to the current proprioceptive state and control objective, and sends these signals to the mechanosensory module and the low-level driving module. The mechanosensory module, based on a world model, receives the control signals, processes information about the surrounding environment, predicts future state changes, and sends these changes to the low-level driving module. The low-level driving module receives the control signals and the predicted environmental changes, activates the soft robot's actuators, and causes it to move. Based on the current state s... t Running the hierarchical drive control system, the functional relationship for obtaining the state at the next moment is:

[0010] s t+1 =F(s) t A(s) t ,S(s t ,C(s t ,g t )),C(s t ,g t ))), where F(·,·) is the forward dynamics simulation, A(·,·,·) is the bottom-level driving module function, S(·,·) is the mechanical perception module function, and C(·,·) is the central decision module function.

[0011] Step (2) is based on the neural circuit structure of larval locomotion adaptation, including a short-term memory-based meta-reinforcement learning module and an uncertainty-metric-oriented active learning module. The short-term memory-based meta-reinforcement learning module uses a model-independent gradient meta-learning algorithm to encode task structure information into network parameters. The uncertainty-metric-oriented active learning module uses a maximum entropy-based active learning algorithm to quantify the uncertainty of the soft robot in the decision-making process, select the decision state with the most information, construct an experience-sharing cache, and achieve precise control.

[0012] In this invention, driving a soft robot to a designated location is defined as a microtask. These microtasks, based on different structural features, constitute macrotasks. Macrotasks include target arrival, interference-resistant arrival, and navigation.

[0013] This invention discloses a multi-task control method for a fully soft robot mimicking a fruit fly larva, specifically comprising the following steps:

[0014] 1. Design a hierarchical drive control system for a soft robot, consisting of a mechanical perception module, a low-level drive module, and a central decision-making module, including the following steps:

[0015] Step 1.1: At time step t, the central decision-making module determines the current proprioceptive state s of the soft robot. t and control target g t Output control signal π H (u t |s t ,g t ) = argmax u Q(s t ,u,g t ;ω). Where, Q(s t ,a t ) = E π [G t |S t =s t A t =a t ;ω] is the action value function, ω represents the reward after time step t, and γ is the discount factor. This invention uses a neural network for parameterization, where ω represents the parameters of the neural network.

[0016] Step 1.2: Mimicking the gating neurons in the larval nervous system, the control signal output by the neural network needs to be modulated by the gating neural network. The gating neural network h(·) measures the similarity between the samples stored in the experience-sharing buffer and the control signals output by the reinforcement learning network in the most recent H time steps. When the output of the gating neural network is greater than a threshold... At that time, that is The Model Predictive Control (MPC) submodule will be enabled to achieve precise control with small data samples;

[0017] Step 1.3: The mechanical sensing module receives proprioceptive sensation s t and control signal u t Based on the world model, generate the predicted state value s for the next time step. t ′ +1 The relation is s t ′ +1=g(s t ,u t )+ε. Where ε is Gaussian noise, i.e., ε~N(0,I), and g(·) is the function parameterized by the neural network;

[0018] Step 1.4: The underlying driver module receives proprioceptive sensation s t Control signal u t and state prediction value s t ′ +1 The underlying driver module π L (a t |s t ,u t ,s t ′ +1 ) is the specific execution module for the control signals, where a t This indicates the action performed at time step t;

[0019] Step 1.5: Perform action a t Inputting the data into the forward dynamics simulation yields the state s at the next time step. t+1 =F(s) t ,a t Repeat the above task until the control task is completed.

[0020] 2. Design a multi-task learning and adaptation algorithm for soft robots, consisting of a short-term memory-based meta-reinforcement learning module and an active learning module oriented towards uncertainty metrics, including the following steps:

[0021] Step 2.1: In the virtual simulation environment, select tasks from the same distribution, i.e. For each task Using Bellman equations, calculate Among them, g t It is a task At each moment's sub-goal, ω - These are the parameters of the target network. The objective function is optimized using gradient descent. It can encode shared structural knowledge of control tasks into network parameters;

[0022] Step 2.2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum information content: Where: π Q (u)=SoftMax[π H (u t |s t ,g t )];

[0023] Step 2.3: Change the decision state And related data are stored in the experience sharing cache.

[0024] Furthermore, the simulated fruit fly larvae movement scenario serves as the algorithm's interactive environment. By utilizing the fruit fly larvae movement data and environmental data, a multi-task learning and adaptation system for soft robots is run to train the underlying driving module, mechanical perception module, and central decision-making module.

[0025] Furthermore, after running the soft robot multi-task learning and adaptation algorithm until all modules converge, the soft robot hierarchical drive control system is run to control the soft robot to complete macro tasks in the specified environment.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] This invention combines biomimetic principles with soft robot control technology to realize a multi-task control method for a fully soft robot inspired by fruit fly larvae. Based on the neural circuit structure of fruit fly larvae in nature, this method provides a verification platform for neuroscience theories. The method is based on the perception-motor-control neural circuit of fruit fly larvae, constructing a mechanical perception module, a low-level driving module, and a central decision-making module. This enables the robot to achieve multiple modalities of motion, realizing continuous peristalsis, turning, and obstacle avoidance functions without complex path planning or control algorithms. Based on the motion adaptation circuit of fruit fly larvae, a meta-reinforcement learning module based on short-term memory and an active learning module oriented towards uncertainty metrics are constructed, enabling the robot to adaptively optimize its own motion rhythm using environmental feedback. In areas requiring precise perception, the soft robot can automatically slow down its rhythm to improve localization and detection capabilities, thereby achieving efficient coverage of target areas and precise completion of micro-environment tasks. This method is widely applicable to navigation exploration and micro-manipulation in complex environments. Attached Figure Description

[0028] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0029] Figure 1 A comparative schematic diagram of the neural circuit principle and system modules of fruit fly larvae provided by the present invention.

[0030] Figure 2 Information flow diagram of the soft robot control algorithm provided by the present invention.

[0031] Figure 3The navigation trajectory diagram of the soft robot provided by the present invention.

[0032] Figure 4 The simulation results are shown in the figure provided by the present invention.

[0033] Figure 5 A comparison image of the motion model of the soft robot provided by this invention and a real fruit fly larva. Detailed Implementation

[0034] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below in conjunction with the accompanying drawings and specific examples. Note that the aspects described below in conjunction with the accompanying drawings and specific examples are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0035] Example 1

[0036] 1. Design a hierarchical drive control system for a soft robot, consisting of a mechanical perception module, a low-level drive module, and a central decision-making module, including the following steps:

[0037] Step 1: At time step t, the central decision-making module determines the current proprioceptive state s of the soft robot. t and control target g t Output control signal π H (u t |s t ,g t ) = argmax u Q(s t ,u,g t ;ω). Where, Q(s t ,a t ) = E π [G t |S t =s t A t =a t ;ω] is the action value function, ω represents the reward after time step t, and γ is the discount factor. This invention uses a neural network for parameterization, where ω represents the parameters of the neural network.

[0038] Step 2: Mimicking the gating neurons in the larval nervous system, the control signals output by the neural network need to be modulated by the gating neural network. The gating neural network h(·) measures the similarity between the samples stored in the experience-sharing buffer and the control signals output by the reinforcement learning network in the most recent H time steps. When the output of the gating neural network exceeds a threshold... At that time, that is The Model Predictive Control (MPC) submodule will be enabled to achieve precise control with small data samples.

[0039] Step 3: The mechanical sensing module receives proprioceptive sensation s t and control signal u t Based on the world model, generate the predicted state value s for the next time step. t ′ +1 The relation is s t ′ +1 =g(s t ,u t )+ε. Where ε is Gaussian noise, i.e., ε~N(0,I), and g(·) is a function parameterized by a neural network.

[0040] Step 4: The underlying driver module receives proprioceptive sensations. t Control signal u t and state prediction value s t ′ +1 The underlying driver module π L (a t |s t ,u t ,s t ′ +1 ) is the specific execution module for the control signals, where a t This indicates the action performed at time step t.

[0041] Step 5: Perform action a t Inputting the data into the forward dynamics simulation yields the state s at the next time step. t+1 =F(s) t ,a t Repeat the above task until the control task is completed.

[0042] 2. Design a multi-task learning and adaptation algorithm for soft robots, consisting of a short-term memory-based meta-reinforcement learning module and an active learning module oriented towards uncertainty metrics, including the following steps:

[0043] Step 1: In the virtual simulation environment, select tasks from the same distribution, i.e. For each task Using Bellman equations, calculate Among them, g t It is a task At each moment's sub-goal, ω - These are the parameters of the target network. The objective function is optimized using gradient descent. It can encode shared structural knowledge of control tasks into network parameters.

[0044] Step 2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum information content: Where: π Q (u)=SoftMax[π H (u t |s t ,g t In this decision state Provide guidance to soft robots.

[0045] Step 3: Change the decision state And related data are stored in the experience sharing cache.

[0046] Collecting and processing fruit fly larvae movement data, simulating fruit fly larvae movement scenarios as the algorithm interaction environment, and using the movement data and environmental data to run a soft robot multi-task learning and adaptation algorithm to train the underlying drive module, mechanical perception module, and central decision-making module, including the following steps:

[0047] Step 1: Build a virtual simulation environment and design environmental dynamics equations;

[0048] Step 2: Construct a biomimetic fruit fly larva soft robot model and import it into a virtual simulation environment;

[0049] Step 3: Train the underlying driver module. Collect and preprocess fruit fly larvae crawling data as the training dataset for the underlying driver module. Define the optimization objective of the driver module as minimizing the difference between the soft robot's state access distribution and the real fruit fly larvae's state access distribution. Use relative entropy to quantify the difference between the two distributions, i.e. Where d π This represents the state access distribution of the soft robot, while This is the true state distribution. Because... Given a fixed state access distribution, the final optimization objective can be obtained:

[0050] Step 4: Train the machine perception module. A semi-supervised learning method is used to train a neural network to approximate the transfer function g(·). This involves training a surrogate model using existing real data. This surrogate model is modeled as a Gaussian process (GP), which generates pseudo-labels for some unlabeled data. These model-generated data, combined with the existing real data, form an augmented dataset, which will be used to train the world model.

[0051] Step 5: Initialize the replay cache and experience sharing cache. The capacity is [size missing]. Update using the least recently used permutation algorithm;

[0052] Step 6: Train the central decision-making module using the DDQN algorithm. Define a sparse reward to capture the structural invariance between the task and the environment, i.e.:

[0053]

[0054] Step 7: Randomly select a virtual simulation environment and place the soft robot in that environment;

[0055] Step 8: Randomly select a control target, run a forward dynamics simulation, and collect sequence data, where u t =argmax u Q(s t ,u,g t ;ω), and the quintuple (s t ,u t ,r t ,s t+1 ,g t Store in the playback cache;

[0056] Step 9: Calculate using the Bellman equation.

[0057] Step 10: Define the objective function to be minimized

[0058] Step 11: Optimize the objective function using a gradient-based meta-learning algorithm. in

[0059] Step 12: Run steps 8-11 in a single virtual environment until convergence;

[0060] Step 13: Switch the virtual simulation environment, randomly select a control target, and let the soft robot explore in an unfamiliar environment;

[0061] Step 14: Using a maximum entropy-based active learning algorithm, select the decision state with the most information:

[0062] Step 15: In the status To enable a soft robot to perform exploratory actions, that is, to sample multiple different instructions. The state at the next time step is obtained using forward dynamics simulation.

[0063] Step 16: Settings And the objective function is optimized using gradient descent.

[0064] Step 17: Set the status Store in the experience-sharing cache;

[0065] Step 18: Run steps 13 to 17 until convergence.

[0066] After running the multi-task learning and adaptation algorithm for the soft robot until all modules converge, the hierarchical drive control algorithm for the soft robot is run to control the soft robot to complete the macro task in the specified environment. This includes the following steps:

[0067] Step 1: Specify a virtual environment, given a series of target points g1, g2, ..., g2 that need to be traversed. N ;

[0068] Step 2: Initialize the counter cnt = 1 and the time step t = 0;

[0069] Step 3: Based on the current ontology state s t , and g cnt The current control signal u is obtained from the central decision-making module. t ;

[0070] Step 4: Based on the control signal u t and the ontological state s t The state prediction value s for the next moment is obtained from the mechanical sensing module. t ′ +1 ;

[0071] Step 5: Based on the control signal u t , ontology state s t and state prediction value s t ′ +1 The action 'a' is obtained using the underlying driver module. t ;

[0072] Step 6: Perform action a t Input forward dynamics simulation to obtain the body state s at the next time step. t+1 And check whether the current control objective g has been achieved. cnt If the operation is completed, increment the counter cnt by 1; otherwise, do nothing.

[0073] Step 7: If cnt > N, the task ends; otherwise, return to step 3.

[0074] This invention simulated three types of environments: a rough flat plate (friction coefficient 0.9, low viscosity), a wet flat plate (friction coefficient 0.3, high viscosity), and a smooth flat plate (friction coefficient 0.3, low viscosity). Two advanced reinforcement learning control algorithms, TD3 and DDPG, were selected for comparison. The table below shows the target arrival success rate of the method under conditions of no external disturbance and with external disturbance.

[0075] Table 1 shows the success rate of the algorithm in the unperturbed experimental case.

[0076]

[0077] Table 2 shows the success rate of the algorithm under perturbation conditions.

[0078]

[0079] illustrate: Figure 1 The diagram comparing the neural circuits of a real fruit fly larva with the various modules designed in this invention clearly shows the location and function of each module. Figure 2 for Figure 1 Further details explain the information flow between the various modules. Figure 3 This is a schematic diagram illustrating a soft robot performing a navigation task in a virtual environment. Figure 4 This is the loss function curve during the simulation iteration process; Figure 5 t-SNE visualization of preprocessed behavioral data of fruit fly larvae and output actions of soft robots.

[0080] As can be seen from Tables 1 and 2, the control method proposed in this invention can excellently complete the control task under both normal conditions and conditions with external disturbances, and its performance far exceeds that of TD3 and PPO. Figure 1 It can be seen that the two systems designed in this invention correspond one-to-one with the neural circuits responsible for motor control and learning adaptation in real fruit fly larvae, exhibiting good biological interpretability and low computational cost and high efficiency. According to Figure 2 It can be seen that in the hierarchical drive control system, the flow of information exhibits a jump residual structure, meaning that the central decision-making module sends control signals to both the mechanical sensing module and the bottom-level drive module, thereby improving control accuracy. According to... Figure 3 It can be seen that the soft robot mimicking a fruit fly larva can successfully complete navigation tasks in a virtual environment, and during navigation, it basically moves along the shortest path between two target points. According to Figure 4 It can be seen that when the soft robot learns different commands (forward, left turn, right turn) in a virtual environment, the model loss decreases robustly. According to Figure 5As can be seen, after training, the motion pattern output of the soft robot highly overlaps with the label data, proving that after training, the soft robot can extract implicit manifold features from the label data and absorb them as its own implicit motion modes. The results above show that, with the support of the control method proposed in this invention, the fruit fly larva-inspired soft robot can continuously learn and optimize its motion patterns in a virtual environment, thereby rapidly and stably improving its motion control performance. When faced with complex and dynamic structured control tasks, the soft robot can efficiently explore the environment, autonomously generate adaptive motion patterns, and thus complete the control task, demonstrating excellent environmental adaptability and task execution efficiency.

[0081] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A multi-task control method for a fully soft robot mimicking a fruit fly larva, characterized in that, The system includes a hierarchical drive control system for soft robots and a multi-task learning and adaptation system for soft robots. The hierarchical drive control system is based on the sensory-motor-control neural circuit of larvae, mimicking the connection between high-level neurons AcNs and low-level neurons A01j and A02j. It consists of a mechanical sensing module, a low-level drive module, and a central decision-making module. The multi-task learning and adaptation system is based on the motor adaptation neural circuit of larvae, mimicking the connection between sensory neurons Basin-1 and Wave neurons, as well as the feedback effect on peristaltic CPG. It consists of a meta-reinforcement learning module based on short-term memory and an active learning module oriented towards uncertainty measurement. The central decision-making module, based on deep reinforcement learning, generates control signals according to the proprioceptive state and control objective, and sends them to the mechanical perception module and the underlying drive module. The mechanical perception module, based on a world model, receives the control signals, processes surrounding environmental information, predicts future state changes, and sends this prediction to the underlying drive module. The underlying drive module receives the control signals and the predicted environmental changes, activates the soft robot's actuators, and causes it to move. Based on the current state s... t Running the hierarchical drive control system, the functional relationship of the state at the next moment is obtained as: s t+1 =F(s) t A(s) t ,S(s t ,C(s t ,g t )),C(s t ,g t ))), where F(·,·) is the forward dynamics simulation, A(·,·,·) is the bottom-level driving module function, S(·,·) is the mechanical perception module function, and C(·,·) is the central decision module function; The information transmission relationships between the central decision-making module, the mechanical sensing module, and the underlying driving module are as follows: Step 4.1: At time step t, the central decision-making module determines the current proprioceptive state s of the soft robot. t and control target g t Output control signal π H (u t |s t ,g t ) = argmax u Q(s t ,u,g t ;ω), where Q(s) t ,a t ) = E π [G t |S t =s t A t =a t ;ω] is the action value function, It is the reward after time step t, γ is the discount factor, which is parameterized using a neural network, and ω represents the parameters of the neural network; Step 4.2: Mimicking the gating neurons in the larval cognitive system, the control signal output by the neural network needs to be modulated by the gating neural network. The gating neural network h(·) measures the similarity between the samples stored in the experience sharing buffer and the control signals output by the reinforcement learning network in the most recent H time steps. When the output of the gating neural network is greater than a threshold... At that time, that is The Model Predictive Control (MPC) submodule will be enabled to achieve precise control with small data samples; Step 4.3: The mechanical sensing module receives proprioceptive sensation s t and control signal u t Based on the world model, generate the predicted state value s′ for the next time step. t+1 The relation is s′ t+1 =g(s t ,u t )+ε, where ε is Gaussian noise, i.e., ε~N(0,I), and g(·) is a function parameterized by a neural network; Step 4.4: The underlying driver module receives proprioceptive sensation s t Control signal u t and the state prediction value s′ t+1 The underlying driver module π L (a t |s t ,u t ,s′ t+1 ) is the specific execution module for the control signals, where a t This indicates the action performed at time step t; Step 4.5: Perform action a t Inputting the data into the forward dynamics simulation yields the state s at the next time step. t+1 =F(s) t ,a t Repeat the above tasks until the control task is completed.

2. The multi-task control method for a fully soft robot mimicking a fruit fly larva as described in claim 1, characterized in that, The short-term memory-based meta-reinforcement learning module uses a model-independent gradient meta-learning algorithm to encode task structure information into network parameters. The uncertainty-oriented active learning module uses the maximum entropy active learning algorithm to quantify the uncertainty of the soft robot in the decision-making process, select the decision state with the most information, build an experience-sharing cache, and achieve precise control.

3. The multi-task control method for a fully soft robot mimicking a fruit fly larva as described in claim 1, characterized in that, The aforementioned short-term memory-based meta-reinforcement learning module and uncertainty-metric-oriented active learning module include the following steps: Step 5.1: In the virtual simulation environment, select tasks from the same distribution, i.e. For each task Using Bellman equations, calculate Among them, g t It is a task At each moment's sub-goal, ω - These are the parameters of the target network, which are optimized using gradient descent. It can encode shared structural knowledge of control tasks into network parameters; Step 5.2: Based on the maximum entropy active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum information content: Where: π Q (u)=SoftMax[π H (u t |s t ,g t In this decision state Provide guidance to soft robots; Step 5.3: Change the decision state And related data are stored in the experience sharing cache.

4. The multi-task control method for a fully soft robot mimicking a fruit fly larva as described in claim 1, characterized in that, After running the soft robot multi-task learning and adaptation system until all modules converge, run the soft robot hierarchical drive control system to control the soft robot to complete macro tasks in a specified environment, including the following steps: Step 6.1: Specify a virtual environment, given a series of target points g1, g2, ..., g that need to be traversed. N ; Step 6.2: Initialize the counter cnt = 1 and the time step t = 0; Step 6.3: Based on the current ontology state s t , and g cnt The current control signal u is obtained from the central decision-making module. t ; Step 6.4: According to the control signal u t and the ontological state s t The state prediction value s′ for the next moment is obtained from the mechanical sensing module. t+1 ; Step 6.5: According to the control signal u t , ontology state s t and the state prediction value s′ t+1 The action 'a' is obtained using the underlying driver module. t ; Step 6.6: Perform action a t Input forward dynamics simulation to obtain the body state s at the next time step. t+1 And check whether the current control objective g has been achieved. cnt If the operation is completed, increment the counter cnt by 1; otherwise, do nothing. Step 6.7: If cnt > N, the task ends; otherwise, return to step 6.

3.

5. The multi-task control method for a fully soft robot mimicking a fruit fly larva according to claim 1, characterized in that, Using a simulated fruit fly larvae movement scenario as the algorithm's interaction environment, and leveraging fruit fly larvae movement data and environmental data, a soft robot multi-task learning and adaptation system is run to train the underlying drive module, mechanical perception module, and central decision-making module. This includes the following steps: Step 7.1: Train the underlying driver module. Collect and preprocess fruit fly larvae crawling data as the training dataset for the underlying driver module. Define the optimization objective of the driver module as minimizing the difference between the soft robot's state access distribution and the real fruit fly larvae's state access distribution. Use relative entropy to quantify the difference between the two distributions, i.e. Where d π This represents the state access distribution of the soft robot, while It is the true state distribution, because Given a fixed state access distribution, the final optimization objective can be obtained: Step 7.2: Train the mechanical perception module. Use semi-supervised learning to train a neural network to approximate the transfer function g(·). That is, train a surrogate model using existing real data. The surrogate model is modeled as a Gaussian process (GP). The Gaussian process model will generate pseudo-labels for some unlabeled data. These model-generated data are combined with existing real data to form an augmented dataset. This augmented dataset will be used to train the world model. Step 7.3: Initialize the replay cache and experience sharing cache, with a capacity of [size missing]. Update using the least recently used permutation algorithm; Step 7.4: Use the DDQN algorithm to train the central decision-making module, and define a sparse reward to capture the structural invariance between the task and the environment, i.e.: Step 7.5: Randomly select a virtual simulation environment and place the soft robot in that environment; Step 7.6: Randomly select the control target, run the forward dynamics simulation, and collect sequence data, where u t =argmax u Q(s t ,u,g t ;ω), and the quintuple (s t ,u t ,r t ,s t+1 ,g t Store in the playback cache; Step 7.7: Calculate using the Bellman equation. Step 7.8: Define the objective function to be minimized. Step 7.9: Optimize the objective function using a gradient-based meta-learning algorithm. in Step 7.10: Run steps 7.6 to 7.9 in a single virtual environment until convergence; Step 7.11: Switch the virtual simulation environment, randomly select a control target, and let the soft robot explore in an unfamiliar environment; Step 7.12: Based on the maximum entropy-based active learning algorithm, quantify the uncertainty of the soft robot in the task decision-making process and select the decision state with the maximum information content: Where: π Q (u)=SoftMax[π H (u t |s t ,g t )]; Step 7.13: In the state To enable a soft robot to perform exploratory actions, that is, to sample multiple different instructions. The state at the next time step is obtained using forward dynamics simulation. Step 7.14: Settings And the objective function is optimized using gradient descent. Step 7.15: Set the status Store in the experience-sharing cache; Step 7.16: Run steps 7.11 to 7.15 until convergence.

Images