Modeling and hierarchical motion control method for a muscle-driven Drosophila larvae-like intelligent agent

Abstract

The present invention discloses a muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method. The multi-segmented soft body of the Drosophila larvae is modeled based on the spring-mass method, the muscle tension calculation nodes are modeled based on the muscle length-velocity-tension nonlinear model, and the hydrostatic skeleton is modeled based on the volume constraint algorithm. The Drosophila larvae-like intelligent body is formed under the above-mentioned simulation modeling framework. For the Drosophila larvae-like intelligent body, a muscle bottom-level controller based on the von-Mises cyclically distributed central pattern generator is constructed to compress the control dimension from the number of muscles to the number of bottom-level controller parameters; a reinforcement learning method is used to optimize the high-level strategy network that controls the bottom-level controller parameters, so that the high-dimensional muscle system can drive the Drosophila larvae-like intelligent body to produce multi-mode peristaltic motion under the coordination of the above-mentioned hierarchical motion control method. The present invention provides a new structural design and control algorithm, which will help promote the application of soft robots in industry, medicine and other fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of bionic soft robots, and in particular relates to a muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method. Background Art

[0002] Compared to traditional rigid-body robots, soft robots offer advantages such as flexible movement and strong environmental adaptability. However, due to the nonlinear properties and complex deformation behavior of flexible materials, their structural design and control face numerous challenges. In terms of structural design, soft robots need to support large deformations, which requires materials that are both flexible and maintain a certain level of strength and durability. The design also needs to consider multimodal motion requirements and process feasibility. Furthermore, due to the lack of well-defined joints and fixed body shapes in soft robots' flexible structures, traditional rigid-body control theory is difficult to directly apply to their kinematic modeling and dynamic analysis.

[0003] Compared to the structural design and control difficulties currently faced by soft robots, soft-bodied organisms in nature, such as octopuses, earthworms, and sea anemones, demonstrate remarkable environmental adaptability and flexibility thanks to their soft body structures and diverse modes of movement. For example, an octopus's tentacles can precisely grasp, bend freely, and adapt to complex terrain, while earthworms can easily traverse narrow spaces by peristalsis. Inspired by the movement behaviors of these organisms, the biomimetic design of soft robots is expected to mimic the deformation and movement patterns of organisms, achieving multimodal functions such as grasping, crawling, and swimming. Therefore, modeling and control technology for biomimetic soft robots is an important technical means to develop new soft robot structural designs and their control systems.

[0004] As a tiny mollusk, fruit fly larvae exhibit high flexibility and adaptability in their movement behavior. They can precisely control muscle contraction to achieve efficient wave-like peristalsis and maintain stable movement performance even in complex terrain or confined spaces. In addition, they have excellent directional control capabilities and rapid environmental response capabilities, and can quickly adjust their movement trajectory under light, temperature or chemical stimulation. They are one of the important target organisms in current bionic soft robot research.

[0005] Existing studies on the modeling and control of Drosophila larvae movement have greatly simplified their bodies. For example, the paper "Integrative neuromechanics of crawling in D. melanogaster larvae" (translated as "Integrated neuromechanics of crawling in D. melanogaster larvae", "eLIFE", Issue 5, 2016) simplified the multi-segmented structure of the Drosophila larvae into eleven mass points connected in series in one-dimensional space, and only studied the straight and backward crawling of the Drosophila larvae; the paper "Modelling the mechanics of exploration in larval Drosophila" (translated as "Simulating the exploration mechanism of Drosophila larvae", "PLOS Computational Biology", Volume 15, 2019) modeled the Drosophila larvae in two-dimensional space as a group of mass points connected by torsion springs. By controlling the stiffness of the torsion springs, the larvae's left and right deflection exploratory movement was studied. In summary, existing research lacks the design of the body structure of fruit fly larvae and the analysis of multimodal motion control in three-dimensional space, which makes it difficult to support in-depth research on fruit fly larvae-like soft robots.

[0006] To address the challenges of existing solutions, the present invention models the multi-segmented, fully soft body of a Drosophila larva in three-dimensional space using a spring-mass method, models muscle tension calculation nodes based on a nonlinear muscle length-velocity-tension model, and models a hydrostatic skeleton based on a volume constraint algorithm. Within this simulation modeling framework, a Drosophila larvae-like intelligent agent is constructed. For this Drosophila larvae-like intelligent agent, a low-level controller for muscle groups based on a von Mises cyclically distributed central pattern generator is constructed, reducing the control dimension from the number of muscles to the number of low-level controller parameters. A reinforcement learning method is then used to optimize the high-level policy network that controls the low-level controller parameters. This allows the high-dimensional muscle system to drive the Drosophila larvae-like intelligent agent to produce multi-modal peristaltic motions, coordinated by the aforementioned hierarchical motion control method. The constructed Drosophila larvae-like intelligent agent provides a novel material structure and control algorithm research platform for biomimetic soft robotics, and will be of great significance for the research of self-propelled soft robots in industrial and medical fields. Summary of the Invention

[0007] The present invention aims to provide a muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method to address the technical problems of existing technologies that are unable to model and control the multi-modal motion of Drosophila larvae in three-dimensional space and are difficult to support the research of biomimetic soft robots. The present invention specifically provides the following technical solutions:

[0008] A muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method includes: muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control for the Drosophila larvae-like intelligent agent. The body of the muscle-driven Drosophila larvae-like intelligent agent is simulated by a mass-damper-spring system. The body contains multiple muscle tension calculation nodes. The muscle tension calculation nodes are controlled by a motion control system. When activated, they generate tension between two mass points, thereby changing the body's morphological structure. The hierarchical motion control method for the Drosophila larvae-like intelligent agent receives body morphology and environmental stimulus information, calculates activation signals for each muscle tension calculation node, and drives the Drosophila larvae-like intelligent agent to continuously adjust the spatial structure of the mass-damper-spring system, generating contact forces on the motion plane, thereby causing the Drosophila larvae-like intelligent agent to produce multi-modal peristaltic motion.

[0009] Furthermore, the mass-damper-spring system's spatial structure is based on the 11-segment structure of a fruit fly larva, with the mass points connected by a parallel spring-damper model. The ventral plane contains 22 mass points, the dorsal curved surface contains 20 mass points, the head and tail segment mass points are located along the symmetric axis of the body in the ventral plane, and the remaining mass points are located at the four corners of the rectangular plane separating adjacent segments. The connection relationship between the mass points in the parallel spring-damper model is as follows: the four mass points within the same segment are interconnected, and adjacent segments share the mass structure at the segmental divisions.

[0010] Furthermore, in the spring model, when the length change is the same, the tensile tension is greater than the compressive tension, so that the elastic structure of the fruit fly larva body modeled by the spring can maintain the body shape under the support of the hydrostatic skeleton without restricting the left and right bending of the body.

[0011] Furthermore, the spatial structure distribution of the muscle tension calculation nodes is simplified based on the anatomical structure of the Drosophila larvae muscles. Each body segment contains four groups of muscle tension calculation nodes, namely, dorsal axial, abdominal axial, circumferential and oblique. Among them, the circumferential muscles can lift the ventral side of the current body segment away from the movement plane during peristalsis, thereby reducing the friction force on the body segment; the dorsal and abdominal axial muscles can pull the body segment to the adjacent volume, promoting the successive displacement of each body segment; the oblique muscles simultaneously play the role of lifting the abdomen and pulling the body segment. By regulating different muscle activation intensities, under the action of the above-mentioned muscle coordination mechanism, the Drosophila larvae-simulated intelligent body can produce peristaltic displacement.

[0012] Furthermore, the contraction tension process generated by the muscle tension calculation node is nonlinear, and the muscle tension calculation includes the following steps:

[0013] S11, traverse each muscle tension calculation node, for calculation node i, calculate its geometric length L i , there is L i =||p 1,i -p2,i ||, where p 1,i and p 2,i Respectively represent the spatial positions at both ends of the computational node, ‖·‖ represents the calculation of the vector two-norm;

[0014] S12, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-muscle length relationship F l , there is F l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters;

[0015] S13, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-contraction speed relationship F v ,have Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction velocity;

[0016] S14, traverse each muscle tension calculation node, and for calculation node i, calculate its passive muscle tension F PeE ,have Among them L opt Indicates the optimal output length of the muscle, c and A are preset parameters;

[0017] S15, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle activation strength a i ,have where c a is the preset parameter, u i Represents the control signal of the muscle computing node;

[0018] S16, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension F m , there is F m =a i F max F l F v +F PEE .

[0019] Furthermore, in the simulated fruit fly larvae peristalsis process, the mass-damper-spring system is subjected to a support force whose total volume of the constrained body remains unchanged, and calculating the volume constraint force includes the following steps:

[0020] S21, traverse each triangle on the surface of the Drosophila larvae intelligent body, and for the i-th triangle, calculate its normal vector toward the outside of the body have where a i 、b i 、c i are the spatial coordinates of the three vertices of the triangle, (a i -c i )×(b i -c i ) represents the cross product operation of two vectors, ‖·‖ represents the calculation of the vector's two-norm;

[0021] S22, traverse all triangles on the surface of the Drosophila larvae-like intelligent body, and calculate the area S of the i-th triangle. i , there is S i =‖(a i -c i )×(b i -c i )‖ / 2;

[0022] S23, calculate the volume V of the space enclosed by each triangle,

[0023] S24, calculate the volume constraint force f on mass point k k ,have where α V Represents the parameter that controls the size of the volume constraint force, V0 represents the volume size of the Drosophila larvae-like intelligent agent in the relaxed state, and Ω represents the triangle surface adjacent to the particle k.

[0024] Furthermore, the hierarchical motion control method for the Drosophila larvae-like agent is a two-layer structure, comprising a low-level motion controller that generates periodic signals to directly control each muscle, and a high-level motion controller that senses external and internal information for motion planning and controls the parameters of the periodic signals output by the low-level motion controller. The control system controls the muscles of the Drosophila larvae-like agent to drive it to produce multi-modal peristaltic motion, including the following steps:

[0025] S31, initializing the Drosophila larvae-like intelligent agent: Initializing the Drosophila larvae-like intelligent agent to a relaxed state, clearing all force buffers, and setting the initial position of each particle to x and the initial velocity to v;

[0026] S32, calculate the force on each particle: calculate the gravity F on each particle g , volume constraint force F v , spring force F s , damping force F d ;

[0027] S33, calculate the contact force of the abdominal mass point: calculate the contact force F of the abdominal mass point in contact with the motion plane c , project the velocity and force of the abdominal mass point to the normal space of the contact plane<n,t,s> , where s is the normal vector perpendicular to the contact surface, if [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ||[f n ,f t ]||<μ i f s , if [v n ,v t ] is not a 0 vector, then the contact force F c exist<n,t> The components of the plane are

[0028] S34, calling the high-level motion controller: inputting the environment and proprioception information into the trained and optimized high-level motion controller to obtain the control parameters k and s of the low-level motion controller;

[0029] S35, call the underlying motion controller and calculate muscle tension: Based on the control parameters k and s, obtain the activation signal a of the underlying motion controller acting on each muscle, and calculate the muscle tension F m ;

[0030] S36, update the mass point velocity: based on the force on the mass point, calculate the updated mass point velocity v t+1 , there is v t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length;

[0031] S37, update the mass point position: based on the mass point velocity update amount, calculate the mass point position update amount x t+1 =x t +v t+1 dt;

[0032] S38, loop iterates the simulation time step: if the upper limit of the simulation time step is not reached, loop iterate the calculation steps S32 to S37 and record the particle position of each time step; if the upper limit of the simulation time step is reached, stop the loop and input the particle position sequence into the renderer to obtain the multi-mode peristaltic motion video image result of the Drosophila larvae intelligent agent.

[0033] Furthermore, the underlying motion controller periodic signal is generated by the von-mises cyclic distribution: Where mod represents the remainder, t represents the simulation time length, n∈{0,1,…,10} represents the segment number, s∈[-1,1] controls the transmission speed of peristaltic waves between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Representing the activation intensity of each muscle group, this periodic signal can cyclically activate the muscle tension calculation nodes of each body segment in the Drosophila larvae-like intelligent body with different ranges, intensities and speeds, thereby generating peristaltic waves that regulate basic motor behavior.

[0034] Furthermore, the external information sensed by the high-level motion controller is the rate of change of the stimulation field intensity of the head mass point position: Where x0 represents the spatial position of the particle on the head of the Drosophila larvae-like intelligent agent, S(x0) represents the intensity of the external stimulus field at x0, represents the rate of change of the external stimulus field intensity over time, and P represents the external information input by the controller. The perceived ontological information is the muscle length of each muscle tension calculation node of the Drosophila larvae agent at the simulation time step.

[0035] Furthermore, the high-level motion controller is constructed based on an artificial neural network, the network architecture is a transformer network, the network parameters are optimized by a proximal strategy optimization algorithm under the reinforcement learning framework, the network input is the environment and proprioceptive perception information of the fruit fly larvae-like intelligent agent at each time step within 1 to N time steps, and the network output is the control parameters of the underlying motion controller within N+1 to 2N time steps.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention uses a computer simulation program for fruit fly larvae as a carrier, and is guided by the flexible motion control of fruit fly larvae-like intelligent bodies. It develops motion control strategies for soft robots based on bionic research. The present invention uses the spring-mass method to finely model the multi-segmented soft body of fruit fly larvae in three-dimensional space, combines muscle anatomical structure modeling with muscle tension calculation nodes, and uses a volume constraint algorithm to simulate a hydrostatic skeleton, forming a more realistic fruit fly larvae-like intelligent body. At the same time, a muscle group bottom-level controller based on the von-Mises cyclically distributed central pattern generator is proposed. Combined with a high-level strategy network optimized by reinforcement learning, a hierarchical motion control method is constructed, which greatly compresses the control dimension of the high-dimensional muscle system and realizes precise control of the multi-mode peristaltic motion of the fruit fly larvae-like intelligent body. Compared with the existing technology, the present invention overcomes the shortcomings of traditional models in three-dimensional modeling and multi-mode motion control, provides new structural design and control algorithms for bionic soft robot research, and helps promote the application of soft robots in industrial, medical and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0039] Figure 1 This is the overall flow chart of the Drosophila larvae-like intelligent agent modeling and hierarchical motion control method provided by the present invention;

[0040] Figure 2 A schematic side view of the spring-mass-damper system structure of the fruit fly larvae-simulating intelligent body provided by the present invention;

[0041] Figure 3 A schematic top view of the muscle computing node structure of the Drosophila larvae-like intelligent body provided by the present invention;

[0042] Figure 4 This is a schematic diagram of the rendering results of the motion of the fruit fly larvae-like intelligent body provided by the present invention;

[0043] Figure 5 A schematic diagram of the movement results of the fruit fly larvae-like intelligent agent provided by the present invention in a complex stimulus environment;

[0044] Figure 6 A comparison chart of the convergence results of the optimization process between the hierarchical motion control method and the single-layer motion control method for the Drosophila larvae-like intelligent agent provided by the present invention;

[0045] Figure 7 A schematic diagram of the morphological changes of the Drosophila larvae-like intelligent body during its movement in a complex stimulus environment provided by the present invention;

[0046] The numbers in the figure represent the following:

[0047] 1-head mass point; 2-tail mass point; 3-fifth body segment; 4-third body segment dividing plane; 5-ventral mass point; 6-dorsal mass point; 7-ventral axial muscle; 8-circumferential muscle; 9-oblique muscle; 10-dorsal axial muscle; 11-Drosophila larvae-like intelligent agent in a complex stimulation environment; 12-stimulus source; 13-motion trajectory of the Drosophila larvae-like intelligent agent; 14-the position with the weakest field strength in the stimulation environment; 15-the average reward value of the hierarchical motion control method changes with training iterations; 16-the average reward value of the single-layer motion control method changes with training iterations. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1 As shown, the present invention provides a muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method, comprising the following steps:

[0051] Step 1: Initialize the Drosophila larvae-like intelligent body, clear all force buffers, and make the Drosophila larvae-like intelligent body spring-damper connection structure and muscle calculation nodes in a relaxed state. In the initial relaxed state, the Drosophila larvae-like intelligent body mass-damper-spring system spatial structure is as follows: Figure 2 As shown, the initial position of the mass point is set as the dot position in the figure, where the initial positions of the head and tail mass points are respectively Figure 2 The positions of the head mass point 1 and the tail mass point 2 are set as shown, and the initial positions of the other ventral and dorsal mass points are respectively as follows Figure 2 The positions of the mid-ventral mass point 5 and the dorsal mass point 6 are shown; the spring-damper parallel connection structure is as shown Figure 2 The midline segment connection relationship is set, and its connection relationship forms the morphological structure of each segment of the fruit fly larva, such as Figure 2 The position shown in the fifth body segment 3 is the spatial position setting of the fifth body segment of the Drosophila larvae intelligent body. Along the body axis toward the head mass point, the fourth body segment is forward, the sixth body segment is backward, and the space between the third and fourth body segments is Figure 2 The third body segment dividing surface shown in the third body segment dividing surface 4; muscle tension calculation node, spatial structure distribution is as follows Figure 3 As shown, based on the simplified anatomical structure of Drosophila larvae muscles, each body segment contains a dorsal axis (such as Figure 3 The space structure is set at the position of the dorsal axial muscle 10), the abdominal axial (such as Figure 3 The space structure is set at the position shown in the middle ventral axial muscle 7), the circumferential (such as Figure 3 8) and obliquely (as shown in the figure Figure 3 The spatial structure is set at the position shown in the middle oblique muscle 9) with a total of four groups of muscle tension calculation nodes. The four groups of muscles are symmetrical along the axial plane perpendicular to the abdominal plane along the head-tail line. Except for the head and tail body segments, the spatial structure of the muscle tension calculation nodes of all other body segments is the same.

[0052] Step 2: Generate a stimulation field in the motion plane of the Drosophila larvae-like intelligent body based on the mixed Gaussian model to simulate the spatial distribution of stimulus sources in the environment and provide the intelligent body with external perception information. First, define multiple Gaussian functions in the range of [-10, 10] on the xy plane. Each Gaussian function represents an independent stimulus source, and its position (mean), intensity (amplitude) and diffusion degree (variance) can be set according to the simulation requirements. These Gaussian functions are linearly combined according to the weights to form a mixed Gaussian model. The model presents multiple peaks and valleys in space, simulating the superposition effect of multiple stimulus sources in the real environment. According to the above process, multiple stimulus source positions are randomly generated (such as Figure 5 The stimulus source 12 in the middle) and the degree of diffusion of the stimulus field generate the stimulus field strength S at each position in the plane, and obtain a complex stimulus environment. The initialized Drosophila larvae-like intelligent agent is randomly placed in the complex stimulus environment (as shown in FIG. Figure 5 (as shown in Figure 11, which shows a Drosophila larva-like intelligent agent in a complex stimulus environment).

[0053] Step 3: Calculate the force based on the position and velocity of each particle. The gravity is calculated by the formula F g =Mg, g represents the gravitational constant; the volume constraint force is calculated by the formula Calculated, where α V represents the parameter for regulating the size of the volume constraint force, V0 represents the volume size of the Drosophila larvae agent in the relaxed state, Ω represents the triangle surface adjacent to the mass point k, Represents the volume of space enclosed by each triangle; the spring force is given by Calculated, where L r represents the relaxed length of the spring, x2 and x1 are the positions of the masses at both ends of the spring, if ||x2-x1||>L r , then Otherwise k=k0; the damping force is calculated by Calculated.

[0054] Step 4: Calculate the contact force based on the position of the ventral mass point, and project the velocity and force of the abdominal mass point into the normal space of the contact plane.<n,t,s> , where s is the normal vector perpendicular to the contact surface, if [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ||[f n ,f t ]||<μ i f s , if [v n ,v t ] is not a 0 vector, then the contact force F c exist<n,t> The components of the plane are

[0055] Step 5: Use the "transformer" neural network to build a high-level motion controller, and optimize the network parameters using the proximal strategy optimization algorithm under the reinforcement learning framework. Input the environment and proprioceptive information of the Drosophila larvae agent at each time step from 1 to N time steps. The perceived external information is the rate of change of the stimulus field intensity of the head mass point position, which can be obtained by the formula Calculated, where x0 represents the spatial position of the particle head of the Drosophila larvae-like intelligent body, S(x0) represents the intensity of the external stimulus field at x0, It represents the rate of change of the intensity of the external stimulus field over time, P represents the external information input by the controller, and the perceived ontological information is the muscle length of each muscle tension calculation node of the Drosophila larvae intelligent agent in the simulation time step.

[0056] Step 6: Construct a parameterized underlying motion controller based on Von-Mises distribution, where each muscle activation signal can be expressed as Calculated, where mod represents the remainder, t represents the simulation time length, n∈{0,1,…,10} represents the segment number, s∈[-1,1] controls the transmission speed of peristaltic waves between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Indicates the activation intensity of each muscle group. This periodic signal can activate the muscle tension calculation nodes of each body segment in the Drosophila larvae intelligent body in different ranges, intensities and speeds, thereby generating peristaltic waves that regulate basic movement behavior. m =a i F max F l F v +F PeE The output tension of the muscle tension calculation node can be calculated, where a i Indicates muscle activation signal, c a is the preset parameter, u i Represents the control signal of this muscle computation node. Among them L opt Indicates the optimal output length of the muscle, c and A are preset parameters. Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction speed. l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters.

[0057] Step 7: Calculate the updated velocity v based on the force on the particle t+1 , there is v t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length;

[0058] Step 8: Calculate the updated mass position x based on the updated mass velocity t+1 =x t +v t+1 dt.

[0059] Step 9: If the upper limit of the simulation time step is not reached, the calculation steps from step 3 to step 8 are iterated, and the particle position of each time step is recorded; if the upper limit of the simulation time step is reached, the loop is stopped, and the particle position sequence is input into the renderer to obtain the following Figure 4 and Figure 7 The video images of the multi-modal peristaltic motion of the Drosophila larvae-like intelligent agent are shown.

[0060] Statistics of the motion trajectory of the head mass point of the Drosophila larvae-like intelligent agent in the complex stimulus environment on the xy plane during the above iteration process, such as Figure 5 As shown in the trajectory of the Drosophila larvae-like intelligent body in Figure 13, it can be observed that the Drosophila larvae-like intelligent body can continuously manipulate the complex muscle structure along the direction of weakening stimulus field strength, and drive the entire soft body to reach the position with the weakest field strength in the complex stimulus environment ( Figure 5 The weakest field strength position in the medium stimulation environment is 14), which shows that the muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method proposed in the present invention has excellent body manipulation ability and environmental navigation adaptability, and can solve the problem of multi-mode soft peristaltic motion control in high-dimensional control space.

[0061] As the "transformer" neural network of the high-level motion controller skeleton network, it consists of an encoder and a decoder. The overall network structure includes 6 layers of encoders and 6 layers of decoders. The model latent space dimension is 512, the feedforward network dimension is 2048, and the number of attention heads is 8. The embedding layer of the encoder maps the perceived external information and ontological information to a 512-dimensional vector space, and adds position encoding generated by sine and cosine functions to retain the temporal information of the sequence. The feedforward neural network is a two-layer fully connected network. The activation function uses ReLU, which increases the dimension from 512 dimensions to 2048 dimensions and then reduces the dimension back to 512 dimensions. Residual connections are added after each sublayer and layer normalization is performed. The structure of the decoder is similar to that of the encoder, but a mask mechanism is used in the self-attention mechanism. The output layer maps the output of the decoder to the control signal space through linear mapping, and uses the Softmax activation function to generate the final control signal u i .

[0062] During parameter optimization of the high-level motion control network, the Drosophila larvae-like agent interacts with the environment under its current policy to collect empirical data, including information such as the state at each time step, the action taken, the reward received, and the next state. Using this collected data, the agent calculates an advantage function at each time step by estimating the difference between the future cumulative reward and the baseline value. This function measures the relative performance of an action relative to the average. When updating network parameters, policy gradient clipping ensures that the new policy does not deviate too far from the old policy, preventing excessive policy updates that could lead to unstable performance. The entropy regularization term in the optimization objective increases the policy's randomness, encouraging the Drosophila larvae-like agent to explore a wider range of action options and avoid prematurely falling into local optima. Network parameter optimization is an iterative process. In each iteration, the Drosophila larvae-like agent interacts with the environment according to its current policy, collecting new data samples. These samples are then used to update the parameters of the high-level motion controller. After multiple iterations, the parameters of the high-level motion controller are gradually optimized, enabling the Drosophila larvae-like agent to achieve flexible, multimodal locomotion.

[0063] The layered motion control method for the Drosophila larvae-like intelligent agent proposed in this invention is replaced by a single-layer motion control method, that is, the neural network is directly used to control the muscles of the Drosophila larvae-like intelligent agent, and the reinforcement learning method is also used to optimize the neural network parameters. The optimization results are as follows: Figure 6 As shown in the figure, the average reward value of the Drosophila larvae agent controlled by the two methods in a complex stimulus environment with the number of training iterations is statistically analyzed. The larger the average reward value, the better the environmental adaptability of the control method used. Figure 6 The average reward value of the hierarchical motion control method changes with training iterations. 15 is the average reward value curve of the hierarchical motion control method proposed in the present invention. Figure 6Figure 16 shows the average reward curve for the single-layer motion control method. It can be observed that the single-layer motion control method consistently fails to achieve a higher average reward, indicating that it is unable to complete the Drosophila larvae-like motion control task in a complex stimulus environment. This demonstrates that the proposed layered motion control method can quickly form a well-performing motion control strategy.

[0064] Combine Figure 5 and Figure 6 The results of the embodiment shown can verify that the muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method proposed in the present invention effectively solves the technical difficulties of existing soft robots in accurately modeling and realizing multi-mode motion control in three-dimensional space. Due to the nonlinear characteristics of material flexibility and complex deformation behavior, the structural design and control strategy of traditional soft robots are complex and it is difficult to achieve efficient motion modes. In addition, existing studies on motion modeling of Drosophila larvae have mostly simplified their body structure and lack in-depth analysis of three-dimensional structure and diversified motion control, which limits the application ability of bionic soft robots in complex environments. The present invention constructs a realistic Drosophila larvae-like intelligent body by using a spring-mass-damper system to accurately simulate the multi-segmented soft body of Drosophila larvae in three-dimensional space, combining the nonlinear model of muscle length-velocity-tension to accurately calculate muscle tension, and introducing a volume constraint algorithm to simulate the hydrostatic skeleton. In addition, the innovative use of a central pattern generator based on von-Mises cyclic distribution as the underlying controller, combined with a high-level policy network optimized by reinforcement learning, realizes hierarchical motion control of high-dimensional muscle systems, greatly compresses the control dimension, and improves the accuracy and adaptability of multi-modal peristaltic motion.

[0065] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method, characterized in that: The method includes: modeling a muscle-driven Drosophila larvae-like intelligent body and hierarchical motion control for the Drosophila larvae-like intelligent body, wherein the muscle-driven Drosophila larvae-like intelligent body is simulated by a mass-damper-spring system, and the body contains multiple muscle tension calculation nodes, which are regulated by a motion control system. After controlled activation, the muscle tension calculation nodes generate tension between two mass points, thereby changing the body shape and structure; the hierarchical motion control method for the Drosophila larvae-like intelligent body receives body shape and environmental stimulus information, calculates activation signals of each muscle tension calculation node, and drives the Drosophila larvae-like intelligent body to continuously change the body shape and structure, thereby generating multi-mode peristaltic motion; The muscle-driven Drosophila larvae-like intelligent agent model is modeled. The muscle tension calculation is regulated by a hierarchical motion controller, and the contraction tension process exhibits nonlinear characteristics. When the actual length of the muscle is inconsistent with the preset optimal output length, the tension generated by the controlled muscle contraction will decrease. Similarly, when the muscle contraction speed is inconsistent with the preset optimal processing contraction speed, the tension generated by the controlled muscle contraction will also decrease accordingly. Calculating muscle tension includes the following steps: S11, traverse each muscle tension calculation node, for calculation node i, calculate its geometric length L i , there is L i =||p 1,i -p 2,i ||, where p 1,i and p 2,i Respectively represent the spatial positions at both ends of the computational node, ‖·‖ represents the calculation of the vector two-norm; S12, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-muscle length relationship F l , there is F l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters; S13, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-contraction speed relationship F v ,have Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction velocity; S14, traverse each muscle tension calculation node, and for calculation node i, calculate its passive muscle tension F PEE ,have Among them L opt Indicates the optimal output length of the muscle, c and A are preset parameters; S15, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle activation strength a i ,have where c a is the preset parameter, u i Represents the control signal of the muscle computing node; S16, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension F m , there is F m =a i F max F l F v +F PEE .

2. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body modeling is implemented using a mass-damper-spring system. The spatial structure of the mass points is set according to the body segment structure of the Drosophila larvae. In a relaxed state with no muscles activated, the two-dimensional geometric shape of the abdominal mass points in contact with the moving surface projected on the moving surface is consistent with the spatial position of the posture points at both ends of adjacent body segments in the relaxed state of a real Drosophila larvae. The mass points are connected by a parallel model of nonlinear springs and dampers. The connection structure between the mass points is that the mass points in the same body segment are connected to each other in pairs, and adjacent body segments share the mass point structure at the body segment. When the change in spring length is the same, the tensile tension is greater than the compressive tension.

3. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body is modeled, and the muscle tension calculation nodes are simplified according to the anatomical structure of the Drosophila larvae muscles. Each body segment contains four groups of muscle tension calculation nodes, namely, dorsal axial, abdominal axial, circumferential, and oblique. The spatial structure distribution of these nodes is symmetrical along the axial plane perpendicular to the abdominal plane along the head-tail line. Except for the head-tail body segment, the spatial structure of the muscle tension calculation nodes of all other body segments is the same.

4. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body modeling, the process of muscle contraction changing the body morphology and structure, the supporting force acting on the constrained body with a constant total volume, and the calculation of the volume constraint force include the following steps: S21, traverse each triangle on the surface of the Drosophila larvae intelligent body, and for the i-th triangle, calculate its normal vector toward the outside of the body have where a i 、b i 、c i are the spatial coordinates of the three vertices of the triangle, (a i -c i )×(b i -c i ) represents the cross product operation of two vectors, ‖·‖ represents the calculation of the vector's two-norm; S22, traverse all triangles on the surface of the Drosophila larvae-like intelligent body, and calculate the area S of the i-th triangle. i , there is S i =‖(a i -c i )×(b i -c i )‖ / 2; S23, calculate the volume V of the space enclosed by each triangle, S24, calculate the volume constraint force f on mass point k k ,have where α V Represents the parameter that controls the size of the volume constraint force, V0 represents the volume size of the Drosophila larvae-like intelligent agent in the relaxed state, and Ω represents the triangle surface adjacent to the particle k.

5. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The hierarchical motion control method for a Drosophila larvae-like intelligent agent is a two-layer structure, comprising a bottom-level motion controller that generates periodic signals to directly control each muscle, and a high-level motion controller that senses external and body information to perform motion planning, regulates the bottom-level motion controller, and outputs periodic signal parameters. The control system regulates each muscle of the Drosophila larvae-like intelligent agent to drive it to produce multi-mode peristaltic motion, comprising the following steps: S31, initialize the Drosophila larvae-like intelligent agent to a relaxed state, clear all force buffers, and set the initial position of each particle to x and the initial velocity to v; S32, calculate the gravity F of each particle g , volume constraint force F v , spring force F s , damping force F d ; S33, calculate the contact force F on the abdominal mass point in contact with the motion plane c , project the velocity and force of the abdominal mass point to the normal space of the contact plane<n,t,s> , where s is the normal vector perpendicular to the contact surface, if [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ||[f n ,f t ]||<μ i f s , if [v n ,v t ] is not a 0 vector, then the contact force F c exist<n,t> The components of the plane are S34, inputting the environment and proprioceptive perception information into the trained and optimized high-level motion controller to obtain the control parameters κ and s of the low-level motion controller; S35, based on the control parameters κ and s, obtain the activation signal a of the underlying motion controller acting on each muscle and calculate the muscle tension F m ; S36, based on the force on the mass point, calculate the updated mass point velocity v t+1 , there is v t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length; S37, based on the mass point velocity update amount, calculate the mass point position update amount x t+1 =x t +v t+1 dt.

6. The hierarchical motion control method for a Drosophila larvae-like intelligent agent according to claim 5, characterized in that: The periodic signal generated directly controls the underlying motion controller of each muscle. The periodic signal is generated by the von Mises cyclic distribution: Where mod represents the remainder, t represents the simulation time length, n∈{0,1,…,10} represents the segment number, s∈[-1,1] controls the transmission speed of peristaltic waves between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Indicates the activation intensity of each muscle group.

7. The hierarchical motion control method for a Drosophila larvae-like intelligent agent according to claim 5, characterized in that: The high-level motion controller that outputs periodic signal parameters senses the external information as the rate of change of the stimulus field intensity of the head mass point position: Where x0 represents the spatial position of the particle on the head of the Drosophila larvae-like intelligent agent, S(x0) represents the intensity of the external stimulus field at x0, It represents the rate of change of the intensity of the external stimulus field over time, P represents the external information input by the controller, and the perceived ontological information is the muscle length of each muscle tension calculation node of the Drosophila larvae intelligent agent in the simulation time step.

8. The hierarchical motion control method for a Drosophila larvae-like intelligent agent according to claim 5, characterized in that: The high-level motion controller that outputs periodic signal parameters is constructed based on an artificial neural network. The network architecture is a transformer network, and the network parameters are optimized by a proximal strategy optimization algorithm under the reinforcement learning framework. The network input is the environmental and proprioceptive perception information of the fruit fly larvae-like intelligent agent at each time step from 1 to N time steps, and the network output is the control parameters of the low-level motion controller in the N+1 to 2N time steps.

Images