Muscle-driven fruit fly larva imitating intelligent body modeling and layered motion control method

Abstract

The invention discloses a muscle-driven fruit fly larva-imitating intelligent body modeling and hierarchical motion control method, which comprises the following steps of: modeling a fruit fly larva multi-body-segment all-soft body based on a spring mass point method, modeling a muscle tension calculation node based on a muscle length-speed-tension nonlinear model, modeling a static fluid skeleton based on a volume constraint algorithm, and performing hierarchical motion control on the static fluid skeleton. And forming a drosophila larva imitating intelligent body under the simulation modeling framework. The method comprises the following steps: constructing a muscle bottom layer controller based on a von-mises cyclic distribution central mode generator for a fruit fly larva imitating intelligent body, and compressing a control dimension from a muscle number to a bottom layer controller parameter number; a reinforcement learning method is adopted to optimize a high-level strategy network for controlling parameters of a bottom-layer controller, so that a high-dimensional muscle system can drive a drosophila larva imitating intelligent body to generate multi-mode wriggling movement under the coordination of the hierarchical movement control method. The invention provides a novel structural design and a novel control algorithm, which are beneficial for promoting the application of the soft robot in the fields of industry, medical treatment and the like.

Description

Technical Field

[0001] The invention belongs to the technical field of bionic soft robots, and in particular relates to a muscle-driven fruit fly larvae-like intelligent body modeling and hierarchical motion control method. Background Art

[0002] Compared with traditional rigid robots, soft robots have advantages such as flexible movement and strong environmental adaptability. However, due to the nonlinear characteristics and complex deformation behavior of flexible materials, their structural design and control face many difficulties. In terms of structural design, soft robots need to support large deformation capabilities, which requires that the materials have sufficient flexibility and maintain a certain strength and durability. In the design, it is also necessary to take into account the multi-modal motion requirements and the feasibility of the processing technology; in terms of control, due to the lack of clear joints and fixed body shape of the soft robot's flexible structure, traditional rigid body control theory is difficult to directly apply to its kinematic modeling and dynamic analysis.

[0003] Compared with the difficulties of structural design and control faced by current soft robots, soft organisms in nature, such as octopuses, earthworms and sea anemones, show excellent environmental adaptability and flexibility with their soft body structure and diverse movement methods. For example, the tentacles of octopuses can accurately grasp, bend freely and adapt to complex terrain, and earthworms can easily pass through narrow spaces by crawling. Inspired by the movement behaviors of these organisms, the bionic design of soft robots is expected to imitate the deformation and movement patterns of organisms and realize multimodal functions such as grasping, crawling, and swimming. Therefore, modeling and control technology for bionic soft robots is an important technical means to develop new soft robot structural design and its control system.

[0004] As a tiny mollusk, fruit fly larvae show high flexibility and adaptability in their movement behavior. They can precisely control muscle contraction to achieve efficient wave-like peristalsis and maintain stable movement performance even in complex terrain or confined spaces. In addition, they have excellent directional control ability and rapid environmental response ability, and can quickly adjust their movement trajectory under light, temperature or chemical stimulation. They are one of the important target organisms in current research on bionic soft robots.

[0005] Existing studies on the modeling and control of Drosophila larvae movement have greatly simplified their bodies. For example, the paper "Integrative neuromechanics of crawling in D. melanogaster larvae" (eLIFE, Issue 5, 2016) simplifies the multi-segment structure of Drosophila larvae into eleven mass points connected in series in one-dimensional space, and only studies the straight and backward crawling of Drosophila larvae; the paper "Modelling the mechanics of exploration in larval Drosophila" (PLOS Computational Biology, Volume 15, 2019) models the Drosophila larvae as a group of mass points connected by torsion springs in two-dimensional space, and studies the larvae's left and right deflection exploratory movement by controlling the stiffness of the torsion springs. In summary, existing research lacks the design of the body structure of fruit fly larvae in three-dimensional space and the analysis of multimodal motion control, which makes it difficult to support in-depth research on fruit fly larvae-like soft robots.

[0006] In view of the problems existing in the existing technical solutions, the present invention models the multi-segmented full soft body of a fruit fly larva in three-dimensional space based on the spring-mass method, models the muscle tension calculation node based on the muscle length-velocity-tension nonlinear model, and models the hydrostatic skeleton based on the volume constraint algorithm, and forms a fruit fly larva-like intelligent body under the above simulation modeling framework. For the fruit fly larva-like intelligent body, a muscle group bottom-level controller based on the von-Mises cyclic distributed central pattern generator is constructed to compress the control dimension from the number of muscles to the number of bottom-level controller parameters; a reinforcement learning method is used to optimize the high-level strategy network that controls the bottom-level controller parameters, so that the high-dimensional muscle system can drive the fruit fly larva-like intelligent body to produce multi-mode peristaltic motion under the coordination of the above-mentioned hierarchical motion control method. The constructed fruit fly larva-like intelligent body can provide a new material structure and control algorithm research platform for the research of bionic soft robots, and will have reference significance for the research of self-driving soft robots in the industrial and medical fields. Summary of the invention

[0007] The purpose of the present invention is to provide a muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method to solve the technical problem that the prior art cannot model and control the multi-mode motion of Drosophila larvae in three-dimensional space and is difficult to support the research of bionic soft robots. The present invention specifically provides the following technical solutions:

[0008] A muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method includes: muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control for Drosophila larvae-like intelligent bodies, wherein the body of the muscle-driven Drosophila larvae-like intelligent body is simulated by a mass-damping-spring system, and the body contains a plurality of muscle tension calculation nodes, which are regulated by a motion control system, and after controlled activation, tension is generated between two mass points, thereby changing the body shape structure. The hierarchical motion control method for Drosophila larvae-like intelligent bodies receives body shape and environmental stimulus information, calculates activation signals of each muscle tension calculation node, drives the Drosophila larvae-like intelligent body to continuously adjust the spatial structure of the mass-damping-spring system, generates contact force on the motion plane, and can make the Drosophila larvae-like intelligent body generate multi-mode peristaltic motion.

[0009] Furthermore, the mass point-damper-spring system has a mass point spatial structure set according to the structure of 11 body segments of the fruit fly larva, and the mass points are connected by a parallel model of springs and dampers, wherein the ventral plane contains 22 mass points, the dorsal surface contains 20 mass points, the head and tail body segment mass points are located on the symmetric central axis of the body on the ventral plane, and the remaining mass points are located at the four corners of the rectangular surface separating adjacent body segments. The connection relationship between the mass points of the spring and damper parallel model is as follows: the four mass points in the same body segment are connected to each other in pairs, and the adjacent body segments share the mass point structure at the body segment segment.

[0010] Furthermore, in the spring model, when the length change is the same, the tensile tension is greater than the compressive tension, so that the elastic structure of the fruit fly larvae body modeled by the spring can maintain the body shape under the support of the hydrostatic skeleton without restricting the left and right bending of the body.

[0011] Furthermore, the spatial structure distribution of the muscle tension calculation nodes is simplified according to the anatomical structure of the Drosophila larvae muscles. Each body segment includes four groups of muscle tension calculation nodes, namely, dorsal axial, abdominal axial, circumferential and oblique. The circumferential muscles can lift the ventral side of the current body segment away from the motion plane during peristalsis, thereby reducing the friction force on the body segment; the dorsal and abdominal axial muscles can pull the body segment to the adjacent volume, promoting the successive displacement of each body segment; the oblique muscles simultaneously play the role of lifting the abdomen and pulling the body segment. By regulating different muscle activation intensities, under the action of the above-mentioned muscle coordination mechanism, the Drosophila larvae-mimicking intelligent body can produce peristaltic displacement.

[0012] Furthermore, the contraction tension process generated by the muscle tension calculation node is nonlinear, and the muscle tension calculation includes the following steps: S11, traverse each muscle tension calculation node, for calculation node i, calculate its geometric length L i , there is L i =||p 1,i -p 2,i ||, where p1,i and p 2,i They represent the spatial positions at both ends of the computational node, respectively, and ‖·‖ represents the calculation of the vector's second norm; S12, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-muscle length relationship F l , there is F l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters; S13, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-contraction speed relationship F v ,have Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction velocity; S14, traverse each muscle tension calculation node, and for calculation node i, calculate its passive muscle tension F PeE ,have Where L opt Indicates the optimal muscle output length, c and A are preset parameters; S15, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle activation strength a i ,have where c a is the preset parameter, u i Represents the control signal of the muscle computing node; S16, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension F m , there is F m =a i F max F l F v +F PEE .

[0013] Furthermore, in the simulated fruit fly larvae creeping process, the mass-damping-spring system is subjected to a supporting force whose total volume of the constrained body remains unchanged, and the calculation of the volume constraint force comprises the following steps: S21, traverse all the triangular faces on the surface of the Drosophila larvae-like intelligent body, and for the i-th triangular face, calculate its normal vector toward the outside of the body have where a i 、b i 、c iare the spatial coordinates of the three vertices of the triangle, (a i -c i )×(b i -c i ) represents the cross product operation of two vectors, ‖·‖ represents the calculation of the vector bi-norm; S22, traverse all the triangular faces on the surface of the Drosophila larvae-like intelligent body, and calculate the area S of the i-th triangular face. i , there is S i =‖(a i -c i )×(b i -c i )‖ / 2; S23, calculate the volume V of the space enclosed by each triangular face, S24, calculate the volume constraint force f on particle k k ,have where α V It represents the parameter for regulating the volume constraint force, V0 represents the volume of the Drosophila larvae agent in a relaxed state, and Ω represents the triangle surface adjacent to the particle k.

[0014] Furthermore, the hierarchical motion control method for the Drosophila larvae-like intelligent body is a two-layer structure, including a bottom-level motion controller that generates periodic signals to directly control each muscle, and a high-level motion controller that senses external and body information for motion planning and regulates the parameters of the periodic signals output by the bottom-level motion controller. The control system regulates each muscle of the Drosophila larvae-like intelligent body to drive it to produce multi-mode peristaltic motion, including the following steps: S31, initializing the fruit fly larvae-like intelligent agent: Initializing the fruit fly larvae-like intelligent agent to a relaxed state, clearing various force buffers, and setting the initial position of each particle to x and the initial velocity to v; S32, calculate the force on each particle: calculate the gravity F on each particle g , volume constraint F v , spring force F s , damping force F d ; S33, calculate the contact force of the abdominal mass point: calculate the contact force F of the abdominal mass point in contact with the motion plane c , project the velocity and force of the abdominal mass point to the normal space of the contact plane<n,t,s> , where s is the normal vector perpendicular to the contact surface. If [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ||[f n ,f t ]||<μ i fs , if [v n ,v t ] is not a zero vector, then the contact force F c exist<n,t> The components of the plane are S34, calling the high-level motion controller: inputting the environment and proprioceptive perception information into the trained and optimized high-level motion controller to obtain the control parameters k and s of the low-level motion controller; S35, calling the underlying motion controller and calculating the muscle tension: based on the control parameters k and s, obtaining the activation signal a of the underlying motion controller acting on each muscle, and calculating the muscle tension F m ; S36, update the mass point velocity: based on the force on the mass point, calculate the updated mass point velocity v t+1 , there are v t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length; S37, update the mass point position: based on the mass point velocity update amount, calculate the mass point position update amount x t+1 =x t +v t+1 dt; S38, loop iterate simulation time step: if the upper limit of simulation time step is not reached, loop iterate calculation steps S32 to S37, record the particle position of each time step; if the upper limit of simulation time step is reached, stop the loop, and input the particle position sequence into the renderer to obtain the multi-mode peristaltic motion video image result of the Drosophila larvae intelligent agent.

[0015] Furthermore, the underlying motion controller periodic signal is generated by a von-mises cyclic distribution: Where mod means remainder, t means simulation time length, n∈{0,1,…,10} means segment number, s∈[-1,1] controls the transmission speed of peristaltic wave between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Representing the activation intensity of each muscle group, this periodic signal can cyclically activate the muscle tension calculation nodes of each body segment in the Drosophila larvae intelligent body with different ranges, intensities and speeds, thereby generating peristaltic waves that regulate basic movement behaviors.

[0016] Furthermore, the external information sensed by the high-level motion controller is the rate of change of the stimulus field intensity of the head mass point position: Where x0 represents the spatial position of the particle on the head of the Drosophila larvae-like intelligent agent, S(x0) represents the intensity of the external stimulus field at x0. represents the rate of change of the external stimulus field intensity over time, and P represents the external information input by the controller. The perceived ontological information is the muscle length of each muscle tension calculation node of the Drosophila larvae intelligent agent at the simulation time step.

[0017] Furthermore, the high-level motion controller is constructed based on an artificial neural network, the network architecture is a transformer network, the network parameters are optimized by a proximal strategy optimization algorithm under a reinforcement learning framework, the network input is the environment and proprioceptive perception information of the fruit fly larvae-like intelligent agent at each time step within 1 to N time steps, and the network output is the control parameters of the underlying motion controller within N+1 to 2N time steps.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses a computer simulation program of fruit fly larvae as a carrier, takes the flexible motion control of fruit fly larvae-like intelligent bodies as a guide, and develops motion control strategies for soft robots based on bionic research. The present invention finely models the multi-segmented full soft body of fruit fly larvae in three-dimensional space based on the spring-mass method, models muscle tension calculation nodes in combination with muscle anatomical structure, and uses a volume constraint algorithm to simulate a hydrostatic skeleton, forming a more realistic fruit fly larvae-like intelligent body. At the same time, a muscle group bottom controller based on a von-Mises cyclically distributed central pattern generator is proposed, and a hierarchical motion control method is constructed in combination with a high-level strategy network optimized by reinforcement learning, which greatly compresses the control dimension of the high-dimensional muscle system and realizes the precise control of the multi-mode peristaltic motion of the fruit fly larvae-like intelligent body. Compared with the prior art, the present invention overcomes the shortcomings of traditional models in three-dimensional modeling and multi-mode motion control, provides new structural design and control algorithms for the research of bionic soft robots, and helps to promote the application of soft robots in the fields of industry, medicine, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0020] Figure 1 The overall flow chart of the fruit fly larvae-like intelligent agent modeling and hierarchical motion control method provided by the present invention;

[0021] Figure 2 A schematic side view of the structure of the spring-mass-damper system of the fruit fly larvae-simulating intelligent body provided by the present invention;

[0022] Figure 3 A schematic top view of the muscle computing node structure of the fruit fly larvae-like intelligent body provided by the present invention;

[0023] Figure 4 A schematic diagram of the rendering result of the motion of the fruit fly larvae-like intelligent body provided by the present invention;

[0024] Figure 5 A schematic diagram of the motion results of the fruit fly larvae-like intelligent agent provided by the present invention in a complex stimulus environment;

[0025] Figure 6 A comparison diagram of the convergence results of the optimization process of the hierarchical motion control method and the single-layer motion control method for the fruit fly larvae-simulating intelligent body provided by the present invention;

[0026] Figure 7 A schematic diagram of the morphological changes of the motion process of the fruit fly larvae-like intelligent body provided by the present invention in a complex stimulus environment;

[0027] The numbers in the figure represent the following: 1-head mass point; 2-tail mass point; 3-fifth body segment; 4-third body segment dividing plane; 5-ventral mass point; 6-dorsal mass point; 7-ventral axial muscle; 8-circumferential muscle; 9-oblique muscle; 10-dorsal axial muscle; 11-Drosophila larvae-like intelligent agent in a complex stimulus environment; 12-stimulus source; 13-motion trajectory of Drosophila larvae-like intelligent agent; 14-the weakest position of field strength in the stimulus environment; 15-the average reward value of the hierarchical motion control method changes with training iterations; 16-the average reward value of the single-layer motion control method changes with training iterations. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Example 1 like Figure 1 As shown, the present invention provides a muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method, comprising the following steps:

[0030] Step 1: Initialize the Drosophila larvae-like intelligent body, clear the force buffers, and make the Drosophila larvae-like intelligent body spring-damper connection structure and muscle calculation nodes in a relaxed state. In the initial relaxed state, the Drosophila larvae-like intelligent body mass-damper-spring system spatial structure is as follows: Figure 2 As shown, the initial position of the mass point is set as the dot position in the figure, where the initial positions of the head and tail mass points are respectively Figure 2 The positions of the head mass point 1 and the tail mass point 2 are set as shown, and the initial positions of the other ventral and dorsal mass points are respectively set as Figure 2 The mid-ventral mass point 5 and the dorsal mass point 6 are arranged at the positions shown; the spring-damper parallel connection structure is as shown Figure 2 The midline segment connection relationship is set, and its connection relationship forms the morphological structure of each segment of the fruit fly larvae, such as Figure 2 The position shown in the fifth body segment 3 is the spatial position setting of the fifth body segment of the Drosophila larvae intelligent body. Along the body axis toward the head mass point, the fourth body segment is forward, the sixth body segment is backward, and the third and fourth body segments are in the middle. Figure 2 The third body segment dividing plane shown in the third body segment dividing plane 4; muscle tension calculation node, spatial structure distribution is as follows Figure 3 As shown, based on the simplified setting of the muscle anatomy of Drosophila larvae, each body segment contains a dorsal axis (such as Figure 3 The mid-dorsal axial muscle 10 is arranged at the position shown in the spatial structure), the abdominal axial (such as Figure 3 The mid-ventral axial muscle 7 is arranged at the position shown in the spatial structure), the circumferential direction (such as Figure 3 8) and obliquely (as shown in Figure 3 A total of four groups of muscle tension calculation nodes are set at the position shown in the middle oblique muscle 9. The four groups of muscles are symmetrical along the axial plane perpendicular to the abdominal plane along the head-tail line. Except for the head and tail body segments, the muscle tension calculation nodes of all other body segments have the same spatial structure.

[0031] Step 2: Generate a stimulus field in the motion plane of the fruit fly larvae-like intelligent agent based on the mixed Gaussian model to simulate the spatial distribution of stimulus sources in the environment and provide the intelligent agent with external perception information. First, define multiple Gaussian functions in the xy plane [-10, 10] range. Each Gaussian function represents an independent stimulus source, and its position (mean), intensity (amplitude) and diffusion degree (variance) can be set according to the simulation requirements. These Gaussian functions are linearly combined according to the weights to form a mixed Gaussian model, which presents multiple peaks and valleys in space to simulate the superposition effect of multiple stimulus sources in the real environment. According to the above process, multiple stimulus source positions are randomly generated (such as Figure 5 The stimulus source 12 in the middle) and the degree of diffusion of the stimulus field generate the stimulus field strength S at each position in the plane, and obtain a complex stimulus environment. The initialized Drosophila larvae-like intelligent agent is randomly placed in the complex stimulus environment (such as Figure 5 (as shown in Figure 11, which shows a Drosophila larvae-like intelligent agent in a complex stimulus environment).

[0032] Step 3: Calculate the force according to the position and velocity of each particle, where gravity is given by the formula F g =Mg, g represents the gravitational constant; the volume constraint force is calculated by the formula Calculated, where α V represents the parameter for regulating the volume constraint force, V0 represents the volume of the Drosophila larvae agent in a relaxed state, Ω represents the triangle surface adjacent to the mass point k, represents the volume of space enclosed by each triangular face; the spring force is given by the formula Calculated, where L r represents the relaxed length of the spring, x2 and x1 are the positions of the masses at both ends of the spring, if ||x2-x1||>L r , then Otherwise k = k0; the damping force is given by Calculated.

[0033] Step 4: Calculate the contact force based on the position of the ventral particle, and project the velocity and force of the abdominal particle into the normal space of the contact plane.<n,t,s> , where s is the normal vector perpendicular to the contact surface. If [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ||[f n ,f t ]||<μ i f s , if [v n ,v t ] is not a zero vector, then the contact force F c exist<n,t> The components of the plane are

[0034] Step 5: Use the "transformer" neural network to build a high-level motion controller, and optimize the network parameters using the proximal strategy optimization algorithm under the reinforcement learning framework. Input the environment and proprioceptive information of the fruit fly larvae agent at each time step from 1 to N time steps. The perceived external information is the rate of change of the stimulus field strength of the head mass point position, which can be obtained by the formula Calculated, where x0 represents the spatial position of the particle on the head of the Drosophila larvae-like intelligent body, S(x0) represents the intensity of the external stimulus field at x0, It represents the rate of change of the intensity of the external stimulus field over time, P represents the external information input by the controller, and the perceived entity information is the muscle length of each muscle tension calculation node of the fruit fly larvae intelligent agent in the simulation time step.

[0035] Step 6: Construct a parameterized underlying motion controller based on Von-Mises distribution, where each muscle activation signal can be expressed by Calculated, where mod means remainder, t means simulation time length, n∈{0,1,…,10} means segment number, s∈[-1,1] controls the transmission speed of peristaltic wave between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Indicates the activation intensity of each muscle group. This periodic signal can activate the muscle tension calculation nodes of each body segment in the Drosophila larvae intelligent body in different ranges, intensities and speeds, thereby generating peristaltic waves that regulate basic movement behavior. m =a i F max F l F v +F PeE The output tension of the muscle tension calculation node can be calculated, where a i Indicates muscle activation signal, c a is the preset parameter, u i Represents the control signal of this muscle computation node. Where L opt Indicates the optimal output length of the muscle, c and A are preset parameters. Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction speed. l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters.

[0036] Step 7: Calculate the updated velocity v of the particle based on the force on the particle t+1 , there are v t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length;

[0037] Step 8: Calculate the particle position update x based on the particle velocity update t+1 =x t +vt+1 dt.

[0038] Step 9: If the upper limit of the simulation time step is not reached, the calculation steps from step 3 to step 8 are iterated, and the particle position of each time step is recorded; if the upper limit of the simulation time step is reached, the loop is stopped, and the particle position sequence is input into the renderer to obtain the following Figure 4 and Figure 7 The video image results of the multi-modal peristaltic motion of the Drosophila larvae-like intelligent agent are shown.

[0039] The motion trajectory of the head mass point of the Drosophila larvae-like intelligent agent in the complex stimulus environment in the above iteration process is counted, as follows: Figure 5 As shown in the motion trajectory of the Drosophila larvae-like intelligent agent 13, it can be observed that the Drosophila larvae-like intelligent agent can continuously manipulate the complex muscle structure along the direction of weakening stimulus field strength, and drive the entire soft body to reach the position with the weakest field strength in the complex stimulus environment ( Figure 5 The weakest position of the field strength in the medium stimulation environment is 14), which shows that the muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method proposed in the present invention has excellent body manipulation ability and environmental navigation adaptability, and can solve the problem of multi-mode soft peristaltic motion control in high-dimensional control space.

[0040] As the "transformer" neural network of the high-level motion controller skeleton network, it consists of an encoder and a decoder. The overall network structure includes 6 layers of encoders and 6 layers of decoders. The model latent space dimension is 512, the feedforward network dimension is 2048, and the number of attention heads is 8. The embedding layer of the encoder maps the perceived external information and ontological information to a 512-dimensional vector space, and adds position encoding generated by sine and cosine functions to retain the timing information of the sequence. The feedforward neural network is a two-layer fully connected network. The activation function uses ReLU, which increases the dimension from 512 dimensions to 2048 dimensions and then reduces the dimension back to 512 dimensions. Residual connections are added after each sublayer and layer normalization is performed. The structure of the decoder is similar to that of the encoder, but a mask mechanism is used in the self-attention mechanism. The output layer maps the output of the decoder to the control signal space through linear mapping, and uses the Softmax activation function to generate the final control signal u i .

[0041] In the parameter optimization of the high-level motion control network, the Drosophila larvae agent interacts with the environment under the current strategy to collect experience data, including the state of each time step, the actions taken, the rewards obtained, and the next state. Using the collected data, the advantage function of each time step is calculated by estimating the difference between the future cumulative reward and the baseline value. It is used to measure the goodness of an action relative to the average level. When updating the network parameters, policy gradient clipping is used to ensure that the new strategy does not deviate too far from the old strategy to prevent the performance from being unstable due to excessive policy updates. In the optimization objective, the entropy regularization term can increase the randomness of the strategy, encourage the Drosophila larvae agent to explore more action options, and avoid falling into the local optimum too early. The optimization of network parameters is a cyclic iterative process. In each round of iteration, the Drosophila larvae agent interacts with the environment according to the current strategy, collects new data samples, and then uses these samples to update the parameters of the high-level motion controller. After multiple iterations, the parameters of the high-level motion controller are gradually optimized, thereby driving the Drosophila larvae agent to achieve flexible multi-mode motion.

[0042] The hierarchical motion control method for the Drosophila larvae-like intelligent agent proposed in the present invention is replaced by a single-layer motion control method, that is, the neural network is directly used to control the muscles of the Drosophila larvae-like intelligent agent, and the reinforcement learning method is also used to optimize the neural network parameters. The optimization results are as follows: Figure 6 As shown in the figure, the average reward value of the fruit fly larvae-like intelligent agent controlled by two methods in a complex stimulus environment with the number of training iterations. The larger the average reward value, the better the environmental adaptability of the control method used. Figure 6 The average reward value of the hierarchical motion control method changes with training iterations 15 is the average reward value curve of the hierarchical motion control method proposed in the present invention, Figure 6 Figure 16 is the average reward value curve of the single-layer motion control method. It can be observed that the single-layer motion control method can never obtain more average rewards, indicating that the control method cannot complete the motion control task of the fruit fly larvae-like intelligent body in a complex stimulus environment, reflecting that the hierarchical motion control method proposed in the present invention can quickly form a motion control strategy with good performance.

[0043] Combination Figure 5 and Figure 6The results of the embodiment shown can verify that the muscle-driven Drosophila larvae-like intelligent body modeling and hierarchical motion control method proposed by the present invention effectively solves the technical problem that existing soft robots are difficult to accurately model and realize multi-mode motion control in three-dimensional space. Due to the nonlinear characteristics of material flexibility and complex deformation behavior, the structural design and control strategy of traditional soft robots are complex and it is difficult to achieve efficient motion modes. In addition, the existing motion modeling research on Drosophila larvae has mostly simplified its body structure, lacking in-depth analysis of three-dimensional structure and diversified motion control, which limits the application ability of bionic soft robots in complex environments. The present invention uses a spring-mass-damper system to finely simulate the multi-segmented full-soft body of Drosophila larvae in three-dimensional space, combines the nonlinear model of muscle length-velocity-tension to accurately calculate muscle tension, and introduces a volume constraint algorithm to simulate a hydrostatic skeleton, thereby constructing a realistic Drosophila larvae-like intelligent body. In addition, the innovative use of a central pattern generator based on von-Mises cyclic distribution as the underlying controller, combined with a high-level policy network optimized by reinforcement learning, enables hierarchical motion control of high-dimensional muscle systems, greatly compresses the control dimension, and improves the accuracy and adaptability of multi-mode peristaltic motion.

[0044] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method, characterized in that: The method comprises: modeling of a muscle-driven Drosophila larvae-like intelligent body and hierarchical motion control for the Drosophila larvae-like intelligent body, wherein the muscle-driven Drosophila larvae-like intelligent body is simulated by a mass-damper-spring system, wherein a body contains a plurality of muscle tension calculation nodes, and the muscle tension calculation nodes are regulated by a motion control system, and after controlled activation, tension is generated between two mass points, thereby changing the body morphology structure; the hierarchical motion control method for the Drosophila larvae-like intelligent body receives body morphology and environmental stimulus information, calculates activation signals of each muscle tension calculation node, drives the Drosophila larvae-like intelligent body to continuously change the body morphology structure, and generates multi-mode peristaltic motion.

2. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven fruit fly larvae-like intelligent body modeling is implemented by a mass point-damper-spring system, the spatial structure of the mass points is set according to the structure of 11 body segments of the fruit fly larvae, and in a relaxed state in which all muscles are not activated, the two-dimensional geometric shape of the abdominal mass points in contact with the moving surface projected on the moving surface is consistent with the spatial position of the posture points at both ends of adjacent body segments in the relaxed state of the real fruit fly larvae, the mass points are connected by a parallel model of nonlinear springs and damping, the connection structure between the mass points is that the mass points in the same body segment are connected to each other in pairs, and the adjacent body segments share the mass point structure at the body segment, and when the change in spring length is the same, the tensile tension is greater than the compressive tension.

3. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body is modeled, and the muscle tension calculation nodes are simplified according to the anatomical structure of the Drosophila larvae muscles. Each body segment includes four groups of muscle tension calculation nodes, namely, dorsal axial, abdominal axial, circumferential and oblique. The spatial structure distribution is symmetrical along the axial plane perpendicular to the abdominal plane along the head-tail line. Except for the head-tail body segment, the spatial structure of the muscle tension calculation nodes of all other body segments is the same.

4. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body modeling, muscle tension calculation is regulated by a hierarchical motion controller to produce a nonlinear characteristic of the contraction tension process. Specifically, when the actual length of the muscle is inconsistent with the preset optimal output length, the tension generated by the controlled contraction of the muscle will be reduced; Similarly, when the contraction speed of the muscle is inconsistent with the preset optimal processing contraction speed, the tension generated by the controlled contraction of the muscle will also decrease accordingly. Calculating muscle tension includes the following steps: S11, traverse each muscle tension calculation node, for calculation node i, calculate its geometric length L i , there is L i =||p 1,i -p 2,i ||, where p 1,i and p 2,i They represent the spatial positions at both ends of the computational node, respectively, and ‖·‖ represents the calculation of the vector's second norm; S12, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-muscle length relationship F l , there is F l =exp[c|(L i -L opt ) / L opt w| 3 ], where L opt Indicates the optimal output length of the muscle, w and c are preset parameters; S13, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension-contraction speed relationship F v ,have Where V m,i Indicates muscle contraction speed, d and b are preset parameters, V max is the maximum muscle contraction velocity; S14, traverse each muscle tension calculation node, and for calculation node i, calculate its passive muscle tension F PEE ,have Where L opt Indicates the optimal muscle output length, c and A are preset parameters; S15, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle activation strength a i ,have where c a is the preset parameter, u i Represents the control signal of the muscle computing node; S16, traverse each muscle tension calculation node, and for calculation node i, calculate its muscle tension F m , there is F m =a i F max F l F v +F PEE .

5. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1 is characterized in that: The muscle-driven Drosophila larvae-like intelligent body modeling, the process of muscle contraction changing the body morphology and structure, the support force constraining the body to remain unchanged, and the calculation of the volume constraint force include the following steps: S21, traverse all the triangular faces on the surface of the Drosophila larvae-like intelligent body, and for the i-th triangular face, calculate its normal vector toward the outside of the body have where a i 、b i 、c i are the spatial coordinates of the three vertices of the triangle, (a i -c i )×(b i -c i ) represents the cross product operation of two vectors, ‖·‖ represents the calculation of the vector bi-norm; S22, traverse all the triangular faces on the surface of the Drosophila larvae-like intelligent body, and calculate the area S of the i-th triangular face. i , there is S i =‖(a i -c i )×(b i -c i )‖ / 2; S23, calculate the volume V of the space enclosed by each triangular face, S24, calculate the volume constraint force f on particle k k ,have where α V It represents the parameter for regulating the volume constraint force, V0 represents the volume of the Drosophila larvae agent in a relaxed state, and Ω represents the triangle surface adjacent to the particle k.

6. The muscle-driven Drosophila larvae-like intelligent agent modeling and hierarchical motion control method according to claim 1, characterized in that: The hierarchical motion control method for the Drosophila larvae-like intelligent body is a double-layer structure, including a bottom-level motion controller that generates periodic signals to directly control each muscle, and a high-level motion controller that senses external and body information to perform motion planning, regulates the bottom-level motion controller, and outputs periodic signal parameters. The control system regulates each muscle of the Drosophila larvae-like intelligent body to drive it to generate multi-mode peristaltic motion, and includes the following steps: S31, initialize the fruit fly larvae-like intelligent body to a relaxed state, clear the force buffers, and set the initial position of each particle to x and the initial velocity to v; S32, calculate the gravity F of each particle g , volume constraint F v , spring force F s , damping force F d ; S33, calculate the contact force F on the abdominal mass point in contact with the motion plane c , project the velocity and force of the abdominal mass point to the normal space of the contact plane<n,t,s> , where s is the normal vector perpendicular to the contact surface. If [v n ,v t ] is a 0 vector, then the contact force F c exist<n,t> The components of the plane are ∥[f n ,f t ]∥<μ i f s , if [v n ,v t ] is not a zero vector, then the contact force F c exist<n,t> The components of the plane are S34, inputting the environment and proprioceptive perception information into the trained and optimized high-level motion controller to obtain the control parameters κ and s of the low-level motion controller; S35, based on the control parameters κ and s, obtain the activation signal α of the underlying motion controller acting on each muscle and calculate the muscle tension F m ; S36, based on the force on the particle, calculate the updated velocity v of the particle t+1 , with V t+1 =v t +M -1 (F g +F v +F s +F d +F c +F m )dt, where M represents the mass matrix of each particle, t represents the simulation time step, and dt represents the simulation time step length; S37, based on the mass point velocity update amount, calculate the mass point position update amount x t+1 =x t +v t+1 dt.

7. The hierarchical motion control method for a Drosophila larvae-like intelligent agent according to claim 6, characterized in that: The periodic signal generated directly controls the underlying motion controller of each muscle, and the periodic signal is generated by the von-mises cyclic distribution: Where mod means remainder, t means simulation time length, n∈{0,1,…,10} means segment number, s∈[-1,1] controls the transmission speed of peristaltic wave between segments, κ∈[0,1] controls the number of segments activated at the same time, and f CPG Indicates the activation intensity of each muscle group.

8. The hierarchical motion control method for a fruit fly larvae-like intelligent agent according to claim 6, characterized in that: The high-level motion controller that outputs periodic signal parameters senses the external information as the rate of change of the stimulus field intensity of the head mass point position: Where x0 represents the spatial position of the particle on the head of the Drosophila larvae-like intelligent agent, S(x0) represents the intensity of the external stimulus field at x0. It represents the rate of change of the intensity of the external stimulus field over time, P represents the external information input by the controller, and the perceived entity information is the muscle length of each muscle tension calculation node of the fruit fly larvae intelligent agent in the simulation time step.

9. The hierarchical motion control method for a Drosophila larvae-like intelligent agent according to claim 6, characterized in that: The high-level motion controller that outputs periodic signal parameters is constructed based on an artificial neural network. The network architecture is a transformer network. The network parameters are optimized by a proximal strategy optimization algorithm under a reinforcement learning framework. The network input is the environment and proprioceptive perception information of a fruit fly larvae-like intelligent agent at each time step within 1 to N time steps. The network output is the control parameters of the underlying motion controller within N+1 to 2N time steps.

Images