Humanoid robot dynamic formation control method, system, equipment, medium and product
By combining liquid neural networks and reinforcement learning networks, efficient formation control of humanoid robots in dynamic environments has been achieved, solving the problems of stability and response speed, and improving the flexibility of formation changes and adaptability to sudden obstacles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-03-10
AI Technical Summary
Traditional humanoid robot formation control methods suffer from poor stability, slow response speed, insufficient flexibility in formation changes, and difficulty in adapting to sudden obstacles in dynamic environments.
A liquid neural network is used to encode the robot's state variables, and a reinforcement learning network is combined for dynamic formation control. The network parameters are updated through a priority experience recycling mechanism to generate efficient control action commands.
It improves the stability of the formation and the response speed to sudden obstacles, enhances the flexibility of formation changes, and improves the robot's ability to work collaboratively in complex environments.
Smart Images
Figure CN121635338A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of robot control, in particular to a humanoid robot dynamic formation control method, system, device, medium and product. BACKGROUND
[0002] With the development of robot technology, the demand for collaborative work of humanoid robots in complex environments is increasing. Traditional formation control methods usually use preset path planning and fixed formation maintenance strategies, which are difficult to adapt to dynamic environments and sudden obstacles. The existing technology has the following problems: (1) poor formation stability in dynamic environments; (2) slow response speed to sudden obstacles; (3) insufficient flexibility of formation transformation. Therefore, there is an urgent need for an intelligent formation control system that can adapt to environmental changes and make autonomous decisions in real time. SUMMARY
[0003] The purpose of the present application is to provide a humanoid robot dynamic formation control method, system, device, medium and product to solve the problems of poor formation stability in dynamic environments, slow response speed to sudden obstacles and insufficient flexibility of formation transformation.
[0004] To achieve the above-mentioned purpose, the present application provides the following solutions: In a first aspect, the present application provides a humanoid robot dynamic formation control method, comprising: obtaining the current state quantity of the humanoid robot; the current state quantity includes the state of the humanoid robot itself, the state of the teammates, the environmental information and the formation parameters; encoding the current state quantity using a liquid state neural network to determine the encoded state quantity; inputting the encoded state quantity into a reinforcement learning network to determine the control action instruction at the current time, executing the control action instruction, generating the state quantity at the next time and the corresponding reward; the reinforcement learning network is a humanoid robot dynamic formation control network model; based on a priority experience recycling mechanism, updating the network parameters of the reinforcement learning network according to the current state quantity, the current time action, the reward and the state quantity at the next time to determine the trained reinforcement learning network; controlling the humanoid robot in dynamic formation according to the trained reinforcement learning network.
[0005] In a second aspect, the present application provides a humanoid robot dynamic formation control system, comprising: an environment perception module for obtaining the current state quantity of the humanoid robot; the current state quantity includes the state of the humanoid robot itself, the state of the teammates, the environmental information and the formation parameters; A liquid neural network encoder is used to encode the current state quantity using a liquid neural network to determine the encoded state quantity; The reinforcement learning decision module is used to input the encoded state variables into the reinforcement learning network, determine the control action command at the current moment, execute the control action command, and generate the state variables and corresponding rewards for the next moment; the reinforcement learning network is a humanoid robot dynamic formation control network model. The update and training module is used to update the network parameters of the reinforcement learning network based on the priority experience retrieval mechanism, according to the current state quantity, the current action, the reward, and the state quantity at the next moment, and to determine the trained reinforcement learning network. The motion execution module is used to perform dynamic formation control of the humanoid robots based on the trained reinforcement learning network.
[0006] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described humanoid robot dynamic formation control method.
[0007] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described humanoid robot dynamic formation control method.
[0008] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described humanoid robot dynamic formation control method.
[0009] According to the specific embodiments provided in this application, this application has the following technical effects: This application introduces a liquid neural network to encode the current state variables of the humanoid robot. Since the liquid neural network has very few neurons, it can greatly reduce dimensionality and computation time, thus improving dynamic stability and response speed. Compared with traditional technical solutions that use simple reinforcement learning networks, which have excessively high dimensionality, reduce computation speed and dynamic stability when the neural network becomes complex, this application introduces a liquid neural network to effectively improve formation stability in dynamic environments and response speed to sudden obstacles. On this basis, this application also introduces a reinforcement learning network, which is more flexible in the face of complex scenarios and can respond better to sudden obstacles compared to the mathematical formulas of traditional methods, further improving the response speed to sudden obstacles. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a flowchart illustrating a dynamic formation control method for humanoid robots provided in an embodiment of this application. Detailed Implementation
[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0013] To make the objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0014] like Figure 1 As shown in the figure, this application provides a method for dynamic formation control of humanoid robots, including: S1: Obtain the current state of the humanoid robot; the current state includes the humanoid robot's own state, teammate state, environmental information, and formation parameters.
[0015] S2: Encode the current state quantity using a liquid neural network to determine the encoded state quantity.
[0016] S3: Input the encoded state quantity into the reinforcement learning network, determine the control action instruction at the current moment, execute the control action instruction, and generate the state quantity and corresponding reward for the next moment; the reinforcement learning network is a humanoid robot dynamic formation control network model.
[0017] S4: Based on the priority experience retrieval mechanism, update the network parameters of the reinforcement learning network according to the current state quantity, the current action, the reward, and the state quantity at the next moment, and determine the trained reinforcement learning network.
[0018] S5: Perform dynamic formation control of the humanoid robots based on the trained reinforcement learning network.
[0019] In an exemplary embodiment, the humanoid robot's own state includes its own position, pose, joint control parameters, and velocity; the teammate's state includes the relative position distance of the teammate; the environmental information includes the distance between the humanoid robot and obstacles; the formation parameters include the ideal formation position and the actual position deviation; the actual position deviation includes the positional and angular deviations between the ideal formation and the actual formation.
[0020] In practical applications, the current state variable is represented as: in, For the current state quantity, ( ) is the first i The location of the humanoid robot For joint control quantity; For speed; The relative position and distance of teammates; The distance between the humanoid robot and the obstacle; For the first i The actual distance between the individual robot and the lead robot; Indicates the actual formation angle Angle of ideal formation deviation.
[0021] In an exemplary embodiment, S2 specifically includes: The current state quantity is input into the liquid neural network to process the spatiotemporal information of the current state quantity and extract the dynamic change features of the environment; the liquid neural network is a continuous-time recurrent neural network.
[0022] The dynamic environmental change characteristics are subjected to dimensionality reduction processing to determine the dimensionality-reduced state vector; the dimensionality-reduced state vector is the encoded state quantity.
[0023] In practical applications, Liquid Neural Networks (LNNs) are used for feature extraction and dimensionality reduction of the original state: Wherein, LNN represents a liquid neural network encoder. This is the current state variable. The state variables are those after feature extraction and dimensionality reduction.
[0024] The liquid neural network adopts a continuous-time recurrent neural network (CT-RNN) structure: in, Indicates neuron state, Represents the time constant. For connection weights, It is the sigmoid activation function. The input is time-varying. This network can effectively process the temporal information of robot motion, extract dynamic features of environmental changes, and output a dimensionality-reduced state vector. .
[0025] In one exemplary embodiment, the reinforcement learning network includes a policy network, a target policy network, a value network, and a target value network.
[0026] The encoded state variables are input into the reinforcement learning network to determine the control action instruction at the current moment, and the control action instruction is executed to generate the state variables and corresponding rewards for the next moment, specifically including: The encoded state variables are input into the policy network, and the action at the current moment is output.
[0027] The encoded state quantity and the current action are input into the value network, and the state value evaluation value of the current state-action pair is output; the state-action pair is the state quantity and action at the same time.
[0028] Determine the state quantity for the next moment based on the current state-action pair.
[0029] The state value at the next time step is input into the target policy network, and the action at the next time step is output.
[0030] The state quantity and action of the next time step are input into the target value network, and the state value evaluation value of the state-action pair at the next time step is output; the state value evaluation value of the state-action pair at the next time step is the reward corresponding to the state quantity at the next time step.
[0031] In practical applications, state variables Input the dynamic formation control network model of humanoid robots to obtain the motion quantities that control the movement of the humanoid robots. It is used to control the movement and turning of humanoid robots. The humanoid robot dynamic formation control network model is a neural network based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning algorithm.
[0032] In practical applications, updating the network parameters of the reinforcement learning network specifically includes: This reinforcement learning network is used for dynamic formation control of humanoid robots and includes Actor networks, Target Actor networks, Critic1 networks, Target Critic1 networks, Critic2 networks, and Target Critic2 networks. Although the Actor networks and Target Actor networks have the same network architecture, their inputs and outputs differ; similarly, the Critic and Target Critic networks also share the same network architecture, but there are subtle differences between them.
[0033] Specifically, the input to the Actor network is the state quantity perceived at the current moment. The output is the amount of motion sent to the robot's joint motors at this moment. The input to the Target Actor network is the state variable at the next time step. The output is the motion quantity sent to the robot's joint motors in the next moment. .
[0034] The four Critic-type networks can be divided into two parts: one part consists of Critic1 and Critic2 networks, which are identical; the other part consists of Target Critic1 and Target Critic2 networks, which are also identical. The main difference between the Critic network and the Target Critic network lies in their inputs and outputs. The input to the Critic network is the current state-action pair (…). The output of the Target Critic network is the state value assessment of the current state-action pair calculated by the neural network; while the input of the Target Critic network is the state-action pair for the next time step. The output is the state value target value based on the state-action pair at the next time step.
[0035] The network update is as follows: (1) Parameter update process of Critic1 and Critic2 networks: Calculate the state variables at the next time step using the Target Actor network. The following action ,action The calculation formula is as follows: in, For the network parameters of the Target Actor network, Represents the Target Actor network; Then, based on the target policy, smooth regularization is applied to the target action. Add noise: in, Indicates the current state quantity; Indicates the parameters of the Actor network; Represents an Actor network; Represents random noise parameters; The function represents when hour ,when hour ,besides, ; This indicates that the distribution follows a normal distribution, where -b and b represent fixed parameters, and b>0.
[0036] Next, based on the idea of a dual network, the state-action pair is calculated using the Target Critic network. State value target value State value target value The calculation formula is as follows: in, For the reward function, For the discounted rate of return, For the network parameters of Target Critic1 and Target Critic2 networks, This represents the Target Critic1 and Target Critic2 networks.
[0037] Finally, the gradient descent algorithm is used to minimize the error between the evaluated value and the target value. This updates the parameters in the Critic1 and Critic2 networks. in This represents the Target Critic1 and Target Critic2 networks. and This represents the state and action quantities at the current moment. These are the network parameters for Critic1 and Critic2 networks.
[0038] (2) Actor network parameter update process: After updating the Ctitic1 and Critic2 networks by step d, the Actor network update is initiated, and the state is calculated using the Actor network. The following action : in, For the network parameters of the Actor network, This represents the Actor network. Note that after calculating the action, no noise needs to be added, because the goal here is for the Actor network to update in the direction of the maximum value, and adding noise would be meaningless.
[0039] Calculate state-action pairs using Critic1 or Critic2 networks State value assessment State value assessment value The calculation formula is as follows, assuming the Critic1 network is used: in, These are the network parameters of the Critic1 network. This represents the Critic1 network; Finally, the gradient ascent algorithm is used to maximize the state value assessment value. This updates the parameters in the Actor network. The reason why either Critic1 or Critic2 can be used to calculate the Q-value is mainly because the purpose of the Actor network is to maximize the cumulative expected return, and there is no need to use the minimum value.
[0040] (3) Update the network parameters of Target Actor network and Target Critic network.
[0041] The TD3 algorithm uses a soft update method, where the parameter update process of the Target Actor network is as follows: The update process for the Target Critic1 and Target Critic2 networks is as follows: in The learning rate (momentum). It is usually 0.005.
[0042] Repeat S1-S4 until the model converges.
[0043] In one exemplary embodiment, rewards The reward function includes the following rewards: ① Spacing Incentive: Encourages maintaining ideal formation spacing.
[0044] ②Speed synchronization reward: Promotes speed consistency within the team.
[0045] ③ Formation Maintenance Reward: Maintain the overall formation angle.
[0046] ④ Robot collision avoidance reward: Avoid collisions between robots.
[0047] ⑤ Smoothness penalty: Reduces energy consumption and maintains smoothness of movement.
[0048] ⑥ Robot obstacle avoidance reward: Avoid collisions between the robot and environmental obstacles.
[0049] Therefore, the reward The reward function is: in, and This is a hyperparameter used to dynamically adjust the weight of rewards in different domains; N is the total number of robots in the cluster. i, j Indexing for robots, ; For ideal formation spacing, Represents the i-th robot With the navigation robot The actual distance The interval reward function is specifically expressed as follows: Ensure that all robots are spaced appropriately from the lead robot. The standard deviation is the Gaussian distribution. To guide the robot's speed, Let be the current speed of the i-th robot. The speed synchronization function is calculated as follows: Ensure all robots' speeds are synchronized with the lead robot; maintain formation using the reward function. Used to assess overall formation angle deviation. Indicates the actual formation angle Angle of ideal formation Deviation; Collision Avoidance Rewards Between Robots Used to calculate the collision risk between all robot pairs. The collision penalty coefficient for the robot. To be a very small constant to avoid division by zero, For safe distance threshold, For distance between robots; reward for smooth motion. Evaluate the average value of all robot motion changes. and Represents the joint control quantities of robot i at times t and t-1; robot obstacle avoidance reward. It is used to calculate the collision risk of all robots with obstacles. This is the penalty coefficient for colliding with obstacles. To be a very small constant to avoid division by zero, To determine the obstacle avoidance safety distance threshold, Let be the distance between robot i and the obstacle.
[0050] In an exemplary embodiment, S1-S4 are repeated to continue obtaining the dimensionality-reduced state variables at the next time step. Dimensionally reduced state quantity at the current moment Motion volume The local reward function at the previous moment As a set of empirical data Storing experience in the experience pool and using priority sampling, the priority experience retrieval mechanism is as follows: in, A quadruple selected from the target empirical data sampling probability ， It refers to the action at the current moment. It is the state quantity at the next moment. It is the state quantity at the current moment. As a reward; n Indicates the sequence number of the extracted empirical data; It is the priority of the extracted empirical data; These are preset parameters used to adjust the priority of training sample sampling; k This refers to the sequence number of the empirical data.
[0051] This application first obtains the current state of the humanoid robot, performs efficient feature encoding using a liquid neural network, inputs it into a reinforcement learning network to obtain control actions, and obtains new state variables and reward values after executing the actions, thus acquiring multiple sets of experience data throughout the formation process. A priority experience replay mechanism is used to sample and train the network model using this experience data, enabling the humanoid robot's dynamic formation control network to learn useful information more efficiently during training, thereby improving the overall learning effect. The trained network model can effectively process spatiotemporal information in dynamic environments, achieving high-quality control strategy generation. Therefore, the control actions output by the trained model are more accurate, the formation stability is higher, and the response speed is faster, effectively improving the formation performance of the humanoid robot in complex environments.
[0052] This application provides a humanoid robot dynamic formation control system, including: The environmental perception module is used to acquire the current state of the humanoid robot; the current state includes the humanoid robot's own state, the state of its teammates, environmental information, and formation parameters.
[0053] A liquid neural network encoder is used to encode the current state quantity using a liquid neural network to determine the encoded state quantity.
[0054] The reinforcement learning decision module is used to input the encoded state variables into the reinforcement learning network, determine the control action command at the current moment, execute the control action command, and generate the state variables and corresponding rewards for the next moment; the reinforcement learning network is a humanoid robot dynamic formation control network model.
[0055] The update and training module is used to update the network parameters of the reinforcement learning network based on the priority experience retrieval mechanism, according to the current state quantity, the current action, the reward, and the state quantity at the next moment, and to determine the trained reinforcement learning network.
[0056] The motion execution module is used to perform dynamic formation control of the humanoid robots based on the trained reinforcement learning network.
[0057] This application acquires environmental information and robot state variables through multimodal sensors, uses a liquid neural network for efficient feature encoding, and inputs the encoded state into a reinforcement learning network based on the TD3 algorithm to generate control commands. An innovative multi-objective cooperative reward function is designed, combined with a priority experience replay mechanism to optimize network training. An optimized formation control network model is obtained through iterative training, enabling adaptive formation control of the humanoid robot and effectively improving formation stability and response speed in complex environments.
[0058] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores data to be processed. The I / O interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal via a network connection. When the computer program is executed by the processor, it implements the above-described methods.
[0059] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0060] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0061] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0062] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0063] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0064] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0066] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A dynamic formation control method for humanoid robots, characterized by, The method comprises the following steps: acquiring a current state quantity of the humanoid robot; the current state quantity comprises a self state of the humanoid robot, a teammate state, environment information, and a formation parameter; encoding the current state quantity by using a liquid neural network to determine an encoded state quantity; inputting the encoded state quantity into a reinforcement learning network to determine a control action instruction at a current time, executing the control action instruction, generating a state quantity at a next time, and determining a reward corresponding to the state quantity at the next time; the reinforcement learning network is a humanoid robot dynamic formation control network model; updating network parameters of the reinforcement learning network based on a priority experience recycling mechanism according to the current state quantity, the action at the current time, the reward, and the state quantity at the next time to determine a trained reinforcement learning network; controlling the humanoid robot in a dynamic formation according to the trained reinforcement learning network.
2. The dynamic formation control method of humanoid robots according to claim 1, wherein, The self state of the humanoid robot comprises a self position, a pose, a joint control quantity, and a speed; the teammate state comprises a relative position distance of a teammate; the environment information comprises a distance between the humanoid robot and an obstacle; the formation parameter comprises an ideal formation position and an actual position deviation; the actual position deviation comprises a position deviation and an angle deviation of an ideal formation and an actual formation. 3.The humanoid robot dynamic formation control method of claim 1, wherein, The method of encoding the current state quantity by using the liquid neural network to determine the encoded state quantity comprises the following steps: inputting the current state quantity into the liquid neural network to process spatiotemporal information of the current state quantity and extract an environment dynamic change feature; the liquid neural network is a continuous-time recurrent neural network; performing dimension reduction processing on the environment dynamic change feature to determine a dimension-reduced state vector; the dimension-reduced state vector is the encoded state quantity. 4.The humanoid robot dynamic formation control method of claim 1, wherein, The reinforcement learning network comprises a policy network, a target policy network, a value network, and a target value network. The method of inputting the encoded state quantity into the reinforcement learning network to determine the control action instruction at the current time, executing the control action instruction, generating the state quantity at the next time, and determining the reward corresponding to the state quantity at the next time comprises the following steps: inputting the encoded state quantity into the policy network to output an action at the current time; inputting the encoded state quantity and the action at the current time into the value network to output a state value evaluation value of a state-action pair at the current time; the state-action pair is a state quantity and an action at the same time; determining a state quantity at a next time according to the state-action pair at the current time; inputting the state quantity at the next time into the target policy network to output an action at the next time; inputting the state quantity at the next time and the action at the next time into the target value network to output a state value evaluation value of a state-action pair at the next time; the state value evaluation value of the state-action pair at the next time is a reward corresponding to the state quantity at the next time.
5. The dynamic formation control method of humanoid robots according to claim 4, wherein, The reward The reward function is: wherein, and are hyperparameters; N is the total number of swarm robots; is the index of a robot, ; is the ideal formation distance, denotes the actual distance of the i th robot to the leader robot; denotes the distance reward function; is the velocity of the leader robot; is the current velocity of the i th robot; is the velocity synchronization function; is the formation keeping reward function; denotes the actual formation angle deviation from the ideal formation angle ; is the inter-robot collision avoidance reward; is the action smoothness reward; is the robot obstacle avoidance reward. 6.The humanoid robot dynamic formation control method of claim 1, wherein, The priority experience recycling mechanism comprises the following steps: wherein, is a quadruple selected from the target experience data is a probability of sampling, the quadruple at each time point being stored in the target experience data , is an action at a current time point, is a state quantity at a next time point, is a state quantity at a current time point, is a reward; n denotes a serial number of the extracted experience data; is a priority of the extracted experience data; is a preset parameter for adjusting a degree of priority sampling of the target experience data; k is an experience data serial number.
7. A dynamic formation control system for humanoid robots, characterized by, The humanoid robot dynamic formation control system executes the humanoid robot dynamic formation control method according to any one of claims 1 to 6, and the humanoid robot dynamic formation control system comprises An environment perception module is configured to acquire a current state quantity of the humanoid robot; the current state quantity comprises a self state of the humanoid robot, a teammate state, environment information, and a formation parameter; A liquid neural network encoder is configured to encode the current state quantity by using a liquid neural network to determine an encoded state quantity; A reinforcement learning decision module is configured to input the encoded state quantity into a reinforcement learning network to determine a control action instruction at a current time, execute the control action instruction, and generate a state quantity at a next time and a corresponding reward; the reinforcement learning network is a humanoid robot dynamic formation control network model; An updating and training module is configured to update network parameters of the reinforcement learning network based on a priority experience recycling mechanism according to the current state quantity, the current time action, the reward, and the state quantity at the next time to determine a trained reinforcement learning network; A motion execution module is configured to perform dynamic formation control on the humanoid robot according to the trained reinforcement learning network.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the humanoid robot dynamic formation control method in any one of claims 1-6.
9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the humanoid robot dynamic formation control method in any one of claims 1-6.
10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the humanoid robot dynamic formation control method in any one of claims 1-6.