Humanoid robot group dance control method and system

Abstract

The application discloses a kind of humanoid robot group dance control method and system, it is related to robot technical field, comprising: the initialization of module is completed after the start of several robot subsystems;Receive choreography instruction, generate reference trajectory vector containing humanoid robot movement and its own posture;Extract environmental features to complete positioning under world coordinate system, output humanoid robot posture estimation, realize multi-machine state data interaction;Quote trajectory checking method based on physical constraint threshold to check robot movement planning trajectory;Output a set of multi-machine cooperative trajectory set, so that all humanoid robots synchronize tracking instruction, present group dance action.The application is provided with reference trajectory vector adapted to the control requirement of 21 degrees of freedom of humanoid robot, realizes the smooth transition of action, also provided with full-duplex communication architecture based on TCP protocol, according to network round trip delay adjusts data transmission amount, both reduces the dependence of distributed system on communication quality, also avoids the communication bottleneck of centralized system.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a method and system for controlling group dance of humanoid robots. Background Technology

[0002] Humanoid robot group dance control technology is at the forefront of the intersection of robotics, multi-agent cooperative control, embodied intelligence, and machine vision. This technology aims to enable multiple humanoid robots to collaborate in complex, dynamic, and artistically expressive group dance performances. Currently, the mainstream technical approaches to realizing robot group dance can be divided into three categories: First, centralized control: relying on a central control system for global planning and command distribution; second, distributed control: using distributed algorithms (such as GHS) to generate a minimum spanning tree for task allocation, or establishing a communication network among robots to share state information; and third, biomimetic learning control: utilizing reinforcement learning, imitation learning, and other methods to enable robots to autonomously learn collaborative strategies through interaction with the environment. While these technical approaches have led to significant progress in humanoid robot group dance control technology, achieving highly reliable and artistically expressive humanoid robot group dances in practical applications still faces three core challenges:

[0003] 1. Limitations of System Architecture: The bottleneck of centralized systems and the fragility of distributed systems make it difficult for existing architectures to achieve a good balance between global consistency and local adaptability. The performance limit of centralized control systems (such as CN119658699A) is limited by the computing power of the master node and there is a risk of single point of failure, resulting in poor system robustness. While pure distributed systems (such as CN120046944A) can avoid single point of failure, their collaborative efficiency is highly dependent on the real-time performance and reliability of the communication network. In group dance applications, network latency or packet loss can easily lead to formation chaos or loss of synchronization. In addition, the safety speed control method described in CN115026829A is extremely dependent on the accurate and real-time acquisition of the positions of other robots. Once communication delays cause information updates to be delayed, the calculated "safe" strategy may fail, leading to the risk of collision.

[0004] 2. Insufficient intelligence in collaborative decision-making: rigidity of pre-programming and lag in online planning. Many existing solutions rely on pre-programmed action libraries or decision-making based on fixed rules. These methods lack the flexibility to cope with dynamic environments. When encountering sudden interference or needing to perform impromptu, the entire system cannot adjust quickly. On the other hand, although optimization-based online planning methods (such as the heavy-load path planning of CN120686690A) can replan, their computational complexity increases sharply with the number of robots, making it difficult to meet the real-time requirements of performance and unable to fundamentally solve the problem of real-time, online, and collaborative decision-making in dynamic environments.

[0005] 3. The burden of communication and computing: The contradiction between massive data sharing and real-time processing. Group dance requires robots to continuously share a large amount of data, including their own state (position, posture), action intentions and environmental perception information. This puts a huge pressure on communication bandwidth and computing resources. For resource-constrained embedded robot platforms, processing dense sensor data (such as vision and LiDAR point clouds) and performing complex collaborative algorithm calculations can easily lead to control delays.

[0006] In summary, existing humanoid robot swarm dance control technologies have significant shortcomings in terms of system architecture, collaborative intelligence, and communication and computing efficiency. These shortcomings collectively restrict the ability of large-scale humanoid robot swarms to perform smooth, reliable, and highly expressive dance performances in real, dynamic environments.

[0007] Based on this, a method and system for controlling group dance of humanoid robots is now provided, which can eliminate the drawbacks of existing technical solutions. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for controlling group dance of humanoid robots, so as to solve the problems of the prior art in terms of system architecture, collaborative intelligence and communication computing efficiency.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A method for controlling group dance of humanoid robots, specifically including the following steps:

[0011] Step S1: After several robot subsystems are started, module initialization is completed. Microsecond-level clock synchronization with the central server is achieved through the IEEE 1588 PTP protocol, a globally unified spatiotemporal reference is established, a compact dance state protocol is loaded, and the exclusive transmission time slot in the communication cycle is calculated based on its own ID.

[0012] Step S2: Based on the global unified spatiotemporal reference, the robot subsystem receives the dance choreography instructions issued by the central server, parses the motion index, formation displacement vector and music beat information, and generates a reference trajectory vector containing the humanoid robot's motion and its own posture.

[0013] Step S3: The robot subsystem extracts environmental features through binocular vision sensors and combines the triangulation principle and PnP algorithm to complete the localization in the world coordinate system. It collects data through sensors and outputs the humanoid robot posture estimate after fusion by the extended Kalman filter algorithm. It establishes a full-duplex communication architecture based on the TCP protocol to realize the interaction of multi-machine status data.

[0014] Step S4: Based on multi-machine state data and reference trajectory vector, the feasibility of the robot motion planning trajectory is verified in real time using a trajectory verification method based on physical constraint thresholds. If the constraints are met, proceed to step S5; otherwise, return to step S2 to request replanning.

[0015] Step S5: Output a set of multi-machine collaborative trajectories, map the robot motion planning trajectory into physical control signals, and send them to the control output modules of each humanoid robot in real time through a TCP protocol long connection, so that all humanoid robots can synchronously track the instructions based on a globally unified spatiotemporal reference, presenting neat and uniform, stable group dance movements.

[0016] Further, step S2 specifically includes:

[0017] Step S21: Define the reference trajectory vector generated by the robot subsystem as a four-dimensional physical quantity to adapt to the 21 degrees of freedom control requirements of the humanoid robot. The four dimensions specifically include:

[0018] The linear velocity reference of the robot body is used to drive the humanoid robot's torso to move in the horizontal plane and achieve formation changes. The linear velocity reference of the robot body is defined as follows: ,in, As a reference for the linear velocity of the machine body, for Directional reference speed, for Directional reference speed, Transpose of a vector;

[0019] The body angular velocity reference is used to control the rate of change of the heading angle of the humanoid robot's torso, enabling rotation and orientation adjustments during dance. The body angular velocity reference is defined as follows: ;

[0020] 21 motor desired positions, used to cover key joints throughout the body to determine specific dance postures, are defined as follows: The key joints throughout the body include 6 hip joints, 2 knee joints, 4 ankle joints, 4 shoulder joints, 2 elbow joints, and 3 head joints;

[0021] The expected speeds of 21 motors are used as feedforward signals to describe the explosive force and pace of the action. These 21 expected speeds are defined as follows: ;

[0022] Step S22: The robot subsystem receives the motion index and formation displacement vector from the central server, and calculates the motion execution progress, i.e., the motion execution phase, based on the currently synchronized global time and music beat information. The calculation formula is as follows: ,in, For the action execution phase, The current global time. For the start time of the music. For music speed, The number of beats at the start of the movement. The number of beats for the duration of the action. For modulo operation;

[0023] Regarding motor data Utilizing action to execute phase B-spline interpolation is performed in the pre-stored motion library to extract the 21 joint motor angles and angular velocities corresponding to the current millisecond.

[0024] Regarding body data The formation displacement vector is decomposed into real-time linear velocity and angular velocity to ensure that the humanoid robot reaches the target position when the dance movement ends.

[0025] Step S23: Combine the reference trajectory vector at a frequency of 1kHz. Send to the control output module.

[0026] Furthermore, in step S3, the robot subsystem extracts environmental features using a binocular vision sensor and combines the triangulation principle with the PnP algorithm to complete positioning in the world coordinate system, specifically including:

[0027] During the initialization phase of several robot subsystems, the binocular vision sensor is calibrated in advance to obtain its intrinsic parameter matrix, lens distortion coefficient, and extrinsic parameters of the right camera relative to the left camera.

[0028] During the operation of several humanoid robots, images from the left and right cameras are acquired in real time. Several environmental feature points are extracted, and the pixel coordinate correspondence of the same environmental feature point on the left and right camera images is calculated. For each pair of successfully matched environmental feature points, the difference in the horizontal coordinates of the environmental feature points on the left and right camera images is calculated. Based on the principle of triangulation, the three-dimensional coordinates of the environmental feature point in the left camera coordinate system are calculated. The three-dimensional coordinates of the environmental feature points in the camera coordinate system are matched with the corresponding three-dimensional coordinates in the world coordinate system in the pre-constructed environmental map. The pose of the current binocular vision sensor relative to the world coordinate system is calculated using the PnP algorithm, thereby obtaining the absolute position of the corresponding humanoid robot in the world coordinate system.

[0029] Further, in step S3, the humanoid robot pose estimate is output after data is collected by sensors and fused using an extended Kalman filter algorithm, specifically including:

[0030] Based on several sensors inside the humanoid robot, including an inertial measurement unit and a motor position feedback unit, the inertial measurement unit acquires the three-axis angular velocity and three-axis acceleration of the humanoid robot's torso, and the motor position feedback unit reads the real-time angles of the motors of the 21 joints of the humanoid robot's body.

[0031] The extended Kalman filter algorithm is used for multi-source data fusion. The angular velocity collected by the inertial measurement unit is used for numerical integration to derive the prior attitude quaternion for the next moment. The acceleration measured by the inertial measurement unit is used to calculate the projection deviation of the gravity vector in its own coordinate system. By minimizing the projection deviation, the tilt angle of the prior attitude is corrected to eliminate the integral drift of the gyroscope.

[0032] Output the optimal pose estimate after eliminating integral drift, ensuring the smoothness of humanoid robot pose data during vigorous dance movements.

[0033] Furthermore, in step S3, a full-duplex communication architecture is established based on the TCP protocol to realize multi-machine status data interaction, specifically including:

[0034] Any humanoid robot is used as the sender, and the central server and other robots are used as receivers. A long connection based on the TCP protocol is established between any humanoid robot, the central server, and other robots. The sender sends its own extended state vector to the communication network. The extended state vector includes the body position, motor position, inertial measurement unit attitude, and ZMP stability index.

[0035] After receiving the data packet and verifying that the CRC is correct, the receiver immediately sends back an acknowledgment packet through the same TCP channel. The acknowledgment packet includes an acknowledgment identifier, a receiving timestamp, and the status of the central server or the corresponding robot buffer.

[0036] The sender confirms the data status based on the received acknowledgment packet and calculates the current network round-trip time using the received timestamp.

[0037] Furthermore, step S3 also includes: the humanoid robot, as the sender, dynamically adjusts its data transmission strategy based on the real-time network round-trip latency value fed back by the TCP protocol. When the real-time network round-trip latency value is less than 10 milliseconds, the sender is in a low-latency state and sends all 21 joint data. When the real-time network round-trip latency value is greater than 50 milliseconds, the sender is in a high-latency state and only sends the humanoid robot's torso center of mass and motion index.

[0038] Furthermore, step S4 specifically includes:

[0039] A trajectory verification method based on physical constraint thresholds is used to verify the robot's motion planning trajectory in real time. The trajectory verification module in the robot subsystem compares the corrected robot motion planning trajectory with the preset physical constraint thresholds to determine the feasibility of the motion planning trajectory.

[0040] If the physical constraint threshold is met, proceed to the next step;

[0041] If the physical constraint threshold is not met, step S2 is executed again. The humanoid robot requests the central server to replan the trajectory. When a new trajectory cannot be generated, it triggers the generation of a stationary step or an emergency stop safety action and initiates a rapid reconnection request until it receives synchronization feedback and resumes the dance action.

[0042] The physical constraint thresholds include kinematic thresholds, dynamic thresholds, and safety thresholds. The kinematic thresholds include the maximum permissible angular position, angular velocity, and angular acceleration of each joint motor. The dynamic thresholds include the maximum output torque of each joint motor and the zero-torque point stability margin. The safety thresholds include the minimum collision avoidance safety distance when several humanoid robots work together.

[0043] Further, step S5 specifically includes:

[0044] Output a set of multi-robot cooperative trajectories including several robot motion planning trajectories. The robot motion planning trajectory of any humanoid robot includes its own action sequence, relative position constraints with adjacent humanoid robots, and phase constraints with music beat information.

[0045] The robot subsystem maps the above-mentioned robot motion planning trajectory into physical control signals, uses the linear velocity and angular velocity of the body as navigation commands to drive the overall movement of the humanoid robot, and uses the desired posture quaternion, desired three-axis angular velocity, and desired angular position and desired angular velocity of the motor as servo commands, and sends them to the control output module of each humanoid robot in real time through a TCP protocol long connection.

[0046] Several humanoid robots synchronously track and control the output module to output multidimensional state commands, presenting a uniform, non-interfering and stable group dance movement in physical space;

[0047] The robot motion planning trajectory includes a body motion reference trajectory, an inertial attitude reference trajectory, and a joint space reference trajectory. The body motion reference trajectory includes the body linear velocity and body angular velocity after S-curve acceleration and deceleration planning. The inertial attitude reference trajectory includes the desired attitude quaternion and desired three-axis angular velocity matching the dance movement. The joint space reference trajectory includes the desired angular position and desired angular velocity of all joint motors in the body.

[0048] A humanoid robot group dance control system, applied to a humanoid robot group dance control method, includes:

[0049] The central server is used to issue dance choreography instructions, motion indexes, and formation displacement vectors, and achieves global clock synchronization through the IEEE 1588PTP protocol;

[0050] A pre-stored motion library is used to store dance motion template data;

[0051] The communication network adopts a TCP protocol architecture to realize data interaction between the central server and several robot subsystems;

[0052] The robot subsystem receives dance choreography instructions from the central server. Through the collaborative work of its integrated multi-modules, it achieves functions such as reference trajectory generation, localization and attitude estimation, multi-machine state interaction, and trajectory feasibility verification. It maps compliant robot motion planning trajectories into physical control signals, driving its 21 joint motors to execute dance movements synchronously. This ensures that it can cooperate with other robots under a globally unified spatiotemporal reference to achieve group dance performances.

[0053] Furthermore, the robotic subsystem includes:

[0054] The reference trajectory input module is used to receive dance choreography instructions and parse them to generate reference trajectory vectors;

[0055] The multi-machine communication module is used to enable data communication with other humanoid robots and the central server;

[0056] The pose estimation module is used to collect data from several sensors and calculate the pose of the humanoid robot in its own coordinate system.

[0057] The positioning module is used to extract environmental features through binocular vision sensors and calculate the absolute position of the humanoid robot in the world coordinate system by combining the triangulation principle and the PnP algorithm.

[0058] The data processing module is used to calculate whether the robot's motion reference trajectory meets the preset requirements based on the current posture of several humanoid robots and their pose and reference trajectory in the world coordinate system.

[0059] The control output module is used to output multi-dimensional state commands for the humanoid robot based on the current input data, driving the humanoid robot to perform synchronized dance movements;

[0060] The trajectory verification module is used to verify the robot's motion planning trajectory in real time by using a trajectory verification method based on physical constraint thresholds.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] 1. This invention achieves microsecond-level clock synchronization through the IEEE 1588 PTP protocol, establishes a globally unified spatiotemporal reference, solves the problem of motion loss due to clock deviation in traditional group control, ensures the time consistency of multiple robots in complex dance movements, and makes the group performance present a higher degree of uniformity.

[0063] 2. This invention sets up a reference trajectory vector covering four physical quantities, which is adapted to the 21 degrees of freedom control requirements of humanoid robots, realizing smooth transition of movements and explosive force control. At the same time, the trajectory verification method based on physical constraint thresholds performs real-time verification from multiple dimensions of kinematics, dynamics and safety distance, reducing the risk of collision.

[0064] 3. This invention sets up a full-duplex communication architecture based on the TCP protocol, which adaptively adjusts the data transmission volume according to the network round-trip delay, thus solving the dependence of distributed systems on communication quality and avoiding the communication bottleneck of centralized systems.

[0065] 4. This invention integrates binocular visual positioning, inertial measurement unit data, and extended Kalman filter algorithm to achieve positioning and attitude estimation in the world coordinate system. This enables the robot to maintain positioning stability even in dynamic and disturbed environments. Combined with the real-time trajectory replanning function, it allows the humanoid robot to respond quickly to environmental changes. Attached Figure Description

[0066] Figure 1 A schematic diagram illustrating the steps of a method for controlling a group dance of humanoid robots.

[0067] Figure 2 This is a flowchart illustrating a method for controlling a group dance of humanoid robots.

[0068] Figure 3 This is a schematic diagram of a control system for a group dance of humanoid robots.

[0069] Figure label annotations: Central server 10, pre-stored motion library 20, communication network 30, robot subsystem 40, reference trajectory input module 41, multi-machine communication module 42, pose estimation module 43, positioning module 44, data processing module 45, control output module 46, trajectory verification module 47. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0071] Considering that multi-robot collaboration is key to realizing group dance performances, and its core lies in solving the closed-loop problem of "perception-multi-robot communication-cooperative decision-control", the present invention includes the following three aspects:

[0072] Distributed perception and communication architecture: This invention establishes a low-latency, highly reliable communication network 30 for a group of robots. Each robot not only needs to know its own precise state, but also needs to share key information with its teammates, such as its own position, action stage, and even understanding of the music beat. According to the performance requirements, this invention adopts a full-duplex communication architecture based on the TCP protocol to ensure that instructions and state synchronization can be transmitted quickly and accurately among the group.

[0073] Collaborative strategy of centralized planning and distributed execution: This invention provides a centralized planner, namely the central server 10, which is responsible for generating global formation change sequences, role assignments and keyframe actions based on the music to ensure overall artistic expression. At the humanoid robot execution layer, each humanoid robot runs a distributed strategy. This strategy not only receives central instructions, but also makes fine adjustments based on local perception (such as identifying neighbors through vision or UWB) to achieve accurate formation maintenance and collision avoidance, thereby taking into account both global consistency and local adaptability.

[0074] Cooperative control based on reinforcement learning: Multi-agent (robot subsystem 40) reinforcement learning is used to train cooperative strategies. The reward function is designed to include both individual performance and group performance. Individual rewards are similar to those for single-robot dance, focusing on the stability, grace, and rhythmic synchronization of the individual's movements. Group rewards include formation consistency rewards, relative position rewards, and collision avoidance penalties. Through this design, the agents will spontaneously exhibit cooperative behaviors during the learning process, working together to present a perfect group dance.

[0075] The details are as follows:

[0076] Example 1

[0077] In this embodiment, as Figure 1 and Figure 2 As shown, a method for controlling a group dance of humanoid robots specifically includes the following steps:

[0078] Step S1: After several robot subsystems 40 start up and complete module initialization, they synchronize with the central server 10 at the microsecond level via the IEEE 1588 PTP protocol (Precision Time Protocol) to establish a globally unified spatiotemporal reference (unified time axis). ), load the Compact Dance State Protocol (CDSP) and calculate the communication cycle based on its own ID (Robot ID). Dedicated transmission slots within ( );

[0079] Step S2: Based on the global unified spatiotemporal reference, the robot subsystem 40 receives the dance choreography instructions issued by the central server 10, parses the motion index, formation displacement vector and music beat information, and generates a reference trajectory vector containing the humanoid robot's motion and its own posture.

[0080] Step S3: Robot subsystem 40 extracts environmental features through binocular vision sensors and completes localization in the world coordinate system by combining the triangulation principle and PnP algorithm. It collects data through sensors and outputs humanoid robot posture estimation after fusion by extended Kalman filter algorithm. It establishes a full-duplex communication architecture based on TCP protocol to realize multi-machine status data interaction.

[0081] Step S4: Based on multi-machine state data and reference trajectory vector, the feasibility of the robot motion planning trajectory is verified in real time using a trajectory verification method based on physical constraint thresholds. If the constraints are met, proceed to step S5; otherwise, return to step S2 to request replanning.

[0082] Step S5: Output a set of multi-machine collaborative trajectories, map the robot motion planning trajectory into physical control signals, and send them to the control output module 46 of each humanoid robot in real time through a TCP protocol long connection, so that all humanoid robots can synchronously track the instructions based on a globally unified spatiotemporal reference and present a neat, uniform, and stable group dance movement.

[0083] Specifically, the Compact Dance State Protocol (CDSP) is a lightweight communication protocol designed for dancing robots, containing timestamps, robot IDs, pose data blocks, etc., and can be exemplified as 43 bytes:

[0084] Packet header (12 bytes):

[0085] CD 53 10 01 / / Magic text 0xCD53, version 1.0, keyframe;

[0086] A0 1F 4E 5C / / Timestamp: 1623453600ms;

[0087] 01 00 / / Serial Number: 1;

[0088] 03 / / Dancer ID: 3;

[0089] 01 / / Number of data blocks: 1;

[0090] Attitude data block (31 bytes):

[0091] 01 00 1B 00 / / Block type = attitude, length = 27 bytes;

[0092] 00 00 7F FF 00 00 00 00 00 00 / / Joint 0: Hip (quaternion 1,0,0,0);

[0093] 01 00 5A 82 00 00 5A 82 00 00 / / Joint 1: Left shoulder (0.707,0,0.707,0);

[0094] 02 00 5A 82 00 00 A6 7D 00 00 / / Joint 2: Right shoulder (0.707,0,-0.707,0);

[0095] Specifically, dedicated transmission slots ( The formula for calculating ) is: ,in, The total number of time slots, For modulo operations, a dedicated transmission time slot is required. This is used to avoid communication conflicts and to achieve time-division multiple access communication.

[0096] Among them, such as Figure 2 As shown, step S2 specifically includes:

[0097] Step S21: Define the reference trajectory vector generated by robot subsystem 40 as a four-dimensional physical quantity to adapt to the 21-degree-of-freedom control requirements of the humanoid robot. The reference trajectory does not include foot landing point information, but directly drives the humanoid robot's movement through the robot's torso velocity and joint states. The four dimensions specifically include:

[0098] The Body Linear Velocity is used to drive the humanoid robot's torso (floating base) to move in the horizontal plane, enabling formation changes (such as transforming a triangle into a circle). The Body Linear Velocity is defined as follows: The unit is ,in, As a reference for the linear velocity of the machine body, for Directional reference speed, for Directional reference speed, Transpose of a vector;

[0099] Body Angular Velocity, used to control the heading angle of the humanoid robot's torso. The rate of change is used to realize rotational and orientation adjustments in dance, defining the body's angular velocity reference as... The unit is ;

[0100] 21 desired motor positions are used to cover key joints throughout the body to determine specific dance postures (such as raising arms, squatting, bending over, etc.). These 21 desired motor positions are defined as follows: The unit is The key joints throughout the body include 6 hip joints, 2 knee joints, 4 ankle joints, 4 shoulder joints, 2 elbow joints, and 3 head joints;

[0101] The 21 motor velocities are used as feedforward signals to describe the explosiveness and tempo of a movement. For example, this value increases when performing a rapid kick. The 21 motor velocities are defined as follows: The unit is ;

[0102] Step S22: The robot subsystem 40 receives the action index (Action ID) and formation vector (Formation Vector) from the central server 10. Based on the currently synchronized global time and music beat information, it calculates the action execution progress, i.e., the action execution phase. The calculation formula is as follows: ,in, For the action execution phase, normalize to an interval within, This is the current global time, in seconds. This is the start time of the music, in seconds. Musical tempo, measured in beats per minute. The number of beats at the start of the movement. The number of beats for the duration of the action. To perform the modulo operation, ensure the result is within the range [0,1);

[0103] Regarding motor data Utilizing action to execute phase B-spline interpolation is performed in the pre-stored motion library 20 (MotionLibrary) to extract the 21 joint motor angles and angular velocities corresponding to the current millisecond;

[0104] Regarding body data The formation displacement vector is decomposed into real-time linear velocity. , and angular velocity This ensures that the humanoid robot reaches the target position at the end of its dance moves;

[0105] Step S23: Combine the reference trajectory vector at a frequency of 1kHz. Send to control output module 46;

[0106] Among them, such as Figure 2 As shown, in step S3, the robot subsystem 40 extracts environmental features using a binocular vision sensor and combines the triangulation principle with the PnP algorithm to complete localization in the world coordinate system, specifically including:

[0107] During the initialization phase of several robot subsystems 40, the binocular vision sensor (located on the humanoid robot body) is pre-calibrated to obtain its intrinsic parameter matrix, lens distortion coefficients, and extrinsic parameters of the right camera relative to the left camera (including rotation matrix). Translation vector The magnitude of the translation vector is the length of the binocular baseline;

[0108] During the operation of several humanoid robots, images from the left and right cameras are acquired in real time. Several environmental feature points (such as ORB and SIFT features) are extracted, and the pixel coordinate correspondence of the same environmental feature point in the left and right camera images is calculated. For each pair of successfully matched environmental feature points, the difference in the horizontal coordinates of the environmental feature points in the left and right camera images is calculated. Based on the principle of triangulation, the three-dimensional coordinates of the environmental feature point in the left camera coordinate system are calculated. Its depth coordinates It is determined by the following relationship: ,in, The length of the binocular baseline. Parallax is the difference in the horizontal coordinates of environmental feature points on the left and right camera images. Given the camera focal length, the 3D coordinates of the aforementioned environmental feature points in the camera coordinate system are matched with the corresponding 3D coordinates in the world coordinate system of a pre-built environmental map (the environmental map is a pre-built static map containing several feature points with known world coordinates; the map construction process can be completed before the performance using SLAM technology or manual measurement, and stored in the central server 10). The pose of the current binocular vision sensor relative to the world coordinate system is calculated using the PnP algorithm (solving for a perspective N point), and this pose is represented as a rotation matrix. and a translation vector Therefore, the absolute position of the corresponding humanoid robot in the world coordinate system is obtained, which is the translation vector. ;

[0109] Among them, such as Figure 2 As shown, in step S3, data is collected by sensors and fused using an extended Kalman filter algorithm to output the humanoid robot's pose estimate, specifically including:

[0110] Based on several sensors inside the humanoid robot, including an inertial measurement unit (IMU) and a motor position feedback unit, the three-axis angular velocity and three-axis acceleration of the humanoid robot's torso are obtained through the inertial measurement unit, and the real-time angles of the motors of the 21 joints of the humanoid robot are read through the motor position feedback unit.

[0111] The Extended Kalman Filter (EKF) algorithm is used for multi-source data fusion. The angular velocity collected by the inertial measurement unit is numerically integrated to derive the prior attitude quaternion for the next time step. The acceleration measured by the inertial measurement unit is used to calculate the projection deviation of the gravity vector in its own coordinate system. By minimizing the projection deviation, the tilt angle of the prior attitude is corrected to eliminate the integral drift of the gyroscope.

[0112] Output the optimal pose estimate after eliminating integral drift, ensuring the smoothness of the humanoid robot's pose data during vigorous dance movements;

[0113] Among them, such as Figure 2 As shown, in step S3, a full-duplex communication architecture is established based on the TCP protocol to realize multi-machine status data interaction. This operation abandons the unreliable UDP broadcast mode to ensure that in highly dynamic dance, every control command and status data can be accurately delivered and receive real-time feedback. Specifically, it includes:

[0114] Any humanoid robot is used as the sender, and the central server 10 and other robots are used as receivers. A long connection based on the TCP protocol is established between any humanoid robot, the central server 10, and other robots. The sender sends its extended state vector to the communication network 30. The extended state vector includes the body position, motor position, inertial measurement unit attitude, and ZMP stability index. The ZMP stability index is used to evaluate the dynamic stability of the humanoid robot in motion. In this invention, the ZMP stability index is calculated from the data of the inertial measurement unit and the foot force sensor and is used for trajectory verification and safety control. The extended state vector is shared among multiple robots and is used for cooperative control and anomaly detection.

[0115] After receiving the data packet and verifying that the CRC checksum is correct, the receiver immediately sends back an acknowledgment packet (Application ACK) through the same TCP channel. The acknowledgment packet includes an acknowledgment identifier (ACK ID) and a reception timestamp. The central server 10 or the corresponding robot buffer status, the acknowledgment identifier (ACK ID) strictly corresponds to the sequence number of the acknowledged data packet, used to uniquely identify the data packet that has been successfully received and processed by the receiver, and the reception timestamp ( The data buffer status records the precise timestamp of when the receiver successfully receives the data packet at the network interface layer. This timestamp is generated based on a global clock synchronized by a precision time protocol. The buffer status is fed back in coded form to reflect the real-time status of the receiver's data buffer, which is used to characterize the receiver's current data processing pressure. Its encoding may include the remaining buffer capacity, queue length, or predefined status level (such as "idle", "normal", "busy", "about to overflow").

[0116] The sender confirms the data status based on the received acknowledgment packet and calculates the current network round-trip latency using the received timestamp. The humanoid robot, acting as the sender, dynamically adjusts its data transmission strategy based on the real-time network round-trip latency value fed back by the TCP protocol. When the real-time network round-trip latency value is less than 10 milliseconds, the sender is in a low-latency state and sends all 21 joint data. When the real-time network round-trip latency value is greater than 50 milliseconds, the sender is in a high-latency state and only sends the humanoid robot's torso center of mass and motion index.

[0117] Among them, such as Figure 2 As shown, step S4 specifically includes:

[0118] The trajectory verification method based on physical constraint thresholds is used to verify the robot motion planning trajectory in real time. The trajectory verification module 47 in the robot subsystem 40 compares the corrected robot motion planning trajectory with the preset physical constraint thresholds to determine the feasibility of the motion planning trajectory.

[0119] If the physical constraint threshold is met, proceed to the next step;

[0120] If the physical constraint threshold is not met, step S2 is executed again. The humanoid robot requests the central server 10 to replan the trajectory. When a new trajectory cannot be generated, it triggers the generation of a stationary stepping or emergency stop safety action and initiates a rapid reconnection request until it receives synchronization feedback and resumes the dance action.

[0121] The physical constraint thresholds include kinematic thresholds, dynamic thresholds, and safety thresholds. Kinematic thresholds include the maximum permissible angular position, angular velocity, and angular acceleration of each joint motor. Dynamic thresholds include the maximum output torque of each joint motor and the zero torque point stability margin. Safety thresholds include the minimum collision avoidance safety distance when several humanoid robots work together. The above thresholds can be dynamically adjusted according to actual needs, expert experience, and industry theories, and can also be adjusted according to the robot model.

[0122] Among them, such as Figure 2 As shown, step S5 specifically includes:

[0123] Output a set of multi-machine cooperative trajectories that includes several robot motion planning trajectories. Any humanoid robot Robot motion planning trajectory It includes its own action sequence, relative position constraints with adjacent humanoid robots, and phase constraints with musical beat information;

[0124] Robot subsystem 40 maps the above-mentioned robot motion planning trajectory into physical control signals, and converts the linear velocity of the robot body into physical control signals. With body angular velocity As navigation commands driving the overall movement of the humanoid robot, the desired pose quaternion is used. Desired triaxial angular velocity With respect to the desired angular position of the motor Desired angular velocity As servo commands, they are sent in real time to the control output modules 46 of each humanoid robot via a long TCP connection;

[0125] Several humanoid robots synchronously track and control the output module 46 to output multi-dimensional state commands, presenting a uniform, non-interfering and stable group dance movement in physical space.

[0126] The robot motion planning trajectory includes the body motion reference trajectory, the inertial attitude reference trajectory, and the joint space reference trajectory. The body motion reference trajectory includes the linear velocity and angular velocity of the body after S-curve acceleration and deceleration planning, ensuring that the acceleration and deceleration process of the humanoid robot is smooth when rapidly changing formations, avoiding inertial instability caused by sudden stops and starts. The inertial attitude reference trajectory includes the desired posture quaternion and desired three-axis angular velocity matched with the dance movements. The joint space reference trajectory includes the desired angular position and desired angular velocity of all joint motors in the body, used to eliminate motor response lag and improve the explosive power and synchronization of the movements.

[0127] Specifically, this invention provides a method for controlling the dance of a group of humanoid robots. Multiple humanoid robots initialize their respective modules to obtain their pose and current posture in the world coordinate system, as well as the input of a target reference trajectory. Simultaneously, a multi-robot communication module 42 establishes a connection for data sharing, and the system determines and corrects each robot's preset motion trajectory, thereby controlling each robot's individual movements to achieve a unified dance. Therefore, this invention can achieve autonomous, real-time, and robust collaboration of a robot group without relying on a powerful central processing unit or high-quality communication network. This method effectively solves the bottleneck problem of centralized control, the communication dependency problem of distributed systems, and improves the collaborative intelligence level of the group in dynamic environments, thus promoting the development of humanoid robot swarm technology.

[0128] Example 2

[0129] In this embodiment, as Figure 3As shown, the present invention also provides a humanoid robot group dance control system, applied to a humanoid robot group dance control method, comprising:

[0130] The central server 10 is used to issue dance choreography instructions, motion indexes, and formation displacement vectors. It achieves global clock synchronization via the IEEE 1588 PTP protocol. As the global control hub, the central server 10 is responsible for unified scheduling, instruction issuance, and clock synchronization, ensuring the global consistency of the group dance. During the initialization phase, it loads a preset dance choreography scheme, including a preset motion library index mapping table, formation change sequences, and music beat information. It establishes a TCP connection session pool with all robot subsystems 40, assigns unique communication identifiers, and records the corresponding ID, communication status, and initial connection information of each humanoid robot. During the dance instruction distribution phase, it issues initial dance choreography instructions to all humanoid robots, including target motion indexes and initial formation displacement vectors. The system uses movement vectors (such as the initial position of humanoid robot 1 (0,0), the initial position of humanoid robot 2 (1m,0), etc.) and music start timestamps. During the dance, it dynamically issues formation update instructions, action switching indexes, and beat correction parameters based on the preset dance choreography scheme or real-time adjustment needs. During the execution of dance instructions, it receives extended state vectors uploaded by each robot subsystem 40 in real time, analyzes the data, and judges whether there are abnormalities such as action step loss or communication delay exceeding the threshold. If an abnormality is detected, it issues an emergency adjustment instruction to the humanoid robot and simultaneously issues formation fine-tuning instructions to other humanoid robots to avoid overall chaos.

[0131] The pre-stored motion library 20 is used to store dance motion template data. The pre-stored motion library 20 is stored inside the central server 10 and is also connected to the robot subsystem built into each humanoid robot through the network. The pre-stored motion template stores key frame data of 21 joints according to motion type. Each motion template includes: joint angle sequence, joint angular velocity sequence, posture quaternion sequence, etc. A motion index and parameter mapping relationship is established so that each motion index is associated with the corresponding continuous beat number, applicable formation type, and physical constraint adaptation parameters. The pre-stored motion template can be recorded by a motion capture system (existing technology) to record real-life dance, and then stored after inverse kinematics analysis into joint angle sequence; or it can be manually designed by dance choreography software, and then stored after interpolation to generate smooth trajectory; or it can be trained in a simulation environment by reinforcement learning algorithm to generate optimized motion sequences and then stored.

[0132] The communication network 30 adopts the TCP protocol architecture to realize data interaction between the central server 10 and several robot subsystems 40. During the initialization phase, the robot subsystem 40 initiates a connection request to the central server 10 as the sender, carrying its own ID, hardware version and other identity information. After verifying the identity, the central server 10 establishes a TCP long connection and enables the TCP NODELAY algorithm to ensure the timely transmission of small data packets and reduce communication delay. The humanoid robots establish point-to-point TCP connections based on a preset neighbor list to form a distributed communication network.

[0133] Robot subsystem 40 is stored inside the corresponding humanoid robot and is used to receive dance choreography instructions issued by the central server 10. Through the collaborative work of its integrated multi-modules, it realizes the functions of reference trajectory generation, localization and attitude estimation, multi-machine state interaction, and trajectory feasibility verification. It maps the compliant robot motion planning trajectory into physical control signals, drives its 21 joint motors to execute dance movements synchronously, and ensures that it can cooperate with other robots under a globally unified spatiotemporal reference to realize group dance performance.

[0134] Among them, such as Figure 3 As shown, the robot subsystem 40 includes:

[0135] The reference trajectory input module 41 is used to receive dance choreography instructions and parse them to generate reference trajectory vectors. It adds data validity identifiers and timestamps to the reference trajectory vectors, sends the vectors to the data processing module 45 and the control output module 46, and caches them in the local trajectory buffer for subsequent verification and backtracking.

[0136] The multi-machine communication module 42 is used to realize data communication with other humanoid robots and the central server 10;

[0137] The pose estimation module 43 is used to collect data through several sensors and calculate the pose of the humanoid robot in its own coordinate system.

[0138] The positioning module 44 is used to extract environmental features through a binocular vision sensor and calculate the absolute position of the humanoid robot in the world coordinate system by combining the triangulation principle and the PnP algorithm.

[0139] Data processing module 45 is used to calculate whether the robot motion reference trajectory meets the preset requirements based on the current posture of several humanoid robots and their pose and reference trajectory in the world coordinate system.

[0140] The control output module 46 is used to output multi-dimensional state commands of the humanoid robot based on the current input data, and drive the humanoid robot to perform synchronized dance movements;

[0141] The trajectory verification module 47 is used to apply a trajectory verification method based on physical constraint thresholds to perform real-time verification of the robot's motion planning trajectory.

[0142] In summary, this invention addresses the complex task of humanoid robot group dance, which is characterized by high real-time performance, high collaboration, and high dynamism. It proposes a control method for humanoid robot group dance centered on real-time multi-machine data sharing and fusion. This method enables dynamic and online optimization of the group strategy during the performance, fundamentally solving the problems of data lag and adjustment failure faced by existing technologies in dynamic environments. By driving the emergence of swarm intelligence through continuous data closed-loop, it achieves significant improvements in real-time performance, adaptability, and robustness.

[0143] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for controlling group dance of humanoid robots, characterized in that, Specifically, the following steps are included: Step S1: After several robot subsystems are started, module initialization is completed. Microsecond-level clock synchronization with the central server is achieved through the IEEE 1588 PTP protocol, a globally unified spatiotemporal reference is established, a compact dance state protocol is loaded, and the exclusive transmission time slot in the communication cycle is calculated based on its own ID. Step S2: Based on the global unified spatiotemporal reference, the robot subsystem receives the dance choreography instructions issued by the central server, parses the motion index, formation displacement vector and music beat information, and generates a reference trajectory vector containing the humanoid robot's motion and its own posture. Step S3: The robot subsystem extracts environmental features through binocular vision sensors and combines the triangulation principle and PnP algorithm to complete the localization in the world coordinate system. It collects data through sensors and outputs the humanoid robot posture estimate after fusion by the extended Kalman filter algorithm. It establishes a full-duplex communication architecture based on the TCP protocol to realize the interaction of multi-machine status data. Step S4: Based on multi-machine state data and reference trajectory vector, the feasibility of the robot motion planning trajectory is verified in real time using a trajectory verification method based on physical constraint thresholds. If the constraints are met, proceed to step S5; otherwise, return to step S2 to request replanning. Step S5: Output a set of multi-machine collaborative trajectories, map the robot motion planning trajectory into physical control signals, and send them to the control output modules of each humanoid robot in real time through a TCP protocol long connection, so that all humanoid robots can synchronously track the instructions based on a globally unified spatiotemporal reference, presenting neat and uniform, stable group dance movements. Specifically, step S2 includes: Step S21: Define the reference trajectory vector generated by the robot subsystem as a four-dimensional physical quantity to adapt to the 21-degree-of-freedom control requirements of the humanoid robot. The four dimensions specifically include: The linear velocity reference of the robot body is used to drive the humanoid robot's torso to move in the horizontal plane and achieve formation changes. The linear velocity reference of the robot body is defined as follows: ,in, As a reference for the linear velocity of the machine body, for Directional reference speed, for Directional reference speed, Transpose of a vector; The body angular velocity reference is used to control the rate of change of the heading angle of the humanoid robot's torso, enabling rotation and orientation adjustments during dance. The body angular velocity reference is defined as follows: ; 21 motor desired positions, used to cover key joints throughout the body to determine specific dance postures, are defined as follows: The key joints throughout the body include 6 hip joints, 2 knee joints, 4 ankle joints, 4 shoulder joints, 2 elbow joints, and 3 head joints; The expected speeds of 21 motors are used as feedforward signals to describe the explosive force and pace of the action. These 21 expected speeds are defined as follows: ; Step S22: The robot subsystem receives the motion index and formation displacement vector sent by the central server, and calculates the motion execution progress, i.e. the motion execution phase, based on the currently synchronized global time and music beat information. Step S23: Combine the reference trajectory vector at a frequency of 1kHz. Send to the control output module.

2. The method for controlling a group dance of humanoid robots according to claim 1, characterized in that, The formula for calculating the action execution progress, i.e., the action execution phase, in step S22 is as follows: ,in, For the action execution phase, The current global time. For the start time of the music. For music speed, The number of beats at the start of the movement. The number of beats for the duration of the action. For modulo operation; Regarding motor data Utilizing action to execute phase B-spline interpolation is performed in the pre-stored motion library to extract the 21 joint motor angles and angular velocities corresponding to the current millisecond. Regarding body data The formation displacement vector is decomposed into real-time linear velocity and angular velocity to ensure that the humanoid robot reaches the target position when the dance movement ends.

3. The method for controlling a group dance of humanoid robots according to claim 1, characterized in that, In step S3, the robot subsystem extracts environmental features using a binocular vision sensor and combines the triangulation principle with the PnP algorithm to complete localization in the world coordinate system, specifically including: During the initialization phase of several robot subsystems, the binocular vision sensor is calibrated in advance to obtain its intrinsic parameter matrix, lens distortion coefficient, and extrinsic parameters of the right camera relative to the left camera. During the operation of several humanoid robots, images from the left and right cameras are acquired in real time. Several environmental feature points are extracted, and the pixel coordinate correspondence of the same environmental feature point on the left and right camera images is calculated. For each pair of successfully matched environmental feature points, the difference in the horizontal coordinates of the environmental feature points on the left and right camera images is calculated. Based on the principle of triangulation, the three-dimensional coordinates of the environmental feature point in the left camera coordinate system are calculated. The three-dimensional coordinates of the environmental feature points in the camera coordinate system are matched with the corresponding three-dimensional coordinates in the world coordinate system in the pre-constructed environmental map. The pose of the current binocular vision sensor relative to the world coordinate system is calculated using the PnP algorithm, thereby obtaining the absolute position of the corresponding humanoid robot in the world coordinate system.

4. The method for controlling a group dance of humanoid robots according to claim 3, characterized in that, In step S3, data is collected by sensors and fused using an extended Kalman filter algorithm to output a humanoid robot pose estimate, specifically including: Based on several sensors inside the humanoid robot, including an inertial measurement unit and a motor position feedback unit, the inertial measurement unit acquires the three-axis angular velocity and three-axis acceleration of the humanoid robot's torso, and the motor position feedback unit reads the real-time angles of the motors of the 21 joints of the humanoid robot's body. The extended Kalman filter algorithm is used for multi-source data fusion. The angular velocity collected by the inertial measurement unit is used for numerical integration to derive the prior attitude quaternion for the next moment. The acceleration measured by the inertial measurement unit is used to calculate the projection deviation of the gravity vector in its own coordinate system. By minimizing the projection deviation, the tilt angle of the prior attitude is corrected to eliminate the integral drift of the gyroscope. Output the optimal pose estimate after eliminating integral drift, ensuring the smoothness of humanoid robot pose data during vigorous dance movements.

5. The method for controlling a group dance of humanoid robots according to claim 4, characterized in that, In step S3, a full-duplex communication architecture is established based on the TCP protocol to realize multi-machine status data interaction, specifically including: Any humanoid robot is used as the sender, and the central server and other robots are used as receivers. A long connection based on the TCP protocol is established between any humanoid robot, the central server, and other robots. The sender sends its own extended state vector to the communication network. The extended state vector includes the body position, motor position, inertial measurement unit attitude, and ZMP stability index. After receiving the data packet and verifying that the CRC is correct, the receiver immediately sends back an acknowledgment packet through the same TCP channel. The acknowledgment packet includes an acknowledgment identifier, a receiving timestamp, and the status of the central server or the corresponding robot buffer. The sender confirms the data status based on the received acknowledgment packet and calculates the current network round-trip time using the received timestamp.

6. The method for controlling a group dance of humanoid robots according to claim 5, characterized in that, Step S3 further includes: the humanoid robot, as the sender, dynamically adjusts its data transmission strategy based on the real-time network round-trip latency value fed back by the TCP protocol. When the real-time network round-trip latency value is less than 10 milliseconds, the sender is in a low-latency state and sends all 21 joint data. When the real-time network round-trip latency value is greater than 50 milliseconds, the sender is in a high-latency state and only sends the humanoid robot's torso center of mass and motion index.

7. The method for controlling a group dance of humanoid robots according to claim 1, characterized in that, Step S4 specifically includes: A trajectory verification method based on physical constraint thresholds is used to verify the robot's motion planning trajectory in real time. The trajectory verification module in the robot subsystem compares the corrected robot motion planning trajectory with the preset physical constraint thresholds to determine the feasibility of the motion planning trajectory. If the physical constraint threshold is met, proceed to the next step; If the physical constraint threshold is not met, step S2 is executed again. The humanoid robot requests the central server to replan the trajectory. When a new trajectory cannot be generated, it triggers the generation of a stationary step or an emergency stop safety action and initiates a rapid reconnection request until it receives synchronization feedback and resumes the dance action. The physical constraint thresholds include kinematic thresholds, dynamic thresholds, and safety thresholds. The kinematic thresholds include the maximum permissible angular position, angular velocity, and angular acceleration of each joint motor. The dynamic thresholds include the maximum output torque of each joint motor and the zero-torque point stability margin. The safety thresholds include the minimum collision avoidance safety distance when several humanoid robots work together.

8. The method for controlling a group dance of humanoid robots according to claim 1, characterized in that, Step S5 specifically includes: Output a set of multi-robot cooperative trajectories including several robot motion planning trajectories. The robot motion planning trajectory of any humanoid robot includes its own action sequence, relative position constraints with adjacent humanoid robots, and phase constraints with music beat information. The robot subsystem maps the above-mentioned robot motion planning trajectory into physical control signals, uses the linear velocity and angular velocity of the body as navigation commands to drive the overall movement of the humanoid robot, and uses the desired posture quaternion, desired three-axis angular velocity, and desired angular position and desired angular velocity of the motor as servo commands, and sends them to the control output module of each humanoid robot in real time through a TCP protocol long connection. Several humanoid robots synchronously track and control the output module to output multidimensional state commands, presenting a uniform, non-interfering and stable group dance movement in physical space; The robot motion planning trajectory includes a body motion reference trajectory, an inertial attitude reference trajectory, and a joint space reference trajectory. The body motion reference trajectory includes the body linear velocity and body angular velocity after S-curve acceleration and deceleration planning. The inertial attitude reference trajectory includes the desired attitude quaternion and desired three-axis angular velocity matching the dance movement. The joint space reference trajectory includes the desired angular position and desired angular velocity of all joint motors in the body.

9. A humanoid robot group dance control system, applied to the humanoid robot group dance control method according to any one of claims 1-8, characterized in that, include: The central server is used to issue dance choreography instructions, motion indexes, and formation displacement vectors, and achieves global clock synchronization through the IEEE 1588 PTP protocol; A pre-stored motion library is used to store dance motion template data; The communication network adopts a TCP protocol architecture to realize data interaction between the central server and several robot subsystems; The robot subsystem receives dance choreography instructions from the central server. Through the collaborative work of its integrated multi-modules, it achieves functions such as reference trajectory generation, localization and attitude estimation, multi-machine state interaction, and trajectory feasibility verification. It maps compliant robot motion planning trajectories into physical control signals, driving its 21 joint motors to execute dance movements synchronously. This ensures that it can cooperate with other robots under a globally unified spatiotemporal reference to achieve group dance performances.

10. A humanoid robot group dance control system according to claim 9, characterized in that, The robot subsystem includes: The reference trajectory input module is used to receive dance choreography instructions and parse them to generate reference trajectory vectors; The multi-machine communication module is used to enable data communication with other humanoid robots and the central server; The pose estimation module is used to collect data from several sensors and calculate the pose of the humanoid robot in its own coordinate system. The positioning module is used to extract environmental features through binocular vision sensors and calculate the absolute position of the humanoid robot in the world coordinate system by combining the triangulation principle and the PnP algorithm. The data processing module is used to calculate whether the robot's motion reference trajectory meets the preset requirements based on the current posture of several humanoid robots and their pose and reference trajectory in the world coordinate system. The control output module is used to output multi-dimensional state commands for the humanoid robot based on the current input data, driving the humanoid robot to perform synchronized dance movements; The trajectory verification module is used to verify the robot's motion planning trajectory in real time by using a trajectory verification method based on physical constraint thresholds.

Images