Abnormal node detection and fault-tolerant reset method in bionic robot dance group control

Abstract

The embodiment of the application discloses a kind of abnormal node detection and fault-tolerant reset method in bionic robot dance group control, method includes: obtaining the real-time action trajectory that each bionic robot periodically uploads in dance formation;Based on the action semantic section division of current dance process to preset dance arrangement information, determine the semantic significant level of corresponding action semantic section;According to the deviation degree between the real-time action trajectory of each bionic robot and target action trajectory, in combination with the semantic significant level of the action semantic section where abnormal node is located and the formation space position corresponding to abnormal node, determine corresponding abnormal influence level;According to abnormal influence level, generate corresponding fault-tolerant reset strategy;Before reset window arrives, control normal robot associated with abnormal node to execute local collaborative compensation according to local collaborative compensation range;In reset window, action synchronization reset instruction is sent to abnormal node.The application realizes accurate abnormal detection and timely abnormal recovery.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method for abnormal node detection and fault-tolerant reset in the control of a group of bionic robots dancing. Background Technology

[0002] With the rapid development of bionic robot technology, bionic robot dance performances are widely used in large-scale artistic performances, commercial activities, and cultural tourism scenarios. Dance group control systems typically consist of multiple bionic robots forming a formation, synchronously executing complex movements according to pre-set choreography information. However, in actual performances, individual robots often exhibit abnormal movements due to communication interruptions, actuator malfunctions, sensor drift, and other reasons. If not addressed promptly, deviations at abnormal nodes can easily propagate through visual and synchronization mechanisms, causing the entire formation's movements to deviate, severely impacting the performance's visual appeal and artistic integrity.

[0003] In existing technologies, fault-tolerant methods for robot swarm control mainly include simple fault detection based on thresholds, resynchronization via consensus protocols, or global replanning. For example, some solutions directly identify faults and perform global resets based on position or posture deviations, but they ignore the artistic semantic characteristics of dance performances, resulting in inaccurate handling of anomalies during climactic movements or visual focus areas, easily causing visual interruptions or secondary disturbances. Other distributed fault-tolerant methods, while capable of local adjustments, lack dynamic evaluation of movement rhythm and visual attention to formation, failing to effectively suppress the visual diffusion effect of anomalies, and the recovery process often affects overall synchronization.

[0004] Therefore, there is an urgent need for a method that can accurately assess the impact of anomalies and perform preventative compensation and seamless reset without interrupting the performance, in order to meet the needs of high-demand dance group control scenarios. Summary of the Invention

[0005] This application provides a method for abnormal node detection and fault-tolerant reset in the control of a group of dancing bionic robots, which achieves accurate abnormal detection and timely abnormal recovery.

[0006] This application provides the following solution:

[0007] According to the first aspect, an abnormal node detection and fault-tolerant reset method is provided in the control of a group of bionic robots dancing. The method includes: acquiring the real-time motion trajectory periodically uploaded by each bionic robot in the dance formation.

[0008] Based on preset dance choreography information, the current dance process is divided into semantic segments of motion. The semantic salience level of each semantic segment is determined according to its corresponding motion content information. Based on the deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, corresponding abnormal nodes are detected. The corresponding abnormal impact level is determined by combining the semantic salience level of the semantic segment containing the abnormal node and the formation spatial position of the abnormal node. A corresponding fault-tolerant reset strategy is generated based on the abnormal impact level. This strategy includes a reset window and a local collaborative compensation range. Before the reset window is reached, the normal robots associated with the abnormal node perform local collaborative compensation according to the local collaborative compensation range to reduce the impact of the abnormal node on the overall motion synchronization of the dance formation. Within the reset window, a motion synchronization reset command is sent to the abnormal node, causing it to re-enter the dance formation synchronization control process according to the target motion rhythm.

[0009] According to one achievable method in the embodiments of this application, the real-time motion trajectory includes: limb joint angle parameters, limb movement speed parameters, motion beat phase parameters, actuator feedback parameters, and spatial coordinate parameters of the robot in the dance formation.

[0010] According to one achievable method in this application embodiment, the step of dividing the current dance process into action semantic segments based on preset dance choreography information includes: acquiring the action time sequence and formation change information in the preset dance choreography information; extracting the action trajectory change rate, limb movement energy change rate, and beat intensity change rate corresponding to each time period based on the action time sequence, and generating the action dynamic feature vector for the corresponding time period; calculating the formation spatial dispersion and formation center offset corresponding to each time period based on the formation change information, and generating the formation dynamic feature vector for the corresponding time period; performing temporal correlation analysis on the action dynamic feature vector and the formation dynamic feature vector to identify the temporal boundary points where the action change trend and the formation change trend occur synchronously; and dividing the current dance process into action semantic segments based on the identified temporal boundary points.

[0011] According to one achievable method in this application embodiment, determining the semantic salience level of the corresponding action semantic segment based on the action content information corresponding to each action semantic segment includes: extracting the action content information corresponding to each action semantic segment, wherein the action content information includes action beat parameters, action timing parameters, action trajectory parameters, and formation change parameters; constructing the action phase distribution of each robot in the corresponding action semantic segment based on the action beat parameters and action timing parameters; calculating the group visual fluctuation intensity of the corresponding action semantic segment based on the dispersion of the action phase distribution; calculating the action amplitude change intensity and formation visual attention weight of the corresponding action semantic segment based on the action trajectory parameters and formation change parameters; generating a visual salience evaluation vector of the corresponding action semantic segment based on the group visual fluctuation intensity and formation visual attention weight, and determining the semantic salience level of the corresponding action semantic segment based on the visual salience evaluation vector.

[0012] According to one achievable method in this application embodiment, the step of detecting corresponding abnormal nodes based on the degree of deviation between the real-time motion trajectory and the target motion trajectory of each bionic robot includes: extracting corresponding motion posture feature sequences, motion beat phase sequences, and trajectory space evolution sequences based on the real-time motion trajectory and the target motion trajectory, respectively; performing temporal alignment processing on the motion posture feature sequences and the target motion posture feature sequences to calculate the posture alignment deviation; performing phase synchronization analysis on the motion beat phase sequences and the target motion beat phase sequences to calculate the phase drift; calculating the trajectory consistency difference of the current robot relative to the local formation neighborhood based on the trajectory space evolution sequence; constructing a multi-dimensional motion consistency mismatch vector based on the posture alignment deviation, phase drift, and trajectory consistency difference, and determining motion anomaly indicators based on the multi-dimensional motion consistency mismatch vector; when the motion anomaly indicators continuously exceed a preset threshold within a continuous time window, determining the corresponding bionic robot as an abnormal node.

[0013] According to one achievable method in this application embodiment, determining the corresponding abnormal impact level by combining the semantic salience level of the action semantic segment where the abnormal node is located and the formation spatial position corresponding to the abnormal node includes: constructing a local visual influence domain for the corresponding abnormal node based on the spatial coordinates of the abnormal node in the dance formation; calculating the correlation degree of the action phase deviation of each normal robot and the formation synchronization coupling strength within the local visual influence domain to generate an action propagation influence factor for the corresponding abnormal node; determining the visual sensitivity coefficient of the corresponding action semantic segment based on the semantic salience level of the action semantic segment where the abnormal node is located; generating an abnormal perception risk value for the corresponding abnormal node based on the action propagation influence factor, the visual sensitivity coefficient, and the positional weight of the abnormal node relative to the formation visual center region; and determining the corresponding abnormal impact level based on the abnormal perception risk value.

[0014] According to one achievable method in an embodiment of this application, the normal robot associated with the abnormal node performs local collaborative compensation based on the local collaborative compensation range, including: determining the local collaborative compensation area corresponding to the abnormal node based on the spatial position and formation connection relationship of the abnormal node in the dance formation; obtaining the real-time motion beat parameters, motion phase parameters, and formation spacing parameters of each normal robot in the local collaborative compensation area; and dynamically adjusting the motion beat synchronization offset, motion amplitude compensation amount, and formation contraction coefficient of each normal robot in the local collaborative compensation area according to the abnormal impact level corresponding to the abnormal node, so as to reduce the visual diffusion degree of the abnormal node's motion deviation in the dance formation.

[0015] Based on the adjusted motion beat synchronization offset, motion amplitude compensation, and formation contraction coefficient, local collaborative compensation control parameters are generated for the corresponding normal robots. Each normal robot is controlled to perform motion synchronization enhancement and local formation reconstruction based on the corresponding local collaborative compensation control parameters in order to maintain the overall motion continuity and visual coordination of the dance formation.

[0016] According to one achievable method in the embodiments of this application, sending a motion synchronization reset command to the abnormal node, so that the abnormal node re-enters the dance formation synchronization control process according to the target motion beat, includes: sending the abnormal node target motion trajectory parameters, target motion beat parameters, and target posture synchronization parameters; controlling the abnormal node to gradually recover to the formation synchronization state according to the target motion trajectory parameters and target motion beat parameters.

[0017] According to one achievable method in an embodiment of this application, the control of the abnormal node gradually restores it to the formation synchronization state according to the target motion trajectory parameters and the target motion beat parameters, including: calculating the trajectory convergence offset based on the target motion trajectory parameters and the current real-time motion trajectory, and generating a segmented trajectory correction sequence based on the trajectory convergence offset; calculating the beat synchronization phase difference based on the target motion beat parameters and the current motion beat phase, and generating a progressive phase correction factor based on the beat synchronization phase difference; constructing a progressive motion recovery control sequence based on the segmented trajectory correction sequence and the progressive phase correction factor; updating the motion execution parameters of the abnormal node in stages according to the progressive motion recovery control sequence, so that the abnormal node gradually reduces the trajectory deviation and beat deviation within the continuous motion cycle; when the trajectory deviation and beat deviation of the abnormal node converge to within the preset synchronization interval, it is determined that the abnormal node has restored to the formation synchronization state.

[0018] According to the second aspect, an abnormal node detection and fault-tolerant reset system for a bionic robot dance troupe is provided. The system includes: a motion trajectory information acquisition unit configured to acquire real-time motion trajectories periodically uploaded by each bionic robot in the dance troupe; a semantic salience level calculation unit configured to divide the current dance process into motion semantic segments based on preset dance choreography information, and determine the semantic salience level of each motion semantic segment based on the motion content information corresponding to each motion semantic segment; and an abnormal impact level calculation unit configured to detect corresponding abnormal nodes based on the degree of deviation between the real-time motion trajectory and the target motion trajectory of each bionic robot, and to combine the abnormal node with the motion semantic segment in which it is located. The semantic saliency level and the formation spatial position corresponding to the abnormal node are used to determine the corresponding abnormal impact level; the fault-tolerant reset strategy generation unit is configured to generate a corresponding fault-tolerant reset strategy based on the abnormal impact level, the fault-tolerant reset strategy including a reset window and a local collaborative compensation range; the local collaborative compensation execution unit is configured to control the normal robot associated with the abnormal node to perform local collaborative compensation before the reset window arrives, so as to reduce the impact of the abnormal node on the overall motion synchronization of the dance formation; the motion synchronization reset control unit is configured to send a motion synchronization reset command to the abnormal node within the reset window, so that the abnormal node re-enters the dance formation synchronization control process according to the target motion beat.

[0019] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0020] This application achieves context-sensitive quantification of the impact of abnormal nodes by dividing action semantic segments and dynamically evaluating semantic saliency levels. Combined with an intelligent generation fault-tolerant reset strategy based on the abnormal impact level, local collaborative compensation is performed by associated normal robots before the reset window, effectively suppressing the visual spread and synchronization disturbance of abnormal actions, maintaining the overall movement continuity and aesthetic coordination of the dance formation. Precise action synchronization reset is performed within the low-saliency window of the semantic boundary, allowing abnormal nodes to gradually and seamlessly integrate into the formation according to the target rhythm, avoiding global restarts or visual interruptions. This method significantly improves the fault tolerance robustness and performance stability of the bionic robot dance group control system, with short recovery time and low visual perception anomaly rate. It is particularly suitable for large-scale real-time dance performance scenarios, solving the technical problem in existing technologies where a single node failure can easily lead to deviations in the entire formation, demonstrating strong practicality and innovation.

[0021] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0024] Figure 2 A flowchart of the abnormal node detection and fault-tolerant reset method in the bionic robot dance group control provided in the embodiments of this application;

[0025] Figure 3 A schematic diagram of a local collaborative compensation region provided in an embodiment of this application;

[0026] Figure 4 A structural block diagram of the abnormal node detection and fault-tolerant reset system in the bionic robot dance group control provided in the embodiments of this application;

[0027] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0029] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0031] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0032] To facilitate understanding of this application, the system architecture on which this application is based will be described first. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture may include: a bionic robot and an abnormal node detection and fault-tolerant reset system in the bionic robot dance group control located on the server side.

[0033] The system transmits the real-time motion trajectory of the bionic robot to the server-side abnormal node detection and fault-tolerant reset system in the bionic robot dance swarm control. The abnormal node detection and fault-tolerant reset system in the bionic robot dance swarm control can employ the method provided in the embodiments of this application to detect and reset abnormal nodes. The server can send local collaborative compensation instructions and motion synchronization reset instructions to the corresponding robots, which then execute the relevant operations.

[0034] Among them, the abnormal node detection and fault-tolerant reset system in the bionic robot dance group control can be set as an independent server, a server group, or a cloud server. In addition to... Figure 1 In addition to the architecture shown, the abnormal node detection and fault-tolerant reset system in the bionic robot dance group control can also be set on a computer terminal with strong computing power.

[0035] It should be understood that Figure 1 The abnormal node detection and fault-tolerant reset system in the robot and bionic robot dance swarm control shown is merely illustrative. Depending on the implementation requirements, any number of abnormal node detection and fault-tolerant reset systems for robot and bionic robot dance swarm control can be implemented.

[0036] Figure 2 This is a flowchart of an abnormal node detection and fault-tolerant reset method in the control of a bionic robot dance group provided in this application embodiment. This method can be... Figure 1 The system shown executes an abnormal node detection and fault-tolerant reset system in the control of a group of bionic robots dancing. For example... Figure 2 As shown, the method may include the following steps:

[0037] Step 201: Obtain the real-time motion trajectories periodically uploaded by each bionic robot in the dance troupe.

[0038] Step 202: Divide the current dance process into action semantic segments based on the preset dance choreography information, and determine the semantic salience level of the corresponding action semantic segment according to the action content information corresponding to each action semantic segment.

[0039] Step 203: Based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, detect the corresponding abnormal nodes, and determine the corresponding abnormal impact level by combining the semantic significance level of the action semantic segment where the abnormal node is located and the formation spatial position corresponding to the abnormal node.

[0040] Step 204: Generate a corresponding fault-tolerant reset strategy based on the level of anomaly impact. The fault-tolerant reset strategy includes a reset window and a collaborative compensation range.

[0041] Step 205: Before the reset window arrives, control the normal robots associated with the abnormal node to perform local collaborative compensation in order to reduce the impact of the abnormal node on the overall motion synchronization of the dance choreography.

[0042] Step 206: In the reset window, send an action synchronization reset command to the abnormal node so that the abnormal node can reconnect to the dance choreography synchronization control process according to the target action beat.

[0043] As can be seen from the above process, this application achieves context-sensitive quantification of the impact of abnormal nodes through action semantic segmentation and dynamic evaluation of semantic saliency levels. Combined with an intelligent generation fault-tolerant reset strategy based on the abnormal impact level, local collaborative compensation is performed by associated normal robots before the reset window, effectively suppressing the visual spread and synchronization disturbance of abnormal actions, maintaining the overall movement continuity and aesthetic coordination of the dance formation. Furthermore, precise action synchronization reset is performed within the low-saliency window of the semantic boundary, allowing abnormal nodes to gradually and seamlessly integrate into the formation according to the target rhythm, avoiding global restarts or visual interruptions. This method significantly improves the fault tolerance robustness and performance stability of the bionic robot dance group control system, with short recovery time and low visual perception anomaly rate. It is particularly suitable for large-scale real-time dance performance scenarios, solving the technical problem in existing technologies where a single node failure can easily lead to deviations in the entire formation, demonstrating strong practicality and innovation.

[0044] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments. First, step 201, namely "obtaining the real-time motion trajectory periodically uploaded by each bionic robot in the dance formation", will be described in detail with reference to the embodiments.

[0045] During the dance performance, multiple bionic robots form a dynamic formation and synchronously execute complex sequences of movements. To achieve real-time monitoring of the formation's state, the control unit periodically receives complete status information of each bionic robot's current movement from a wireless communication network. This periodic upload mechanism is typically set to occur every 50 to 200 milliseconds to ensure that the data's timeliness matches the high-frequency rhythm of the dance movements, thus providing continuous and reliable data support for subsequent semantic segment analysis and anomaly detection.

[0046] Real-time motion trajectories contain several key parameters that comprehensively reflect the robot's motion state, synchronization, and formation position. First, there are the limb joint angle parameters, which record the current rotation angle and relative position of each joint, directly reflecting whether its posture conforms to the preset dance movement requirements. Second, there are the limb motion velocity parameters, used to describe the movement speed and acceleration changes of each limb, helping to determine the smoothness of the movement and whether energy transfer is normal. Finally, the motion beat phase parameters reflect the degree of alignment between the robot's current movement and the music beat or overall formation rhythm, and are one of the core indicators for measuring synchronization.

[0047] In addition, the real-time motion trajectory also includes actuator feedback parameters and the robot's spatial coordinate parameters within the dance formation. Actuator feedback parameters encompass information such as motor current, torque, temperature, and positional errors, enabling timely detection of potential hardware faults or performance degradation. Spatial coordinate parameters precisely record the robot's position, orientation, and relative distance to neighboring robots in the three-dimensional stage coordinate system, providing spatial basis for assessing the overall formation consistency and local visual impact. These parameters collectively constitute a multi-dimensional motion state vector, allowing the control unit to comprehensively determine whether the robot is exhibiting abnormalities from multiple dimensions, including posture, rhythm, dynamics, and space. This provides a comprehensive and accurate data foundation for subsequent semantic saliency level assessment and fault-tolerant compensation decisions.

[0048] By acquiring this detailed and structured real-time motion trajectory, the present invention can achieve high-precision, real-time perception of dance choreography, significantly improving the accuracy of anomaly detection and the timeliness of fault-tolerant response.

[0049] The following describes in detail step 202, namely, "dividing the current dance process into action semantic segments based on preset dance choreography information, and determining the semantic significance level of the corresponding action semantic segment according to the action content information corresponding to each action semantic segment," with reference to the embodiments.

[0050] Segmenting the current dance process into semantic segments based on pre-set choreography information is a key technical step in achieving semantic context-sensitive anomaly detection and fault tolerance. This process can rely on simple time-interval segmentation, or it can fully consider the artistic structure and dynamic characteristics of the dance itself. Through deep correlation analysis of movements and formations, the entire dance process can be naturally segmented into semantic segments with different expressive focuses, thereby providing accurate temporal context for subsequent saliency level assessment and fault tolerance strategies.

[0051] Specifically, firstly, the control unit acquires the motion time sequence and formation change information from the preset dance choreography information. The motion time sequence records the target motion trajectory, joint angle sequence, and beat point information at each moment of the dance; the formation change information includes the formation layout, the relative positional relationships of the robots, and the timing of formation switching at each moment. This preset information serves as prior knowledge, guiding the system to understand the structured features of the current dance.

[0052] Next, the system extracts dynamic features corresponding to each time period based on the action time series, generating a motion dynamic feature vector. Specifically, for each time window, the system calculates the rate of change of the action trajectory to reflect the intensity of the spatial movement, the rate of change of limb motion energy to quantify the overall intensity change of joint drive, and the rate of change of beat intensity to capture rhythmic fluctuations. These features are fused to form the motion dynamic feature vector, which can be represented in the following form:

[0053] Motion dynamic feature vector = [trajectory change rate, kinetic energy change rate, beat intensity change rate]

[0054] Simultaneously, the system calculates the dynamic feature vector of the formation for each time period based on the formation change information. This vector mainly includes the spatial dispersion of the formation, which measures the degree of dispersion in the positional distribution between robots, and the center-of-gravity offset of the formation, which describes the stability of the entire formation's center of gravity over time. These indicators collectively characterize the visual structural features of the formation changes, and the dynamic feature vector of the formation can be represented as:

[0055] Formation dynamic feature vector = [spatial dispersion, centroid offset]

[0056] Then, the system performs a temporal correlation analysis on the dynamic feature vectors of the movements and the dynamic feature vectors of the formations. By calculating the correlation coefficient, trend synchronicity, and consistency of abrupt change points of the two vector sequences in the time dimension, the system identifies the temporal boundary points where the trends of movement change and formation change occur synchronously. These boundary points typically correspond to important turning points in the dance, such as the transition from a slow movement to a climax, or the switch from a static pose to a rapid formation change.

[0057] Let the sequence of dynamic feature vectors of the action be... (in For discrete time points, (where t is the action feature vector at time t), the sequence of formation dynamic feature vectors is: (where t is a discrete time point, (This is the dynamic feature vector of the formation at time t).

[0058] The sliding window Pearson correlation coefficient is used to measure the linear correlation between two sequences:

[0059] For window length At any moment calculate:

[0060]

[0061] in, This represents the mean value within the window. The higher the value, the more synchronized the movement and formation changes.

[0062] When calculating trend synchronicity, the first difference of each sequence is first calculated:

[0063]

[0064] Then, the cosine similarity of the trend vectors is calculated as an indicator of trend synchronicity:

[0065]

[0066] A value close to 1 indicates that the trend of movement changes is highly synchronized with the trend of formation changes.

[0067] The sliding window standard deviation mutation detection method was used to calculate the intensity of local changes in the two sequences respectively:

[0068]

[0069]

[0070] Next, calculate the consistency score for mutation points:

[0071]

[0072] in The global mean. The global mean of the sequence of dynamic feature vectors of actions. The global mean of the dynamic feature vector sequence of the formation is given. The mutation threshold, This is an indicator function (it takes the value 1 if the condition is met, and 0 otherwise). Use small positive numbers to prevent the denominator from being zero.

[0073] The final temporal boundary point score is defined as a weighted fusion of the three factors:

[0074]

[0075] in , usually take .

[0076] when When the value exceeds a preset threshold and is a local maximum, it is determined that... The temporal boundary points are the points where the trends of action change and formation change occur simultaneously, thus completing the natural division of the action semantic segment.

[0077] Finally, the current dance process is divided into semantic segments based on the identified temporal boundary points. Each semantic segment has a relatively consistent movement style, rhythm intensity, and formation characteristics, while the boundaries between segments mark significant visual and rhythmic shifts.

[0078] After segmenting the action semantic segments, the semantic salience level of each segment is determined based on the action content information corresponding to that segment. This process transforms the artistic expressiveness of dance into a quantifiable saliency indicator, enabling the system to distinguish the visual importance of different performance segments and thus assign stricter anomaly handling priorities to semantic segments with high attention.

[0079] First, the system extracts the action content information corresponding to each action semantic segment. This information includes action beat parameters, action timing parameters, action trajectory parameters, and formation change parameters. Action beat parameters record the intensity and interval of each beat point, action timing parameters describe the temporal sequence and duration of action execution, action trajectory parameters depict the movement path of the robot's limbs in space, and formation change parameters reflect the dynamic adjustments of the overall formation layout. These elements together constitute the complete data foundation for evaluating the visual importance of semantic segments.

[0080] Next, based on the action beat parameters and action timing parameters, the action phase distribution of each robot in the corresponding action semantic segment is constructed. The action progress of each robot is mapped to a unified phase space, typically normalized to 0 to 2π or 0 to 1, thus obtaining the phase distribution of the entire formation within that semantic segment. This distribution intuitively reflects the tightness of the action synchronization between robots.

[0081] Then, the intensity of group visual fluctuations for the corresponding action semantic segments is calculated based on the dispersion of the action phase distribution. Higher dispersion indicates greater differences in robot action rhythm, resulting in a stronger sense of fluctuation and dynamism in the overall visual presentation. The intensity of group visual fluctuations can be calculated using the following formula:

[0082]

[0083] in, For the number of robots, The number of sampling points. For the first Taiwan Robot Phase value at each sampling point This represents the average phase of the formation. The larger the value, the more significant the group visual fluctuation of the semantic segment.

[0084] Simultaneously, the system calculates the intensity of motion amplitude changes and the visual attention weight of the formation based on motion trajectory parameters and formation change parameters. The intensity of motion amplitude changes reflects the intensity and spatial span of limb movements, while the visual attention weight of the formation considers the ability of formation changes to attract the visual focus of the stage. The intensity of motion amplitude changes can be expressed as:

[0085]

[0086] in Let T be the trajectory position vector, and T be the duration of the corresponding action semantic segment. The formation visual attention weight is derived by considering factors such as formation dispersion, movement speed, and center offset.

[0087] Finally, a visual saliency evaluation vector for the corresponding action semantic segment is generated based on the intensity of group visual fluctuations and the formation visual attention weight, and the semantic saliency level is determined based on this vector. The visual saliency evaluation vector can be fused into:

[0088]

[0089] in These are the weighting coefficients. Visual attention weights are assigned to the formation. By weighted summation or clustering mapping of the vectors, the semantic saliency level is ultimately divided into three levels: low, medium, and high, or more fine-grained levels. High levels typically correspond to dance climaxes, rapid formation changes, or segments where visual focus is concentrated.

[0090] The semantic saliency level determined by the above method enables the anomaly detection and fault tolerance strategies to intelligently adapt to the artistic rhythm of the dance, prioritizing visual effects in high saliency segments and flexibly performing recovery operations in low saliency segments, thus significantly improving the overall artistic expressiveness and system robustness of the bionic robot dance performance.

[0091] The following describes in detail step 203, namely, "based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, detect the corresponding abnormal node, and determine the corresponding abnormal impact level by combining the semantic significance level of the action semantic segment where the abnormal node is located and the formation spatial position corresponding to the abnormal node".

[0092] In this invention, abnormal nodes are determined by the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory. Abnormal points can be determined directly by the distance difference between the real-time motion trajectory and the target motion trajectory, or by multimodal fusion analysis from three dimensions: posture, beat, and spatial trajectory. Based on the real-time motion trajectory and the target motion trajectory, corresponding motion posture feature sequences, motion beat phase sequences, and trajectory spatial evolution sequences are extracted respectively. The motion posture feature sequences and the target motion posture feature sequences are temporally aligned, and the posture alignment deviation is calculated. The motion beat phase sequences and the target motion beat phase sequences are phase synchronized, and the phase drift is calculated. Based on the trajectory spatial evolution sequence, the trajectory consistency difference of the current robot relative to its local formation neighborhood is calculated. A multidimensional motion consistency mismatch vector is constructed based on the posture alignment deviation, phase drift, and trajectory consistency difference, and an action anomaly index is determined based on the multidimensional motion consistency mismatch vector. When the action anomaly index continuously exceeds a preset threshold within a continuous time window, the corresponding bionic robot is determined to be an abnormal node.

[0093] Specifically, firstly, based on the real-time motion trajectory and the target motion trajectory, the system extracts the corresponding motion posture feature sequence, motion beat phase sequence, and trajectory space evolution sequence, respectively. The motion posture feature sequence mainly consists of joint angle sequences and relative limb positions, reflecting the robot's current posture; the motion beat phase sequence records the alignment progress between the motion and the music beat; and the trajectory space evolution sequence describes the robot's continuous motion path in three-dimensional space. These sequences together constitute the data foundation for multi-dimensional deviation analysis.

[0094] Next, the action posture feature sequence and the target action posture feature sequence are temporally aligned, and the posture alignment deviation is calculated. The system uses a dynamic time warping algorithm to align the two sequences on the time axis to eliminate the influence of slight velocity differences. After alignment, the posture alignment deviation is calculated using the following formula:

[0095]

[0096] in and These are the real-time and target pose feature vectors, respectively, where K is the number of feature dimensions. A larger value indicates a more significant attitude deviation.

[0097] Then, phase synchronization analysis is performed on the action beat phase sequence and the target action beat phase sequence to calculate the phase drift. The system normalizes the phase values ​​to the 0 to 2π interval and compares the current phase... Phase with target The difference between the two values ​​yields the drift amount, calculated using the following formula:

[0098]

[0099] This indicator can effectively capture whether the robot's movement rhythm is lagging or ahead, and is a key parameter for measuring beat synchronization.

[0100] Simultaneously, based on the trajectory spatial evolution sequence, the system calculates the trajectory consistency difference of the current robot relative to its local formation neighborhood. The system first determines the set of neighboring robots for the anomalous candidate robot (typically the M closest robots in spatial distance), and then calculates the difference between its trajectory and the average trajectory of the neighborhood, expressed by the formula:

[0101]

[0102] in Let T be the current robot trajectory vector. m Let m be the trajectory of the neighboring robot. This reflects the degree of significant deviation of the robot in local formation.

[0103] Finally, a multi-dimensional motion consistency mismatch vector is constructed based on attitude alignment deviation, phase drift, and trajectory consistency differences, and motion anomaly indicators are determined based on this vector. The multi-dimensional mismatch vector can be represented as:

[0104]

[0105] in These are weighting coefficients. The abnormal behavior index is obtained by norm calculation or weighted fusion of vectors. When this indicator continuously exceeds a preset threshold within a continuous time window, such as 3 to 5 sampling periods, the corresponding bionic robot is determined to be an abnormal node.

[0106] Through the above-mentioned multi-dimensional and time-stable anomaly detection method, the present invention can identify potential abnormal robots early and accurately, avoiding misjudgment based on a single indicator.

[0107] Next, the semantic salience level of the action semantic segment where the abnormal node is located and the formation space position of the abnormal node are combined to determine the corresponding abnormal impact level.

[0108] First, the system constructs a local visual influence domain for each anomalous node based on its spatial coordinates within the dance choreography. Centered on the current position of the anomalous robot, a radius of 2 to 5 meters is extended outwards, forming an elliptical or circular influence area. This area covers the vicinity of nearby robots where the audience's vision is most easily disturbed by the anomalous movements, providing a spatial basis for subsequent propagation impact analysis.

[0109] Then, the system calculates the correlation degree of motion phase deviation and the formation synchronization coupling strength of each normal robot within the local visual influence domain, generating the motion propagation influence factor for the corresponding abnormal node. The motion phase deviation correlation degree measures the mutual influence of rhythm deviations between the abnormal node and neighboring robots, while the formation synchronization coupling strength reflects the tightness of position and posture linkage. The motion propagation influence factor is obtained by fusing these two indicators, and the calculation formula is as follows:

[0110]

[0111] in To affect the number of normal robots within the domain, For the first Phase deviation of the robot For association weights, For formation coupling strength, and This is the balance coefficient. The higher the value, the easier it is for abnormal actions to spread through local formations, affecting the overall visual effect.

[0112] Next, based on the semantic salience level of the action semantic segment containing the abnormal node, the visual sensitivity coefficient of the corresponding action semantic segment is determined. Semantic segments with high salience levels (such as climax actions or rapid formation changes) correspond to higher visual sensitivity coefficients, while those with low salience levels have lower coefficients. This coefficient is directly inherited from the aforementioned semantic salience level assessment results and serves as an important weighting factor for moderating the impact of abnormalities.

[0113] Then, based on the motion propagation influence factor, visual sensitivity coefficient, and the positional weight of the abnormal node relative to the formation's visual center region, an anomaly perception risk value for the corresponding abnormal node is generated. The system performs a weighted calculation based on these three factors, which can be expressed by the formula:

[0114]

[0115] in Visual sensitivity coefficient, As a positional weight, the closer the abnormal node is to the visual center of the stage or the direction of the audience's main line of sight, the higher its positional weight. The higher the value, the greater the perceived severity of the anomaly within the current semantic segment and spatial location.

[0116] Finally, the corresponding impact level of the anomaly is determined based on the perceived risk value. The system typically maps risk values ​​to three or more fine-grained levels: low, medium, and high. For example, a risk value below threshold T1 is considered low, between T1 and T2 is medium, and above T2 is high. A high impact level will trigger a higher reset priority and a wider scope of collaborative compensation.

[0117] By using the above-mentioned multi-factor fusion method for determining the level of abnormal impact, this invention can achieve intelligent hierarchical evaluation of abnormal nodes, making the fault tolerance strategy highly adaptable to the artistic rhythm and visual focus of the dance performance. While ensuring the continuity of the performance, it minimizes the abnormal impact perceived by the audience, significantly improving the practicality and artistic expressiveness of the bionic robot dance group control system.

[0118] The following describes step 204, namely "generating a corresponding fault-tolerant reset strategy based on the level of abnormal impact, wherein the fault-tolerant reset strategy includes a reset window and a collaborative compensation range," in detail with reference to the embodiments.

[0119] This step intelligently formulates differentiated handling plans based on the severity of the anomaly, ensuring that optimal fault-tolerant measures are taken under different levels of severity and artistic scenarios, so as to both quickly suppress the spread of the anomaly and maintain the continuity and visual aesthetics of the dance performance to the greatest extent.

[0120] The system first maps specific fault-tolerant reset strategies based on the severity of the anomaly. Low-impact levels correspond to lightweight handling strategies, medium-impact levels employ moderate-intensity intervention, while high-impact levels trigger the highest-priority emergency fault-tolerant scheme. This hierarchical mapping ensures the targeted and efficient nature of the strategies, enabling the rational allocation of system resources.

[0121] The fault-tolerant reset strategy mainly includes: reset window and collaborative compensation range.

[0122] The reset window refers to the optimal time interval that allows for the synchronous reset of abnormal nodes. The system combines the salience level and boundary information of the current action semantic segment, selecting a low-salience period with minimal visual fluctuations as the reset window, typically located within several beat cycles before and after the semantic segment boundary. This window design ensures that the reset operation does not occur during the dance climax or visual focus, thus avoiding noticeable interruptions for the audience.

[0123] The collaborative compensation range defines the set of normal robots that need to participate in local compensation and the intensity of their impact. For anomalies with high impact levels, the compensation range expands to a wider range of neighboring robots and applies a stronger compensation magnitude; for anomalies with low impact levels, the compensation range is limited to a small number of adjacent robots to reduce interference with the overall formation. The size of the compensation range can be dynamically adjusted according to the anomaly's impact level.

[0124] Preferably, the fault-tolerant reset strategy also includes a reset priority that determines the processing order of the abnormal node within the current control cycle. High-priority abnormalities will immediately enter the compensation and reset process, while low-priority abnormalities can be appropriately delayed to avoid straining system computing resources by processing multiple abnormalities simultaneously. The reset priority can be quantified using the following formula:

[0125]

[0126] in This represents the level of impact of the anomaly. R is the risk value for anomaly perception. max This represents the maximum possible value of the anomaly-perceived risk value. and This is the adjustment coefficient.

[0127] Through the above-mentioned fault-tolerant reset strategy generation mechanism based on the anomaly impact level, this invention achieves an intelligent closed loop from evaluation to execution.

[0128] The following describes step 205, namely, "before the reset window arrives, control the normal robot associated with the abnormal node to perform local collaborative compensation in order to reduce the impact of the abnormal node on the overall synchronization of the dance choreography," in detail with reference to the embodiments.

[0129] Before the reset window arrives, controlling the normal robots associated with the abnormal node to perform local collaborative compensation is an important technical means to achieve preventive anomaly suppression. This step proactively intervenes before the abnormal node is officially reset. Through the collaborative adjustment of neighboring normal robots, it reduces the visual and synchronization impact of abnormal movements on the entire dance choreography in advance, avoids the spread of abnormal deviations in highly significant semantic segments, and thus ensures the continuity and artistic integrity of the performance.

[0130] The system first determines the corresponding local collaborative compensation area based on the spatial location of the abnormal node in the dance formation and the formation connection relationship. Figure 3 This is a schematic diagram of the local collaborative compensation region provided in an embodiment of this application. Centered on the current position of the malfunctioning robot, and combined with the current formation topology (such as connecting edges between adjacent robots), a dynamic compensation region is automatically defined. This region typically includes several normal robots closest to the malfunctioning node, ensuring that the compensation actions have spatial relevance and visual continuity.

[0131] Subsequently, the system acquires the real-time motion beat parameters, motion phase parameters, and formation spacing parameters of each normal robot within the local collaborative compensation area. This real-time data reflects the current motion state of the neighboring robots, providing precise input for compensation adjustments. The motion beat parameters and phase parameters are used for rhythm-level alignment, while the formation spacing parameters are used for spatial structure optimization.

[0132] Based on the impact level of the abnormal node, the system dynamically adjusts the synchronization offset of the motion beat, the compensation amount of motion amplitude, and the formation contraction coefficient of each normal robot within the area to reduce the visual diffusion of abnormal motion deviations in the dance formation. The adjustment intensity is positively correlated with the impact level: the compensation magnitude is greater at higher impact levels. The adjustment formula can be expressed as:

[0133]

[0134]

[0135]

[0136] in This is the beat synchronization offset. This is the phase shift amount. D is the amplitude compensation amount. pose For attitude alignment deviation, This is the formation contraction coefficient. The difference between its trajectory and the average trajectory of its neighborhood. The level of abnormal impact is: These are the adjustment coefficients. Through these formulas, the system achieves adaptive and refined compensation control.

[0137] Based on the adjusted parameters, the system generates local cooperative compensation control parameters for the corresponding normal robot. These control parameters are issued to each normal robot in the form of commands, including the corrected joint velocity target, phase offset command, and position fine-tuning vector.

[0138] Finally, each normal robot is controlled to perform motion synchronization enhancement and local formation reconstruction based on corresponding local cooperative compensation control parameters. Motion synchronization enhancement mainly achieves rhythmic consistency enhancement through beat and phase fine-tuning, while local formation reconstruction reshapes local visual balance through spacing adjustment and slight posture shifts. Throughout the compensation process, all adjustments are kept within a small range that is not easily perceptible to human vision, ensuring that the overall dance style does not change significantly, while significantly reducing the visual prominence of abnormal nodes.

[0139] The following describes step 206, namely, "In the reset window, send an action synchronization reset command to the abnormal node so that the abnormal node can reconnect to the dance choreography synchronization control process according to the target action beat," in detail with reference to the embodiments.

[0140] This step is performed within a pre-determined reset window. Through precise parameter distribution and progressive recovery control, it ensures that the abnormal robot can smoothly and seamlessly reintegrate into the overall formation, avoiding sudden changes in movement that could cause visual interruptions to the dance performance and maximizing the artistic continuity and synchronicity of the performance.

[0141] The system first sends a motion synchronization reset command to the abnormal node, and simultaneously issues three types of key target parameters: target motion trajectory parameters, target motion beat parameters, and target posture synchronization parameters. The target motion trajectory parameters contain the expected spatial motion paths of each joint of the robot over several future cycles; the target motion beat parameters provide a phase target that is precisely aligned with the current dance music and the overall rhythm of the formation; and the target posture synchronization parameters provide the precise angles and relative positions of key posture nodes. These parameters are all derived from preset dance choreography information and the current real-time state of the formation, ensuring that the reset command is highly consistent with the overall performance.

[0142] Upon receiving a reset command, the system controls the abnormal node to gradually restore it to formation synchronization according to the target motion trajectory parameters and target motion beat parameters. The restoration process employs a gradual rather than abrupt jump. The robot first calculates the convergence offset between the current real-time trajectory and the target trajectory, generating a segmented trajectory correction sequence; simultaneously, it calculates the difference between the current motion beat phase and the target phase, generating a progressive phase correction factor. By fusing these two methods, a gradual motion restoration control sequence is constructed, updating the robot's execution parameters in stages.

[0143] As an implementable approach, controlling the abnormal node to gradually recover to the formation synchronization state according to the target motion trajectory parameters and the target motion beat parameters includes: calculating the trajectory convergence offset based on the target motion trajectory parameters and the current real-time motion trajectory, and generating a segmented trajectory correction sequence based on the trajectory convergence offset; calculating the beat synchronization phase difference based on the target motion beat parameters and the current motion beat phase, and generating a progressive phase correction factor based on the beat synchronization phase difference; and constructing a progressive motion recovery control sequence based on the segmented trajectory correction sequence and the progressive phase correction factor.

[0144] The action execution parameters of the abnormal node are updated in stages according to the progressive action recovery control sequence, so that the abnormal node gradually reduces the trajectory deviation and beat deviation within the continuous action cycle; when the trajectory deviation and beat deviation of the abnormal node converge to the preset synchronization interval, it is determined that the abnormal node has recovered to the formation synchronization state.

[0145] The system first calculates the trajectory convergence offset based on the target motion trajectory parameters and the current real-time motion trajectory, and then generates a segmented trajectory correction sequence based on this offset. The trajectory convergence offset reflects the overall spatial difference between the current attitude and the target attitude. By segmenting this offset, it is decomposed into several consecutive small-step correction sequences, making the recovery process smoother. The trajectory convergence offset can be calculated as follows:

[0146]

[0147] in, For the target motion trajectory parameters, This represents the current real-time motion trajectory. Based on this offset, the system generates segmented trajectory correction sequences by evenly dividing the time or motion cycle, with each segment corresponding to the trajectory target of a recovery stage.

[0148] Simultaneously, the system calculates the beat synchronization phase difference based on the target action beat parameters and the current action beat phase, and generates progressive phase correction factors based on this phase difference. The phase difference quantifies the degree of lag or lead between the robot's current rhythm and the target rhythm, subsequently generating a series of progressively decaying correction factors to ensure smooth convergence of the rhythm recovery process. The formula for calculating the beat synchronization phase difference is:

[0149]

[0150] in, For the target action beat parameters, The phase correction factor for the current action beat phase can be designed as an exponential decay form, for example... ,in This is the recovery phase sequence number. This represents the convergence rate coefficient.

[0151] Then, the system constructs a progressive motion recovery control sequence based on the segmented trajectory correction sequence and the progressive phase correction factor. This sequence integrates trajectory correction and phase correction to form a complete, multi-stage control command sequence. Each stage includes specific joint target angles, motion speeds, and beat offset commands, ensuring that the recovery process takes into account both spatial attitude and temporal rhythm.

[0152] According to the progressive motion recovery control sequence, the system updates the motion execution parameters of abnormal nodes in stages, gradually reducing trajectory and rhythm deviations within continuous motion cycles. After each motion cycle, the system reassesses the current deviation and enters the next stage of update, until the deviation continues to decrease. This staged, progressive update method ensures the smoothness and controllability of the recovery process, avoiding disruption to the overall rhythm of the dance.

[0153] To further illustrate, the following is a specific example of a progressive motion recovery control sequence. Assume the anomalous robot is recovering a dance move: "The right arm is raised from its natural hanging position to a horizontal position"; the movement lasts approximately 1.2 seconds, corresponding to 8 control cycles, each cycle being 150ms. The target movement is for the right shoulder joint to rise from 0° to 90°, with the elbow joint remaining straight, and the movement's beat phase aligned with the formation on the 4th beat. The current anomalous state is that the shoulder joint only rises to 35°, with a phase lag of approximately 0.28π (approximately 50ms). The progressive motion recovery control sequence is shown in Table 1:

[0154] Table 1

[0155] Wherein, the trajectory convergence offset is:

[0156] That is, the offset gradually decreases at each stage to ensure that the action becomes smoother and smoother.

[0157] Stepwise phase correction factor:

[0158] That is, the first stage corrects 75% of the phase difference, and the difference is gradually reduced in subsequent stages to avoid abrupt changes in rhythm.

[0159] When the trajectory deviation and rhythm deviation of the abnormal node converge to within the preset synchronization interval, the system determines that the abnormal node has recovered to the formation synchronization state. At this time, the abnormal robot rejoins the overall synchronization control process and maintains consistent movements and rhythm with the other robots in the formation.

[0160] To further illustrate the technical effects of this application, a specific implementation method and the test results of this implementation method are given below.

[0161] This specific implementation was applied to a dance troupe composed of 32 bionic humanoid robots, and validated during a complete dance performance of approximately 4 minutes. The control unit employed an edge computing server equipped with an RTX 3060 GPU. Each robot uploaded its real-time motion trajectory, including joint angles, movement speed, beat phase, actuator feedback, and spatial coordinates, via a 5G Wi-Fi module at 50-millisecond intervals. The preset dance choreography information included a complete time sequence of movements and a formation change table.

[0162] At 2 minutes and 15 seconds, robot number 17 experienced a momentary malfunction in its simulated actuator, causing its right arm posture to lag by approximately 42° and its beat phase to drift by 0.31π. The system first divided the current dance sequence into 12 semantic segments based on pre-defined choreography information and determined that the current moment fell within the "rapid formation rotation" segment, which has a medium-to-high visual saliency level. Next, the system extracted multi-dimensional features from the real-time trajectory and the target trajectory, calculated the posture alignment deviation (0.47), phase drift (0.31π), and trajectory consistency difference (0.28), and constructed a multi-dimensional mismatch vector to identify it as an abnormal node.

[0163] Subsequently, based on the visual sensitivity coefficient of 0.82 and the analysis of the local visual influence domain of the semantic segment, an anomaly perception risk value of 0.76 was generated, classifying it as a high-impact level. The system immediately generated a fault-tolerant strategy: reset with the highest priority, and the reset window was set at the third beat after the current semantic segment boundary, approximately 0.45 seconds later. The collaborative compensation range was 8 normal robots within a 3.5-meter radius centered on the abnormal robot. Before the reset window arrived, the system dynamically adjusted the beat synchronization offset of these 8 robots by a maximum of 0.12π, the motion amplitude compensation by 8%-15%, and the formation contraction coefficient by 0.93, successfully reducing the degree of anomaly visual diffusion by approximately 68%. After entering the reset window, the system issued a progressive recovery control sequence to robot number 17, which consisted of 6 stages. The trajectory and beat convergence were completed smoothly within 1.05 seconds, with the final trajectory deviation converging to 2.8° and the beat deviation converging to 0.018π. The robot smoothly re-entered the formation synchronization control.

[0164] To verify the effectiveness of this solution, simulation tests were conducted using the ROS2+Gazebo platform, involving 200 repeated fault injections, and a small-scale physical test using 12 real bionic robots, involving 50 repeated tests. Simulation results showed an anomaly detection accuracy of 96.5%, an average recovery time of 0.78 seconds, and the overall formation synchronization error decreased from 0.41 rad before the fault to 0.037 rad after recovery. The visual anomaly perception score was reduced by 67.3% compared to the traditional global reset method. In the physical test, the performance continuity index improved to 98.7%, with no significant visual interruption, and the audience's perceived anomaly rate was below 4%. Compared to existing threshold detection + global reset schemes, this method improves visual protection in highly salient semantic segments by 2.1 times, while increasing system computational overhead by only 12%, fully demonstrating the significant advantages of this invention in terms of real-time performance, robustness, and artistic adaptability.

[0165] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0166] According to another embodiment, an abnormal node detection and fault-tolerant reset system for bionic robot dance group control is provided. Figure 4 A schematic block diagram of an abnormal node detection and fault-tolerant reset system in the control of a biomimetic robot dance troop, according to one embodiment, is shown. Figure 4 As shown, the device 400 includes:

[0167] The motion trajectory information acquisition unit 401 is configured to acquire the real-time motion trajectories periodically uploaded by each bionic robot in the dance troupe.

[0168] The semantic salience level calculation unit 402 is configured to divide the current dance process into action semantic segments based on preset dance choreography information, and determine the semantic salience level of the corresponding action semantic segment according to the action content information corresponding to each action semantic segment.

[0169] The anomaly impact level calculation unit 403 is configured to detect corresponding abnormal nodes based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, and determine the corresponding anomaly impact level by combining the semantic significance level of the action semantic segment where the abnormal node is located and the formation spatial position corresponding to the abnormal node.

[0170] The fault tolerance reset strategy generation unit 404 is configured to generate a corresponding fault tolerance reset strategy based on the level of the anomaly. The fault tolerance reset strategy includes a reset window and a local collaborative compensation range.

[0171] The local collaborative compensation execution unit 405 is configured to control the normal robot associated with the abnormal node to perform local collaborative compensation before the reset window arrives, so as to reduce the impact of the abnormal node on the overall motion synchronization of the dance choreography.

[0172] The motion synchronization reset control unit 406 is configured to send a motion synchronization reset command to the abnormal node within the reset window, so that the abnormal node can reconnect to the dance choreography synchronization control process according to the target motion beat.

[0173] As an feasible approach, the real-time motion trajectory in the motion trajectory information acquisition unit 401 includes: limb joint angle parameters, limb movement speed parameters, motion beat phase parameters, actuator feedback parameters, and the robot's spatial coordinate parameters in the dance formation.

[0174] As an implementable approach, the semantic saliency level calculation unit 402, when dividing the current dance process into semantic segments based on preset dance choreography information, includes: acquiring the action time sequence and formation change information from the preset dance choreography information; extracting the action trajectory change rate, limb movement energy change rate, and beat intensity change rate corresponding to each time period based on the action time sequence, and generating the action dynamic feature vector for the corresponding time period; calculating the formation spatial dispersion and formation center offset corresponding to each time period based on the formation change information, and generating the formation dynamic feature vector for the corresponding time period; performing temporal correlation analysis on the action dynamic feature vector and the formation dynamic feature vector to identify the temporal boundary points where the action change trend and the formation change trend occur synchronously; and dividing the current dance process into semantic segments based on the identified temporal boundary points.

[0175] As an implementable approach, when the semantic saliency level calculation unit 402 determines the semantic saliency level of a corresponding action semantic segment based on the action content information corresponding to each action semantic segment, the process includes: extracting the action content information corresponding to each action semantic segment, wherein the action content information includes action beat parameters, action timing parameters, action trajectory parameters, and formation change parameters; constructing the action phase distribution of each robot in the corresponding action semantic segment based on the action beat parameters and action timing parameters; and calculating the group visual fluctuation intensity of the corresponding action semantic segment based on the dispersion of the action phase distribution.

[0176] Based on the motion trajectory parameters and formation change parameters, the motion amplitude change intensity and formation visual attention weight of the corresponding motion semantic segment are calculated; according to the group visual fluctuation intensity and formation visual attention weight, the visual salience evaluation vector of the corresponding motion semantic segment is generated, and the semantic salience level of the corresponding motion semantic segment is determined based on the visual salience evaluation vector.

[0177] As an implementable approach, the anomaly impact level calculation unit 403 detects corresponding anomaly nodes based on the degree of deviation between the real-time motion trajectory and the target motion trajectory of each bionic robot. This includes: extracting corresponding motion posture feature sequences, motion beat phase sequences, and trajectory space evolution sequences based on the real-time motion trajectory and the target motion trajectory, respectively; performing temporal alignment processing on the motion posture feature sequences and the target motion posture feature sequences to calculate the posture alignment deviation; performing phase synchronization analysis on the motion beat phase sequences and the target motion beat phase sequences to calculate the phase drift; calculating the trajectory consistency difference of the current robot relative to its local formation neighborhood based on the trajectory space evolution sequence; constructing a multi-dimensional motion consistency mismatch vector based on the posture alignment deviation, phase drift, and trajectory consistency difference, and determining motion anomaly indicators based on the multi-dimensional motion consistency mismatch vector; and determining the corresponding bionic robot as an anomaly node when the motion anomaly indicator continuously exceeds a preset threshold within a continuous time window.

[0178] As an implementable approach, the anomaly impact level calculation unit 403 determines the corresponding anomaly impact level by combining the semantic salience level of the action semantic segment where the anomaly node is located and the formation spatial position corresponding to the anomaly node. This includes: constructing a local visual impact domain for the corresponding anomaly node based on its spatial coordinates in the dance formation; calculating the correlation degree of the action phase deviation of each normal robot and the formation synchronization coupling strength within the local visual impact domain to generate an action propagation impact factor for the corresponding anomaly node; determining the visual sensitivity coefficient of the corresponding action semantic segment based on its semantic salience level; generating an anomaly perception risk value for the corresponding anomaly node based on the action propagation impact factor, the visual sensitivity coefficient, and the positional weight of the anomaly node relative to the formation visual center region; and determining the corresponding anomaly impact level based on the anomaly perception risk value.

[0179] As an implementable approach, the local collaborative compensation execution unit 405 performs local collaborative compensation according to the local collaborative compensation range for the normal robots associated with the abnormal node. This includes: determining the local collaborative compensation area corresponding to the abnormal node based on the spatial position and formation connection relationship of the abnormal node in the dance formation; acquiring the real-time motion beat parameters, motion phase parameters, and formation spacing parameters of each normal robot within the local collaborative compensation area; dynamically adjusting the motion beat synchronization offset, motion amplitude compensation amount, and formation contraction coefficient of each normal robot within the local collaborative compensation area according to the abnormality impact level corresponding to the abnormal node, so as to reduce the visual diffusion of the abnormal node's motion deviation in the dance formation; generating local collaborative compensation control parameters for the corresponding normal robot based on the adjusted motion beat synchronization offset, motion amplitude compensation amount, and formation contraction coefficient; and controlling each normal robot to perform motion synchronization enhancement and local formation reconstruction based on the corresponding local collaborative compensation control parameters, so as to maintain the overall motion continuity and visual coordination of the dance formation.

[0180] As an implementable method, the motion synchronization reset control unit 406 sends a motion synchronization reset command to the abnormal node, causing the abnormal node to reconnect to the dance formation synchronization control process according to the target motion beat. This includes: sending the target motion trajectory parameters, target motion beat parameters, and target posture synchronization parameters to the abnormal node; and controlling the abnormal node to gradually restore to the formation synchronization state according to the target motion trajectory parameters and target motion beat parameters.

[0181] As an implementable method, the motion synchronization reset control unit 406 gradually restores the abnormal node to the formation synchronization state according to the target motion trajectory parameters and target motion beat parameters. This includes: calculating the trajectory convergence offset based on the target motion trajectory parameters and the current real-time motion trajectory, and generating a segmented trajectory correction sequence based on the trajectory convergence offset; calculating the beat synchronization phase difference based on the target motion beat parameters and the current motion beat phase, and generating a progressive phase correction factor based on the beat synchronization phase difference; constructing a progressive motion recovery control sequence based on the segmented trajectory correction sequence and the progressive phase correction factor; updating the motion execution parameters of the abnormal node in stages according to the progressive motion recovery control sequence, so that the abnormal node gradually reduces the trajectory deviation and beat deviation within the continuous motion cycle; and determining that the abnormal node has recovered to the formation synchronization state when both the trajectory deviation and beat deviation of the abnormal node converge to within the preset synchronization interval.

[0182] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0183] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0184] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0185] And an electronic device comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0186] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0187] in, Figure 5 An exemplary architecture of an electronic device is shown, which may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520 can communicate with each other via a communication bus 530.

[0188] The processor 510 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solution provided in this application.

[0189] The memory 520 can be implemented using ROM, RAM, static storage devices, dynamic storage devices, etc. The memory 520 can store the operating system 521 for controlling the operation of the electronic device 500, and the basic input / output system (BIOS) 522 for controlling the low-level operations of the electronic device 500. Additionally, it can store a web browser 523, a data storage management system 524, and an abnormal node detection and fault-tolerant reset system 525 for the bionic robot dance ensemble control, etc. The aforementioned abnormal node detection and fault-tolerant reset system 525 for the bionic robot dance ensemble control can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 520 and executed by the processor 510.

[0190] Input / output interface 513 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0191] Network interface 514 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0192] Bus 530 includes a pathway for transmitting information between various components of the device, such as processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520.

[0193] It should be noted that although the above-described device only shows the processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, memory 520, bus 530, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0194] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0195] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for abnormal node detection and fault-tolerant reset in the group control of a bionic robot dance, characterized in that, The method includes: Obtain the real-time motion trajectories periodically uploaded by each bionic robot in the dance troupe; Based on the preset dance choreography information, the current dance process is divided into action semantic segments, and the semantic salience level of the corresponding action semantic segment is determined according to the action content information corresponding to each action semantic segment. Based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, corresponding abnormal nodes are detected, and the corresponding abnormal impact level is determined by combining the semantic significance level of the action semantic segment where the abnormal node is located and the formation space position corresponding to the abnormal node. A corresponding fault-tolerant reset strategy is generated based on the level of anomaly impact. The fault-tolerant reset strategy includes a reset window and a local collaborative compensation range. Before the reset window arrives, the normal robots associated with the abnormal nodes perform local collaborative compensation according to the local collaborative compensation range to reduce the impact of the abnormal nodes on the overall synchronization of the dance choreography. Within the reset window, a motion synchronization reset command is sent to the abnormal node, causing the abnormal node to reconnect to the dance choreography synchronization control process according to the target motion beat.

2. The method according to claim 1, characterized in that, The real-time motion trajectory includes: limb joint angle parameters, limb movement speed parameters, motion beat phase parameters, actuator feedback parameters, and the robot's spatial coordinate parameters in the dance formation.

3. The method according to claim 1, characterized in that, The step of dividing the current dance process into semantic segments based on preset dance choreography information includes: Obtain the motion time sequence and formation change information from the preset dance choreography information; Based on the action time series, the action trajectory change rate, limb movement energy change rate and beat intensity change rate are extracted for each time period to generate the action dynamic feature vector for the corresponding time period. Based on the formation transformation information, the spatial dispersion of the formation and the offset of the formation center of gravity for each time period are calculated, and the dynamic feature vector of the formation for the corresponding time period is generated. Perform time-series correlation analysis on the dynamic feature vectors of the actions and the dynamic feature vectors of the formations to identify the time-series boundary points where the trends of action change and formation change occur synchronously; The current dance process is divided into action semantic segments based on the identified temporal boundary points.

4. The method according to claim 1, characterized in that, The step of determining the semantic salience level of the corresponding action semantic segment based on the action content information corresponding to each action semantic segment includes: Extract the action content information corresponding to each action semantic segment. The action content information includes action beat parameters, action timing parameters, action trajectory parameters, and formation change parameters. Based on the action beat parameters and action timing parameters, the action phase distribution of each robot in the corresponding action semantic segment is constructed; Based on the degree of dispersion of the action phase distribution, the intensity of group visual fluctuation of the corresponding action semantic segment is calculated; Based on the motion trajectory parameters and formation change parameters, the intensity of motion amplitude change and the visual attention weight of formation for the corresponding motion semantic segment are calculated. Based on the intensity of the group visual fluctuations and the formation visual attention weights, a visual salience evaluation vector for the corresponding action semantic segment is generated, and the semantic salience level of the corresponding action semantic segment is determined based on the visual salience evaluation vector.

5. The method according to claim 1, characterized in that, The step of detecting corresponding abnormal nodes based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory includes: Based on the real-time motion trajectory and the target motion trajectory, the corresponding motion posture feature sequence, motion beat phase sequence and trajectory space evolution sequence are extracted respectively; The action posture feature sequence and the target action posture feature sequence are time-aligned, and the posture alignment deviation is calculated. Phase synchronization analysis is performed on the action beat phase sequence and the target action beat phase sequence to calculate the phase drift. Based on the trajectory space evolution sequence, the trajectory consistency difference of the current robot relative to the local formation neighborhood is calculated; A multidimensional motion consistency mismatch vector is constructed based on the attitude alignment deviation, phase drift, and trajectory consistency difference, and motion anomaly indicators are determined based on the multidimensional motion consistency mismatch vector. When the abnormal action index continuously exceeds a preset threshold within a continuous time window, the corresponding bionic robot is identified as an abnormal node.

6. The method according to claim 1, characterized in that, The step of determining the corresponding anomaly impact level by combining the semantic salience level of the action semantic segment where the anomaly node is located and the formation space position corresponding to the anomaly node includes: Based on the spatial coordinates of the abnormal nodes in the dance formation, the local visual influence domain of the corresponding abnormal nodes is constructed. Calculate the correlation degree of the action phase deviation of each normal robot and the formation synchronization coupling strength within the local visual influence domain, and generate the action propagation influence factor of the corresponding abnormal node. Based on the semantic salience level of the action semantic segment where the abnormal node is located, determine the visual sensitivity coefficient of the corresponding action semantic segment; Based on the motion propagation influence factor, visual sensitivity coefficient, and positional weight of the abnormal node relative to the visual center region of the formation, an abnormal perception risk value is generated for the corresponding abnormal node. The corresponding level of anomaly impact is determined based on the anomaly perception risk value.

7. The method according to claim 1, characterized in that, The normal robot associated with the abnormal control node performs local collaborative compensation according to the local collaborative compensation range, including: Based on the spatial location of abnormal nodes in the dance formation and the formation connection relationship, the local collaborative compensation area corresponding to the abnormal node is determined. Obtain the real-time motion beat parameters, motion phase parameters, and formation spacing parameters of each normal robot within the local collaborative compensation area; Based on the level of abnormal impact corresponding to the abnormal node, the motion beat synchronization offset, motion amplitude compensation amount, and formation contraction coefficient of each normal robot in the local collaborative compensation area are dynamically adjusted to reduce the visual diffusion of the abnormal node motion deviation in the dance formation. Based on the adjusted motion beat synchronization offset, motion amplitude compensation, and formation contraction coefficient, local cooperative compensation control parameters corresponding to the normal robot are generated. The control of each normal robot is based on the corresponding local cooperative compensation control parameters to perform action synchronization enhancement and local formation reconstruction in order to maintain the overall movement continuity and visual coordination of the dance formation.

8. The method according to claim 1, characterized in that, The step of sending a motion synchronization reset command to the abnormal node, enabling the abnormal node to reconnect to the dance choreography synchronization control process according to the target motion beat, includes: Send target motion trajectory parameters, target motion beat parameters, and target attitude synchronization parameters to the abnormal node; The abnormal node is controlled to gradually recover to the formation synchronization state according to the target motion trajectory parameters and target motion beat parameters.

9. The method according to claim 8, characterized in that, The control anomaly node is gradually restored to the formation synchronization state according to the target motion trajectory parameters and target motion beat parameters, including: Based on the target motion trajectory parameters and the current real-time motion trajectory, the trajectory convergence offset is calculated, and a segmented trajectory correction sequence is generated based on the trajectory convergence offset. Based on the target action beat parameters and the current action beat phase, calculate the beat synchronization phase difference, and generate a progressive phase correction factor based on the beat synchronization phase difference; A progressive motion recovery control sequence is constructed based on the segmented trajectory correction sequence and the progressive phase correction factor; The action execution parameters of the abnormal node are updated in stages according to the progressive action recovery control sequence, so that the trajectory deviation and beat deviation of the abnormal node are gradually reduced within the continuous action cycle. When the trajectory deviation and beat deviation of the abnormal node converge to the preset synchronization interval, the abnormal node is determined to have recovered to the formation synchronization state.

10. An abnormal node detection and fault-tolerant reset system for the group control of a bionic robot dance, characterized in that, The system includes: The motion trajectory information acquisition unit is configured to acquire the real-time motion trajectories periodically uploaded by each bionic robot in the dance troupe; The semantic salience level calculation unit is configured to divide the current dance process into action semantic segments based on preset dance choreography information, and determine the semantic salience level of the corresponding action semantic segment according to the action content information corresponding to each action semantic segment. The anomaly impact level calculation unit is configured to detect corresponding anomaly nodes based on the degree of deviation between the real-time motion trajectory of each bionic robot and the target motion trajectory, and determine the corresponding anomaly impact level by combining the semantic significance level of the action semantic segment where the anomaly node is located and the formation space position corresponding to the anomaly node. The fault tolerance reset strategy generation unit is configured to generate a corresponding fault tolerance reset strategy based on the level of the anomaly. The fault tolerance reset strategy includes a reset window and a local collaborative compensation range. The local collaborative compensation execution unit is configured to control the normal robot associated with the abnormal node to perform local collaborative compensation before the reset window arrives, so as to reduce the impact of the abnormal node on the overall motion synchronization of the dance choreography. The motion synchronization reset control unit is configured to send a motion synchronization reset command to the abnormal node within the reset window, so that the abnormal node can reconnect to the dance choreography synchronization control process according to the target motion beat.

Images