Robot dance motion control method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a robot dance movement control method and device, equipment and a storage medium, and aims to realize automatic and accurate matching and smooth and stable execution of dance movement and music of a foot robot. The robot dance motion control method comprises the following steps: acquiring target music data, and generating a music feature vector based on the target music data; the music feature vectors are used for indicating beat information, music styles and music paragraph division information; based on the music feature vector, retrieving a matched dance action from a dance action library of the adaptive foot robot, and generating an initial action sequence; the dance movement library comprises multi-dimensional annotation information of the movement; inputting the initial action sequence into an action sequence optimization model trained in an imitation learning stage and a reinforcement learning stage for optimization, and determining an optimized action sequence; carrying out migration processing from a simulation environment to a real machine environment on the optimized action sequence through a domain adaptation loss function, and determining an adjusted action sequence adaptive to mechanical response characteristics of a real machine; and based on the adjusted action sequence, controlling the foot-type robot to execute a dance action.

Description

Technical Field

[0001] This disclosure relates to the field of robotics technology, and in particular to a method, apparatus, device, and storage medium for controlling robot dance movements. Background Technology

[0002] In recent years, the demand for robot dance performances has been growing. Related robot dance control methods generally employ pre-programmed dance techniques, meaning a complete sequence of movements is choreographed beforehand and then permanently executed. While this method ensures accurate movements, it cannot adaptively adjust to the music; each time the music changes, the dance must be re-choreographed, lacking flexibility. Furthermore, although some methods rely on simple beat synchronization, such as triggering several preset movements by detecting the music's beat, these methods typically ignore the deeper characteristics of the music, resulting in monotonous dance movements, insufficient expressiveness, and difficulty in achieving a natural harmony with the emotions conveyed by the music. Summary of the Invention

[0003] This disclosure provides a robot dance motion control method to achieve automatic, precise matching and smooth, stable execution of legged robot dance motions with music.

[0004] Firstly, a method for controlling robot dance movements is provided, comprising: acquiring target music data; generating a music feature vector based on the target music data; the music feature vector being used to indicate beat information, music style, and music segment division information; based on the music feature vector, retrieving matching dance movements from a dance movement library adapted to a legged robot, and generating an initial movement sequence; the dance movement library containing multi-dimensional annotation information of the movements, including emotional semantic tags, duration information, and connection relationship information; inputting the initial movement sequence into a movement sequence optimization model trained through imitation learning and reinforcement learning stages for optimization, and determining an optimized movement sequence; the reinforcement learning stage performing strategy optimization based on a multi-dimensional reward function including beat alignment reward, connection fluency reward, style matching reward, and mechanical feasibility reward; performing a domain adaptation loss function to transfer the optimized movement sequence from a simulation environment to a real machine environment, and determining an adjusted movement sequence adapted to the mechanical response characteristics of the real machine; and controlling the legged robot to perform dance movements based on the adjusted movement sequence.

[0005] In one embodiment, the dance movement library is constructed through the following steps: collecting basic sub-movement data and composite movement data of the legged robot, and recording joint angle change sequence information associated with each basic sub-movement data and composite movement data; calculating the similarity between different movements based on the joint angle change sequence information; filtering movement versions according to the similarity between different movements, as well as the mechanical wear and execution stability information of different movements; wherein, the same movement version is associated with at least one basic sub-movement and / or composite movement; obtaining multidimensional annotation information of the movement version, and writing the multidimensional annotation information, along with the basic sub-movement data and / or composite movement data corresponding to the movement version and the joint angle change sequence information, into the dance movement library.

[0006] In one implementation, the multidimensional annotation information includes emotional semantic tags, duration information, and connection relationship information; the emotional semantic tags include type tags corresponding to music styles after quantifying the joint movement rate and amplitude of the action; the duration information includes action execution duration information, start time information, and finish time information; wherein, for fixed-duration actions, the action execution duration information includes a fixed duration; for adjustable-duration actions, the action duration information includes a duration range and step interval; the start time information indicates the time taken for a joint to switch from an initial posture to an action posture, and the finish time... The information indicates the time taken for the joint to return to the transition posture after the action ends; the connection relationship information includes at least one preceding connection action information and at least one following connection action information that are adapted to the current action: the preceding and following connection action information are determined according to the posture compatibility requirements, emotional consistency requirements and duration matching requirements of adjacent actions, the posture compatibility requirements indicate that the joint angle difference between adjacent actions is less than a preset angle; the emotional consistency requirements indicate that actions with the same emotional label are prioritized for connection; the duration matching requirements indicate that the difference between the duration of the connection action and the duration of the original action is less than a preset duration.

[0007] In one implementation, the beat information includes the beat point position and the number of beats per unit time; generating a music feature vector based on the target music data includes: preprocessing the target music data to obtain a preprocessed audio signal, and extracting frame-level audio features from the preprocessed audio signal; inputting the frame-level audio features into a beat detector to determine the beat point position; calculating the time interval between adjacent beat points based on the beat point position to determine the number of beats per unit time, and labeling the beat strength attribute; inputting the frame-level audio features into a music style classifier to determine the music style label; inputting the frame-level audio features into a music segment recognition module to identify and output the start and end times of the verse, chorus, and interlude segments, generating music segment division information; and fusing the beat point position, the number of beats per unit time, the beat strength attribute, the music style label, and the music segment division information to generate the music feature vector.

[0008] In one implementation, the step of retrieving matching dance movements from a dance movement library adapted to the legged robot based on the music feature vector to generate an initial movement sequence includes: selecting an initial movement package with a corresponding emotional tag from the dance movement library based on the music style indicated by the music feature vector; filtering candidate movements with suitable movement durations from the initial movement package according to the number of beats per unit time indicated by the music feature vector; and generating an initial movement sequence based on the beat point position indicated by the music feature vector and the pre-annotated movement connection relationships in the dance movement library.

[0009] In one implementation, the action sequence optimization model can be trained as follows: an expert demonstration dataset is obtained, and a basic action combination strategy is trained through imitation learning, wherein the imitation learning is used to learn the action connection logic and beat alignment; based on the basic action combination strategy, a reinforcement learning model including a state space, an action space and a multi-dimensional reward function is constructed, and the action selection strategy is optimized through a joint iterative framework of policy optimization and value evaluation to obtain the action sequence optimization model.

[0010] In one implementation, the multidimensional reward function includes a weighted sum of beat alignment reward, smooth transition reward, style matching reward, and mechanical feasibility reward; the beat alignment reward is obtained by calculating the deviation between the start or end time of the movement and the beat point; the smooth transition reward is obtained by calculating the difference in joint angles between preceding and following movements and the duration matching degree; the style matching reward is obtained by comparing the consistency between the emotional semantic tags of movements in the dance movement library and the music style indicated by the music feature vector; and the mechanical feasibility reward is obtained by estimating the mechanical wear level and execution difficulty of the movement.

[0011] In one implementation, controlling the legged robot to perform dance movements based on the adjusted action sequence includes: inputting the posture feature sequences corresponding to multiple sets of the adjusted action sequences and the music feature vector into a scoring network based on a convolutional neural network and a long short-term memory network, and outputting a quantized score; selecting the action sequence with the highest score and greater than a set score threshold from the multiple sets of the adjusted action sequences based on the quantized score, and determining it as the target action sequence; and controlling the legged robot to perform dance movements based on the target action sequence.

[0012] In one embodiment, the dance movement library is further expanded by: acquiring target music and style seeds; generating stylized movement features based on the target music and style seeds; performing stylized deformation and recombination based on the stylized movement features and basic movements in the dance movement library to generate a dance sequence with specific performer style characteristics; and adding the dance sequence with specific performer style characteristics to the dance movement library after multi-dimensional annotation.

[0013] In one implementation, the optimized motion sequence is transferred from the simulation environment to the real machine environment using a domain adaptation loss function to determine an adjusted motion sequence that adapts to the mechanical response characteristics of the real machine. This includes: performing domain adaptation training on the motion sequence optimization model based on the collected posture data of the legged robot performing actual actions and the domain adaptation loss function; re-optimizing the optimized motion sequence based on the domain adaptation-trained motion sequence optimization model to generate an adjusted motion sequence adapted to the motion distribution of the real machine; verifying the adjusted motion sequence adapted to the motion distribution of the real machine in a simulation model, and outputting the adjusted motion sequence after confirming that the smoothness, rhythm adaptation, and mechanical execution safety meet the requirements.

[0014] In one implementation, controlling the legged robot to perform dance movements based on the adjusted action sequence includes: performing dance movement execution tests on the legged robot in the order of single-movement debugging, segment debugging, and full-song debugging, and obtaining test results of multiple consecutive executions of the adjusted action sequence; wherein the test results indicate movement consistency, beat alignment accuracy, and mechanical stability; the method further includes: iteratively optimizing the connection relationship information marked in the dance movement library, the parameters of the music feature extraction model, and the weight parameters of the action sequence scoring network based on the real machine execution data and human feedback collected during the testing process.

[0015] In one implementation, controlling the legged robot to perform dance movements based on the adjusted movement sequence includes: controlling the legged robot to perform dance movements based on the adjusted movement sequence, and performing dynamic beat compensation and movement adjustment during the execution process based on real-time music analysis and robot status feedback.

[0016] In one implementation, the dynamic beat compensation and motion adjustment based on real-time music analysis and robot status feedback during execution includes: real-time acquisition of the robot's joint actual angles, angular velocities, motor currents, and execution timestamps as robot status feedback signals; calculation of the expected execution time window for the current target action based on the current music beat point, beat interval, and rhythm intensity continuously output through real-time music analysis; comparison of the expected execution time window of the target action with the robot's actual execution progress; if a progress deviation is detected to exceed a preset threshold, translation compensation is performed on the start time of subsequent actions to be executed, and the joint motion curve is time-scaled to realign with the music beat; if the robot status feedback signal indicates abnormal joint load, unstable posture, or insufficient power, a replacement action matching the current music style and with lower mechanical wear and more stable execution is retrieved from the dance action library, and the action replacement and posture smooth transition are completed before the next beat point.

[0017] Secondly, embodiments of this disclosure also provide a robot dance motion control device, comprising:

[0018] The music parsing module is used to acquire target music data and generate a music feature vector based on the target music data; the music feature vector is used to indicate beat information, music style, and music segment division information. The dance movement retrieval module is used to retrieve matching dance movements from the dance movement library adapted to the legged robot based on the music feature vector, and generate an initial movement sequence; the dance movement library contains multi-dimensional annotation information of the movements, including emotional semantic tags, duration information and connection relationship information; The action sequence optimization module is used to input the initial action sequence into the action sequence optimization model trained by the imitation learning stage and the reinforcement learning stage for optimization, and to determine the optimized action sequence. The reinforcement learning stage performs policy optimization based on a multi-dimensional reward function that includes beat alignment reward, smooth transition reward, style matching reward and mechanical feasibility reward. The optimized action sequence is then transferred from the simulation environment to the real machine environment through the domain adaptation loss function to determine the adjusted action sequence that adapts to the mechanical response characteristics of the real machine. The control module is used to control the legged robot to perform dance movements based on the adjusted action sequence.

[0019] Thirdly, an apparatus is provided, comprising a processor and a memory, wherein computer instructions are stored in the memory, and the processor is configured to execute the computer instructions to perform the steps of the robot dance motion control method described in any of the above embodiments.

[0020] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, performs the robot dance motion control method described in any of the above embodiments.

[0021] This embodiment of the disclosure acquires target music data and extracts music feature vectors containing beat and style, enabling the legged robot's dance movements to adaptively match based on music content. These music feature vectors not only contain beat information but also encompass music styles that reflect emotional characteristics, allowing dance movements to match the music style. By retrieving initial movement sequences that match the music feature vectors from a dance movement library and optimizing these initial movement sequences using a movement sequence optimization model trained through imitation learning and reinforcement learning, the quality of the movement sequences can be improved, thereby enhancing the coherence, rhythm, and execution stability of the legged robot's dance.

[0022] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0023] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a robot dance motion control method provided in this disclosure embodiment; Figure 2 This is a schematic diagram of a robot dance motion control device provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of a device 200 provided in an embodiment of the present disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0028] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0030] The robot dance motion control method of this disclosure embodiment will be further described in detail below.

[0031] This disclosure provides a method for controlling robot dance movements. The robot can be a legged robot; exemplarily, the legged robot can refer to a quadruped robot (such as a robot dog). Figure 1 As shown, the robot dance motion control method provided in this embodiment may include: S101: Obtain target music data and generate a music feature vector based on the target music data; the music feature vector is used to indicate beat information, music style and music segment division information.

[0032] In this embodiment, target music data (which can be an audio file or a real-time audio stream) is first received. Key features are then extracted using a feature extraction model (or music analysis model) to form a structured music feature vector. For example, this music feature vector can indicate not only beat location and beat information such as beats per minute (BPM), but also the classification or probability distribution of music style (such as soothing, energetic, or general), as well as music segment division information.

[0033] In some embodiments, the beat information may include the beat point position and the number of beats per unit time; generating a music feature vector based on the target music data may include: preprocessing the target music data to obtain a preprocessed audio signal, and extracting frame-level audio features from the preprocessed audio signal; inputting the frame-level audio features into a beat detector to determine the beat point position (e.g., outputting the probability that each frame is a beat point); calculating the time interval between adjacent beat points based on the beat point position to determine the number of beats per unit time, and labeling the beat strength attribute (labeling the beat as a strong beat or a weak beat); inputting the frame-level audio features into a music style classifier to determine music style labels (e.g., determining the probability distribution of styles including soothing, energetic, or general); inputting the frame-level audio features into a music segment recognition module, which, based on the spectral energy distribution changes and temporal structure features of the audio signal, can identify and output the start and end times of the verse, chorus, and interlude segments, generating music segment division information; and fusing the beat point position, the number of beats per unit time, the beat strength attribute, the music style label, and the music segment division information to generate the music feature vector.

[0034] In practice, multiple music samples can be collected, covering genres such as ballads, rock, electronic, folk, dance, and classical. For example, each genre can have at least 500 samples, with durations ranging from 30 seconds to 3 minutes, a uniform sampling rate of 44.1kHz, a bit depth of 16bit, and the format converted to WAV (to avoid compression distortion). Simultaneously, each song is manually labeled with its beat points, tempo (represented by beats per minute, BPM), style tags (corresponding to emotional tags, such as soothing, energetic, and general), and section divisions (verse, chorus, interlude).

[0035] Next, the acquired initial music data can be denoised, normalized, framed, and preprocessed for features. Specifically, the initial music data can be acquired, environmental noise can be removed (e.g., using a Wiener filter algorithm) to obtain a denoised audio signal; the amplitude of the denoised audio signal can be normalized to obtain a normalized audio signal; the normalized audio signal can be framed according to a set frame length and frame shift (e.g., framed according to a 20ms frame length and a 10ms frame shift) and a Hanning window can be applied (to reduce spectral leakage) to obtain the preprocessed audio signal.

[0036] The above-mentioned preprocessed audio signal extracts frame-level audio features, such as Mel-Frequency Cepstral Coefficients (MFCC, 13-dimensional), Spectral Flux, Zero-Crossing Rate (ZCR), and Spectral Centroid. Among these, Spectral Flux reflects changes in audio energy and is a core feature for beat detection; MFCC and Spectral Centroid help distinguish beat intensity from musical layers.

[0037] The aforementioned beat detector (beat detection model) can be implemented using a Convolutional Neural Network (CNN-LSTM). For example, a CNN module (containing 3 convolutional layers + pooling layers) extracts frame-level beat features (through convolution and pooling processing) to capture local energy change patterns; these local energy change patterns are input into an LSTM module (containing 2 hidden layers) to model temporal dependencies, identify the periodicity of the beat sequence, and output the probability of each frame being a beat point; based on an adaptive threshold dynamically adjusted by music energy, the probabilities are thresholded to determine the beat point location.

[0038] For the estimation of tempo mentioned above, for example, the time interval between adjacent beat points can be calculated based on the detected beat points and converted into BPM (beats per minute). A statistical filtering algorithm is used to remove outliers (such as false beat points caused by sudden noise), and the average of 10 consecutive beat intervals is taken as the final tempo, with the error controlled within ±2 BPM. Simultaneously, tempo stability (stable / fluctuating) is labeled, and for fluctuating music (such as tempo-changing dance music), the range and rate of BPM variation are recorded.

[0039] The strong and weak beat attribute divides the beat into strong beats (e.g., beats 1 and 3) and weak beats (e.g., beats 2 and 4). The position of strong beats can be determined based on the peak spectral flux (strong beats correspond to peak spectral flux), providing a basis for aligning the duration of subsequent actions with the beat (e.g., strong beats perform actions with large amplitude, and weak beats perform transitional actions).

[0040] The aforementioned music style classifier (music style classification model) can adopt a Transformer-based architecture. The Transformer-based music style classifier captures long-term audio feature associations through a self-attention mechanism, which specifically includes: inputting the frame-level audio feature sequence into a Transformer encoder, calculating the association weights between feature frames through a self-attention mechanism; performing weighted fusion of feature frames based on the association weights to extract a global audio feature representation; inputting the global audio feature representation into a classification layer to output the probability distributions of soothing, energetic, and general styles.

[0041] Here, the input to the music style classifier is a preprocessed sequence of audio features (e.g., MFCC + spectral features, with a length equal to the number of frames multiplied by the feature dimension). The Transformer's self-attention mechanism captures long-term audio feature correlations, enabling precise differentiation of different music styles (e.g., soothing music has a high proportion of low-spectral energy and low ZCR; energetic music has a high proportion of high-spectral energy and drastic fluctuations in spectral flux). The model outputs a style probability distribution (soothing, energetic, general), and the label with the highest probability can be used as the final style label.

[0042] Furthermore, the music segment recognition module can further analyze the spectral energy, harmonic and structural temporal changes of the audio, automatically identifying segments such as the verse, chorus, and interlude. The recognition results are output as type labels and start and end timestamps for each segment and integrated into the music feature vector. Based on the segment division information, segment perception can be adapted in dance choreography; for example, using soothing movements in the verse, switching to energetic movements in the chorus, and inserting transitional movements in the interlude, thereby enhancing the overall layering and expressiveness of the dance at the musical structure level.

[0043] In addition to the core style tags, the following auxiliary style features can also be extracted, such as dynamic range (the ratio of the maximum to the minimum amplitude of the audio), which reflects changes in music intensity; and melody complexity (calculated based on spectral entropy), which reflects the richness of the melody.

[0044] After determining the above-mentioned beat point location, number of beats per unit time, beat strength attributes, music style tags, and music segment division information, the above-mentioned beat information (tempo, strong / weak beats, beat interval), style tags, music segment division information, and auxiliary style features can be integrated into a unified music feature vector, such as a music feature vector with 20 dimensions (including tempo, mean beat interval, style probability, dynamic range, etc.).

[0045] After the aforementioned feature extraction model (including the module for audio feature extraction, the beat detector, and the music style classifier) ​​has been trained, the model can be validated using a test set (e.g., containing 100 multi-style music tracks that were not used in the training). For complex music (e.g., mixed styles, music without obvious beats), rule constraints can be added (e.g., when there is no obvious beat, the default tempo is 80-100 BPM, and the style label is general) to improve the robustness of the model.

[0046] S102: Based on the music feature vector, retrieve matching dance movements from the dance movement library adapted to the legged robot and generate an initial movement sequence; the dance movement library contains multi-dimensional annotation information of the movements, including emotional semantic tags, duration information and connection relationship information.

[0047] In practice, matching dance movements can be retrieved from a pre-built dance movement library based on music feature vectors and combined into an initial movement sequence. Each movement in the dance movement library can be labeled with emotional tags (matching music style), movement duration, and transition information. Based on the music style and rhythm information indicated by the music feature vectors, matching movements can be selected from the dance movement library and initially choreographed according to music sections (such as verse, chorus, and interlude) and rhythmic strength, generating one or more initial movement sequences that meet basic rhythmic and stylistic requirements.

[0048] In some embodiments, the dance movement library can be constructed through the following steps: S102-A1: Collect the basic sub-motion data and compound motion data of the legged robot, and record the joint angle change sequence information associated with each of the basic sub-motion data and compound motion data.

[0049] Here, basic sub-movements (such as leg lift, tail swing, head nod, turn, jump, squat, etc.) and compound movements (such as continuous arm swing jump, rotating leg kick, and left and right stepping combinations) of legged robots can be collected. For example, 80-100 basic sub-movements and 20-30 compound movements can be designed. The above movements can be sampled at a certain sampling frequency (such as a sampling frequency of 100Hz) through a simulation platform to adjust the movement posture and record the joint angle change sequence (referring to the data sequence recording the change of the angle values ​​of each joint of the robot over time during the execution of a certain movement), forming a standardized movement file (such as JSON format), which can include joint identifiers (IDs), timestamps, angle values, and other data.

[0050] S102-A2: Calculate the similarity between different actions based on the joint angle change sequence information; filter action versions according to the similarity between different actions, as well as the mechanical loss and execution stability information of different actions; wherein, the same action version is associated with at least one of the basic sub-actions and / or composite actions.

[0051] Here, based on the recorded joint angle change sequence information, the Dynamic Time Warping (DTW) algorithm can be used to calculate the similarity between different actions. If the similarity is higher than a preset threshold (e.g., 85%), it is considered a redundant action. Furthermore, mechanical wear level (low / medium / high) and execution stability information can be combined to retain the action with the lowest mechanical wear and highest execution stability as the preferred version. The same version can be associated with one or more basic sub-actions and composite actions. Through this step, the action library reduces redundancy while maintaining diversity, and also improves action execution efficiency and mechanical lifespan through action selection.

[0052] S102-A3: Obtain the multidimensional annotation information of the action version, associate the multidimensional annotation information with the basic sub-action data and / or composite action data corresponding to the action version, and the joint angle change sequence information, and write it into the dance action library.

[0053] In specific implementation, the multi-dimensional annotation information may include annotation information from multiple dimensions such as sentiment semantic tags, duration information and connection relationship information, and additional annotations. Among them: 1) The emotional semantic tags include type tags corresponding to music styles after quantifying the joint movement rate and amplitude of the action.

[0054] For example, emotional semantic tags can be categorized into three types—general, soothing, and energetic—based on the speed and amplitude of joint movements. Matching the music style indicated by the music feature vector with the emotional tag achieves action matching based on emotional semantics. For instance, the general type of emotional tag corresponds to movements with moderate amplitude, slow joint movement speed (≤30° / s), and no violent jumping or rapid rotation, such as slow tail wagging, slight head nodding, and stepping in place, which are suitable for most music transitions. The soothing type of emotional tag corresponds to movements with small amplitude, low speed (≤15° / s), and slow rhythm, such as slow turning, bowing the head and hunching the chest, and slightly raising the limbs, which are suitable for slow-tempo music (such as folk songs and ballads). The energetic type of emotional tag corresponds to movements with large amplitude and high speed (≥45° / s), including elements such as jumping, rapid rotation, and continuous arm swinging, such as leaping, rapid left and right tail wagging, and spinning in place, which are suitable for fast-tempo music (such as rock, electronic, and dance music).

[0055] 2) The duration information includes action execution duration information, start time information, and end time information; wherein, for fixed-duration actions (such as nodding, 0.8 seconds), the action execution duration information includes a fixed duration; for adjustable-duration actions (such as stepping, 1.0-2.0 seconds, with a marked duration range and step interval of 0.1 seconds), the action duration information includes a duration range and a step interval; the start time information indicates the time taken for the joint to switch from the initial posture to the action posture, and the end time information indicates the time taken for the joint to return to the transition posture after the action ends.

[0056] 3) The connection relationship information includes at least one preceding connection action information and at least one following connection action information that are adapted to the current action: the preceding connection action information and the following connection action information are determined according to the posture compatibility requirements, emotional consistency requirements and duration matching requirements of adjacent actions. The posture compatibility requirements indicate that the joint angle difference between adjacent actions is less than a preset angle; the emotional consistency requirements indicate that actions with the same emotional tag are prioritized for connection; the duration matching requirements indicate that the difference between the duration of the connection action and the duration of the original action is less than a preset duration.

[0057] For example, for linking relationships, based on the starting and ending postures, 3-5 compatible preceding and following actions can be labeled, along with 3-5 suitable preceding and following actions, forming a linking relationship table. The following labeling principles can be followed when labeling linking relationships: Posture compatibility: The joint angle difference between preceding and following actions should be ≤20° to reduce mechanical impact; Emotional consistency: Actions with the same emotional label should be prioritized for linking (e.g., a passionate action should be followed by a passionate or general action to avoid directly linking to a soothing action); Duration matching: The duration difference between the linking action and the original action should be ≤0.5 seconds to facilitate beat alignment. For example, the following linking action for "airborne jump" (passionate, 1.2 seconds) can be labeled as "rapid tail swing" (passionate, 0.8 seconds), "stationary step" (general, 1.0 second), and "side kick" (passionate, 1.5 seconds).

[0058] 4) Additional annotations, including mechanical wear level (low / medium / high), execution difficulty (easy / medium / difficult), and range of compatible joints, to provide a basis for mechanical protection and feasibility assessment when selecting subsequent actions.

[0059] After associating the above multidimensional annotation information with the corresponding motion data and joint angle change sequences, it can be structured and stored in a database (such as MySQL) and a file server to form a searchable and scalable dance motion library, providing data support for subsequent music-driven motion matching and sequence optimization.

[0060] In some embodiments, based on the music feature vector, retrieving matching dance movements from a dance movement library adapted to the legged robot to generate an initial movement sequence may include: Based on the music style and music segment division information indicated by the music feature vector, an initial action package with corresponding emotional tags and adapted to each segment is selected from the dance action library; according to the number of beats per unit time indicated by the music feature vector, candidate actions with suitable action duration are filtered from the initial action package; based on the beat point position indicated by the music feature vector, the music segment division information, and the pre-annotated action connection relationship in the dance action library, an initial action sequence is generated (using the beat point as the temporal basis, and based on the ending posture of the current action, actions that meet the requirements of posture compatibility, emotional consistency, and time duration matching are selected from the pre-annotated connecting actions as subsequent actions, until the music duration is covered).

[0061] In the above embodiments, based on the music style and music segment division information indicated by the music feature vector, a set of movements corresponding to the emotional tags and adapted to each segment is selected from the dance movement library. Specifically, the emotional tag can be determined according to the music style (e.g., energetic, soothing), and then combined with the segment division information (e.g., verse, chorus, interlude) to select a suitable subset of movements for each segment. For example, for energetic music, the chorus may select energetic tag movements with larger amplitude and higher speed, while the interlude can supplement with transitional movements of some general tags to form a hierarchical movement set. Based on the beat count (BPM) indicated by the music feature vector, candidate movements with suitable movement duration are further filtered from the movement sets selected for each segment. For example, fast-paced (high BPM) segments tend to select short-duration movements, while slow-paced (low BPM) segments select long-duration movements to ensure that the basic duration of the movements matches the music tempo. Based on the beat point position indicated by the music feature vector, the music segment division information, and the pre-labeled movement connection relationships in the dance movement library, an initial movement sequence is generated in segment order. Specifically, using beat points as the temporal basis, and following the order of musical sections (e.g., verse first, then chorus, then interlude), and based on the ending posture of the current movement, actions that meet the requirements of posture compatibility, emotional consistency, and time duration are selected from pre-marked connecting movements as subsequent movements. These movements are then spliced ​​together sequentially until the entire music duration is covered, generating one or more initial movement sequences. By incorporating musical section division information, the generated dance movement sequence structurally corresponds to the undulating sections of the music, enhancing the overall layering and expressiveness of the dance choreography.

[0062] Exemplarily, when selecting the initial action package corresponding to the emotion label from the dance action library, if the style label is enthusiastic, an action package can be obtained by selecting actions with an enthusiastic emotion label and 30% of the actions with a general emotion label to form the enthusiastic style action package; when the style label is soothing, an action package can be obtained by selecting actions with a soothing emotion label and 50% of the actions with a general emotion label to form the soothing style action package; when the style label is general, an action package can be obtained by selecting actions with a general emotion label to form the general style action package; the action package can contain 10-15 core actions, taking into account diversity and connection compatibility. Based on the BPM and the action package, short actions are selected according to a fast BPM (such as BPM≥120), long actions (1.5-2.5 seconds) are selected according to a slow BPM (such as BPM≤80), and medium tempo (80<BPM<120) is flexibly combined to generate the multiple groups of initial action sequences, that is, the initial action package. Then, based on the music paragraph and beat characteristics, the initial action sequence can be generated according to the logic of strong beat action + weak beat action + transition action. For example, the action density in the verse paragraph is low (1 action per 2 beats), mainly using soothing / general actions; the action density in the chorus paragraph is high (1 action per 1 beat), mainly using enthusiastic actions; the interlude paragraph mainly uses general transition actions to connect the verse and the chorus. When initially combining, follow the connection suggestions of the action library to avoid pose conflicts, and generate at least one group of initial action sequences (each group has the same length as the music duration).

[0063] S103: Input the initial action sequence into an action sequence optimization model trained through an imitation learning stage and a reinforcement learning stage for optimization, and determine the optimized action sequence; the reinforcement learning stage optimizes the policy based on a multi-dimensional reward function including a beat alignment reward, a connection smoothness reward, a style matching reward, and a mechanical feasibility reward.

[0064] Here, the initial action sequence is further optimized by using an action sequence optimization model trained through imitation learning and reinforcement learning. In a specific implementation, the action sequence optimization model can score and adjust the initial action sequence in multiple dimensions such as beat alignment, connection smoothness, style consistency, and mechanical feasibility, and iteratively optimize action selection, duration fine-tuning, pose transition, etc. through a reinforcement learning strategy, and finally output a dance action sequence that is better in terms of smoothness, expressiveness, and executability.

[0065] In some embodiments, the action sequence optimization model can be trained as follows: an expert demonstration dataset is obtained, and a basic action combination policy is trained through imitation learning, wherein the imitation learning is used to learn the action connection logic and beat alignment; based on the basic action combination policy, a reinforcement learning model including a state space, an action space and a multi-dimensional reward function is constructed, and the action selection policy is optimized through a joint iterative framework of policy optimization and value evaluation to obtain the action sequence optimization model.

[0066] In this embodiment, the training of the motion sequence optimization model is divided into two stages, combining imitation learning and reinforcement learning, with the aim of efficiently and stably learning strategies for generating high-quality dance motion sequences.

[0067] In the imitation learning phase, high-quality dance sequence sets covering various music styles, designed by experts, are first collected to form an expert demonstration dataset. These sequences demonstrate good action transition logic, beat alignment, and style matching relationships. The model learns the action selection and combination rules in the expert sequences through imitation learning (such as behavior cloning). The goal of this phase is to enable the model to quickly master the basic action transition logic and beat alignment patterns, avoiding exploration from scratch, thereby significantly reducing the policy search space and accelerating subsequent training convergence.

[0068] Next, based on the initial strategy obtained through imitation learning, a reinforcement learning model is constructed for fine-tuning. Among the elements of the reinforcement learning model, the state space indicates the current action posture information (joint angles, duration), the current beat position and tempo of the music, and the information of the executed action sequence. The action space indicates the selection of candidate actions and duration adjustment (within the labeled duration range), and fine-tuning of action transition postures (e.g., joint angles ±5°). The multi-dimensional reward function is the core of the guided optimization, integrating multiple key indicators, such as beat alignment reward (encouraging precise alignment of action start / end times with music beat points), smooth transition reward (encouraging smooth transitions in posture and duration between actions), style matching reward (encouraging consistency between action emotional labels and music style labels), and mechanical feasibility reward (encouraging avoidance of high-loss, high-difficulty actions to ensure execution safety). That is, the beat alignment reward is obtained by calculating the deviation between the start or end time of the movement and the beat point; the smooth transition reward is obtained by calculating the difference in joint angles between the preceding and following movements and the duration matching degree; the style matching reward is obtained by comparing the consistency between the emotional semantic tags of the movements in the dance movement library and the music style indicated by the music feature vector; and the mechanical feasibility reward is obtained by estimating the mechanical wear level and execution difficulty of the movement. The total reward value can be obtained by weighted summing of the beat alignment reward (e.g., weight 0.4), smooth transition reward (e.g., weight 0.3), style matching reward (e.g., weight 0.2), and mechanical feasibility reward (e.g., weight 0.1).

[0069] The reinforcement learning model described above employs a joint iterative framework of policy optimization and value evaluation (the Actor-Critic framework) to optimize action selection strategies. The Actor network (policy network) is responsible for selecting actions based on the current state. The Critic network (value network) is responsible for evaluating the quality of the current state or state-action pair, providing optimization direction for the Actor network. During training, the model starts with an initial action sequence and continuously attempts to generate new action sequences by interacting with the environment (simulation or rule simulator), receiving feedback (reward or penalty) based on a multi-dimensional reward function. Through iterative updates, the Actor and Critic networks gradually learn a strategy that maximizes cumulative rewards—that is, an action sequence generation strategy that achieves the optimal balance between beat alignment, fluency, style matching, and mechanical feasibility.

[0070] Through the above two stages of training, the final motion sequence optimization model not only possesses reliable basic combination capabilities learned from expert experience, but also has the ability to autonomously optimize and adapt to new music in complex, multi-objective environments through reinforcement learning, thus enabling it to stably generate high-quality, highly expressive robot dance motion sequences.

[0071] S104: The optimized action sequence is transferred from the simulation environment to the real machine environment through the domain adaptation loss function to determine the adjusted action sequence that adapts to the mechanical response characteristics of the real machine.

[0072] To address the execution deviation issue caused by differences in mechanical characteristics (such as joint delay, friction, and dynamic response) when executing motion sequences trained and optimized in a simulation environment on a real robot, this step employs a domain adaptation technique. By constructing a domain adaptation loss function, the differences in motion posture distribution between the simulation and the real machine are aligned, thereby effectively transferring the simulation-optimized motion sequences to the real machine environment. Fine-tuning of the motion ensures that the sequences maintain high smoothness, beat alignment, and execution safety on the real machine.

[0073] In some embodiments, the optimized motion sequence is transferred from the simulation environment to the real machine environment using a domain adaptation loss function to determine an adjusted motion sequence that adapts to the mechanical response characteristics of the real machine. This may include: performing domain adaptation training on the motion sequence optimization model based on the collected posture data of the legged robot performing actual actions and the domain adaptation loss function; re-optimizing the optimized motion sequence based on the domain adaptation-trained motion sequence optimization model to generate an adjusted motion sequence adapted to the motion distribution of the real machine; verifying the adjusted motion sequence adapted to the motion distribution of the real machine in a simulation model, and outputting the adjusted motion sequence after confirming that the smoothness, rhythm adaptation, and mechanical execution safety meet the requirements.

[0074] The above embodiment inserts a process of real machine data feedback and re-optimization between the original simulation optimization and real machine execution. Specifically, using the optimized motion sequence (generated by the motion sequence optimization model in a pure simulation environment) and the collected real machine posture data (the real machine performs a small number of basic or specific actions, recording its actual joint angles, speeds, etc.), the original motion sequence optimization model is fine-tuned using a domain adaptation loss function (usually adversarial loss or a distribution difference metric, such as Maximum Mean Discrepancy (MMD)). The goal is to align the model's cognition from the simulated motion data distribution to the real machine's motion data distribution, thereby obtaining a new motion sequence optimization model calibrated with real machine cognition. Specifically, a feature extractor and a domain discriminator can be constructed. The feature extractor extracts the action and posture features from the simulation data and the real machine data, and the domain discriminator determines whether the features originate from the simulation domain or the real machine domain, thus obtaining the domain discrimination result. Based on the domain discrimination result, the difference in feature distribution between domains is minimized through adversarial training, enabling the feature extractor to generate domain-invariant features. In this way, the input to the subsequent action sequence model is no longer the original posture data, but the domain-invariant features output by the feature extractor, thereby completing the model transfer from the simulation environment to the real machine.

[0075] Then, the domain-adapted training-optimized motion sequence model (including the optimized feature extractor mentioned above) is reapplied to the optimized motion sequence for fine-tuning and refinement, resulting in an adjusted motion sequence adapted to the motion distribution of the real machine. Next, the generated motion sequence, theoretically adapted to the real machine, is input into a high-fidelity simulation model (finely calibrated with real machine parameters) for further verification. The verification primarily checks smoothness (whether motion transitions remain natural), rhythm adaptation (whether the alignment of motion with the music's rhythm remains accurate after fine-tuning), and mechanical execution safety (checking for risks such as joint over-limits, energy overload, and self-collision in a more realistic simulation). After all verifications pass, the adjusted motion sequence is finally output to control the real machine's execution. If verification fails, it may be necessary to return to the first two steps to readjust the motion sequence optimization model and re-optimize the motion sequence. The high-fidelity simulation model can be constructed by adjusting the simulation platform parameters based on the real machine's mechanical parameters (joint degrees of freedom, motor power, joint angle limits, friction coefficient), thus minimizing the difference between the simulation and the real machine.

[0076] S105: Based on the adjusted action sequence, control the legged robot to perform dance movements.

[0077] In practical implementation, the adjusted motion sequence can be converted into robot joint control commands to drive the legged robot to perform dance movements. For example, control signals can be generated based on the joint angles and timing information of each movement in the motion sequence, and a real-time communication mechanism can be used to ensure synchronization between the movements and the music playback. During execution, further fine-tuning can be achieved through real-machine debugging and feedback mechanisms to ensure the stability, expressiveness, and actual effect of the dance.

[0078] There can be multiple sets of the adjusted action sequences. In practice, these multiple sets of adjusted action sequences can be quantitatively evaluated and screened to select the final target action sequence used to control the legged robot. In some embodiments, controlling the legged robot to perform dance movements based on the adjusted action sequences may include: inputting the posture feature sequences (mathematical and structured representations of the optimized action sequences) corresponding to the multiple sets of adjusted action sequences and the music feature vector into a scoring network based on convolutional neural networks and long short-term memory networks, and outputting a quantitative score; selecting the action sequence with the highest score and greater than a set score threshold from the multiple sets of adjusted action sequences based on the quantitative score, and determining it as the target action sequence; and controlling the legged robot to perform dance movements based on the target action sequence.

[0079] Here, multiple sets of adjusted motion sequences (their mathematical representation, i.e., posture feature sequences) and corresponding music feature vectors (containing information such as beat and style) are used as input. A pre-trained scoring network based on a Convolutional Neural Network (CNN-LSTM) integrates and analyzes these two types of information, outputting a quantitative score (e.g., 0-100 points) to comprehensively evaluate the quality of the dance sequence. Then, the motion sequence with the highest score among all candidate adjusted motion sequences is selected. If the quantitative score of this motion sequence is greater than a preset score threshold (e.g., 80 points mentioned in the document), it is identified as the target motion sequence, and the legged robot is controlled to execute this target motion sequence. If the highest-scoring motion sequence does not meet the requirement of exceeding the preset score threshold, a failure feedback mechanism can be triggered, returning to the reinforcement learning model to re-optimize the motion sequence, ensuring that the legged robot ultimately executes a high-quality motion sequence.

[0080] For example, the scoring network based on Convolutional Neural Network - Long Short-Term Memory Network (CNN-LSTM) can be trained based on the following scoring rules: beat fit scoring rules, including the matching degree between action duration and beat interval and the matching degree between strong and weak beat actions; fluency and coherence scoring rules, including the smoothness of action transitions and the overall fluency of the sequence; style fit scoring rules, including the consistency of sentiment labels and paragraph fit; mechanical feasibility scoring rules, including the safety of action execution and the rationality of energy consumption; and diversity scoring rules, which score based on the proportion of repetitive actions in the action sequence.

[0081] Here, beat fit (e.g., accounting for 30 points out of 100) is used to evaluate whether the dance is on time. It can include: ① Matching of movement duration with beat interval (15 points). For example, a deviation of ≤0.1 seconds between the movement duration and the beat interval earns full marks, a deviation of 0.1-0.2 seconds earns 10 points, and a deviation >0.2 seconds earns less than 5 points. ② Matching of strong / weak beat movements (15 points), which is the correspondence between the strength of the movements and the strength of the music's beats. For example, movements with large amplitude and strong emotions on strong beats earn full marks, and transitional movements on weak beats earn full marks. Mismatches deduct 5-10 points.

[0082] Fluency and coherence (e.g., accounting for 25 out of 100 points) are used to evaluate whether the dance itself is smooth, focusing on whether the transitions between movements are natural. This can include: ① Smoothness of movement transitions (15 points), for example, full marks are awarded for seamless transitions in joint angles between movements (angular velocity change ≤20° / s), 10 marks are awarded for slight abrupt changes (20-30° / s), and 5 marks or less for obvious abrupt changes (>30° / s); ② Overall smoothness of the sequence (10 points): full marks are awarded for no pauses, repetitions, or redundancies, and for the sequence rhythm to be consistent with the music rhythm. 3-8 marks are deducted for pauses or redundancies.

[0083] Style matching (e.g., accounting for 20 points out of 100) is used to evaluate whether the dance matches the music's emotion, and can include: ① Consistency of emotional labels (12 points), for example, full marks are awarded if the emotional labels of the movement package are completely consistent with the music style labels, 8 marks are awarded if they are partially consistent (e.g., passionate + general), and 0 marks are awarded if they are completely inconsistent; ② Section fit (8 points), for example, full marks are awarded if the density and intensity of the movements match the music section (verse / chorus / interlude), and 3-6 marks are deducted if they do not match.

[0084] Mechanical feasibility (e.g., accounting for 15 points out of 100) is used to evaluate whether dance is safe and feasible for the legged robot itself, avoiding dangerous combinations of movements that exceed mechanical limits, cause excessive wear and tear, or consume too much energy. This can include: ① Safety of movement execution (10 points): Full marks are awarded for no movements that exceed mechanical joint limits or involve high wear and tear; 3-5 points are deducted for minor risks; and 0 points are awarded for serious risks. ② Reasonableness of energy consumption (5 points): Full marks are awarded for moderate energy consumption of the movement sequence (calculated based on the joint motor power consumption model); 2-5 points are deducted for excessive energy consumption.

[0085] Diversity (e.g., accounting for 10 out of 100 points) is used to evaluate whether a dance is rich and not monotonous, encouraging the use of diverse movements and avoiding repetition. For example, a full score is awarded if the percentage of repetitive movements in a movement sequence is ≤20%, 7 points for 20%-30%, 4 points for 30%-40%, and 0 points for >40%, encouraging a diverse combination of movements.

[0086] Based on the above scoring rules, multiple adjusted action sequences generated by the reinforcement learning model are scored by a scoring network, and the sequence with the highest score and ≥ a preset score threshold (e.g., 80 points) is selected as the optimal sequence; if all sequence scores are < the preset score threshold, the process returns to the reinforcement learning module for re-optimization (e.g., adjusting the action selection strategy and reward function weights) until a qualified action sequence is generated.

[0087] The aforementioned dimensions constitute a comprehensive and balanced evaluation system for dance movement quality. During training, experts can score a large number of dance sequence samples based on these rules, thereby guiding the scoring network to learn evaluation criteria that conform to human aesthetics and engineering constraints. After learning, the scoring network can automatically perform consistent quantitative scoring on new, unguided movement sequences, thus selecting the optimal target movement sequence.

[0088] In some embodiments, the dance movement library can be expanded by: acquiring target music and style seeds; generating stylized movement features based on the target music and style seeds; performing stylized deformation and recombination based on the stylized movement features and basic movements in the dance movement library to generate a dance sequence with specific performer style characteristics; and adding the dance sequence with specific performer style characteristics to the dance movement library after multi-dimensional annotation.

[0089] Here, the style seed is, for example, a ballet style, a robotic dance style, or a specific choreographer's style. This style seed can be a demonstration dance video of that style, movement data, or a trained model encoding that represents that style. In practice, generative models can be used to deeply integrate the beat and emotional characteristics of the target music with the unique movement patterns of the style seed (such as the way hips are lifted and legs are kicked) to generate abstract stylized movement features. These stylized movement features indicate a set of style rules used to guide movement transformation. Based on these style rules (stylized movement features), existing basic movements in the dance movement library can be modified. For example, a simple side step, after being transformed by a specific choreographer's style rules, might be adjusted to be more rhythmic, smoother, and incorporate signature arm and head movements, thus becoming a moonwalk-style slide. A large number of stylized and recombined new movements are combined into a complete dance movement sequence adapted to the target music. Finally, the dance sequence can be annotated in multiple dimensions (emotional tags, duration, transitions, etc.) and then added to the dance movement library as a new composite movement or style movement package. This allows the legged robot's dance to not only match music but also imitate or even integrate specific human performance styles.

[0090] In practical implementation, when generating stylized motion features based on the target music and style seed, the target music and style seed can be input into a Generative Adversarial Network (GAN) or a Variational Autoencoder (VAE) to generate the stylized motion features. For example, the GAN may include a generator and a discriminator. The generator attempts to forge realistic motion features that conform to the style based on the target music and style seed. The discriminator learns to distinguish whether the input motion features are genuinely from standard data of that style or forged by the generator. The generator and discriminator continuously learn against each other, ultimately enabling the generator to produce high-quality stylized motion features. As another example, if a Variational Autoencoder (including an encoder-decoder structure) is used, a large amount of style dance data can first be encoded into a low-dimensional, continuous style latent space, where different points represent different style mixtures. When generation is needed, a general direction is determined based on the target music, and then a specific coordinate point (i.e., the stylized motion feature) is located in this latent space based on the style seed. Finally, the decoder decodes this coordinate point and restores it into a specific, executable sequence of action features.

[0091] The aforementioned motion sequence optimization model can be trained in a simulation environment. Due to the differences in mechanical characteristics and physical parameters (such as friction coefficient and joint response speed) between the simulation environment and the real machine, in an optional implementation, the motion sequence can be optimized using Sim2Real technology to ensure that the execution effect of the real machine is consistent with the simulation.

[0092] In some embodiments, before controlling the legged robot to perform dance movements based on the optimized motion sequence, the method further includes setting up the deployment environment and establishing a real-time synchronization mechanism. Exemplarily, the legged robot can employ a small to medium-sized quadruped robot platform (such as UnitreeGo1) integrated with an embedded controller (such as the NVIDIA Jetson series). The software architecture is built on a Robot Operating System (ROS) and integrates a music parsing module, a motion control module, and a real-time communication module. Controlling the legged robot to perform dance movements specifically includes: parsing input music data in real time through the embedded controller to generate a music feature vector. The input supports local audio files or real-time audio stream input via a microphone. The motion control module receives the music feature vector and the optimized motion sequence, and issues control commands to the robot's joint servos via ROS topics at a control frequency of not less than 100Hz. A high-precision time synchronization mechanism aligns the timeline of music playback with the timeline of motion command execution, ensuring that the time error between motion execution and music beat is less than a set time (e.g., not greater than 0.05 seconds).

[0093] In some embodiments, to ensure the accurate implementation and continuous optimization of robot dance from simulation to real machine, this disclosure proposes a real machine debugging mechanism. It can ensure execution stability through three-level testing: single action, segment, and full song. It also uses real machine data and human feedback to dynamically optimize the action library, music analysis and scoring model, so as to realize the system's leap from accurate execution to autonomous evolution.

[0094] For example, controlling the legged robot to perform dance movements based on the adjusted action sequence can include: testing the legged robot's dance movement execution in the order of single-movement debugging, segment debugging, and full-song debugging, and obtaining test results of multiple consecutive executions of the adjusted action sequence; wherein, the test results indicate the consistency of movements, the accuracy of beat alignment, and the mechanical stability; the method also includes: iteratively optimizing the connection relationship information marked in the dance movement library, the parameters of the music feature extraction model, and the weight parameters of the action sequence scoring network based on the real machine execution data and human feedback collected during the testing process.

[0095] The above process includes two parts: the first part is to carry out a hierarchical and progressive real device debugging process; the second part is to feed back the problems exposed and the data collected during the adjustment process to the preceding modules to drive the optimization of the action library, music analysis and scoring model, etc.

[0096] In the first part, the adjustment process is divided into single-action debugging, segmented debugging, and full-song debugging. Specifically, single-action debugging is performed first to verify the execution effect of each action on the actual device and adjust joint parameters. Then, segmented debugging is performed to verify the smoothness of the action sequence in the verse, chorus, and interlude. Finally, full-song debugging is performed to verify the overall adaptation effect. Single-action debugging verifies the execution effect of each basic action unit in the dance motion library on the actual device. Specifically, the adjusted action sequence can be broken down, allowing the robot to execute each leg lift, tail swing, jump, etc., individually. The proportional-integral-derivative (PID) parameters of the joints and the torque limit are adjusted to ensure that each action itself is stable, accurate, and jitter-free. Segmented debugging mainly verifies the matching degree between the action connection logic and the music segments. The entire music can be divided into segments (verse, chorus, interlude), and the robot can execute the corresponding action sequence segment by segment to verify the effectiveness of the action combination logic and music analysis. Full-song debugging primarily verifies the integrity and endurance of the overall dance. This involves having the robot perform the entire song's dance multiple times consecutively (as described in the documentation, 10 times) to evaluate movement consistency, beat alignment accuracy, and mechanical stability. This debugging approach, proceeding in the order of individual movements, segments, and the entire song, reduces debugging complexity and allows for rapid problem localization. For example, if the entire song is faulty, the problem can be traced back to a segment; if a segment is faulty, the problem can be traced back to an individual movement.

[0097] In some embodiments, during the segmented or full-length debugging process described above, the parameters of the motion sequence can be fine-tuned online based on the actual execution effect of the legged robot to resolve problems encountered during execution. The fine-tuning includes at least one of the following: if there is a stutter during motion transitions, the allowable adjustment range of the joint angles of the transition postures between the transitional motions is increased to make the transition smoother; if the motion execution is misaligned with the music beat, the start time offset of one or more motions is fine-tuned to realign with the beat; if the motion amplitude does not match expectations, the target value of the joint angle of the relevant motion is fine-tuned within the joint torque safety limit.

[0098] Furthermore, during the aforementioned full-cycle debugging, including stability testing, the legged robot can be controlled to continuously execute the adjusted action sequence multiple times (e.g., 10 times). By analyzing the sensor data from multiple executions, the stability of the system is verified, with standards including, for example, action consistency: during multiple consecutive executions, the trajectory deviation of key joints is less than a preset threshold; mechanical stability: after continuous operation, the robot has no component fault alarms, and parameters such as joint motor temperature and current are all within the normal and safe range; no error accumulation: compared with the last execution, there is no significant increase in the overall posture drift or beat alignment error at the end of the action.

[0099] In the second part, the problems exposed during debugging and the collected data are fed back to the preceding core modules to drive the evolution of the entire system. This includes optimizing the connection relationships in the dance movement library: for example, if debugging reveals a stuttering transition from a jump to a squat, even if the original annotation table considers them connectable, the annotations can be automatically or manually updated to lower the connection priority of these two movements, or a transitional movement can be suggested. This makes the movement library increasingly rich in experience and reliable. Another example is optimizing the parameters of the music feature extraction model: if deviations are found in beat detection for a certain complex rhythm (such as syncopation), beat misalignment data exposed during real-device execution can be used to fine-tune the music parsing model, improving its parsing accuracy for this type of music. Yet another example is optimizing the weights of the movement sequence scoring network: if human feedback generally considers a dance to be excellent, but the scoring network gives a low score, it indicates a discrepancy between the scoring criteria and human aesthetics. This feedback data can be used to retrain or adjust the weights of various dimensions (beat, smoothness, style, etc.) of the scoring network, making its scoring criteria more aligned with human subjective feelings, thereby guiding the reinforcement learning model to generate more well-received dance movements.

[0100] In some embodiments, during the execution of dance movements by the legged robot based on the adjusted action sequence, dynamic beat compensation and action adjustment can be performed according to real-time music analysis and robot status feedback.

[0101] Here, while the robot performs a dance based on the adjusted movement sequence, real-time music analysis (continuously tracking music beats and intensity changes) and robot status monitoring (collecting real-time data such as joint angles, angular velocities, and motor currents) can be continuously performed. Based on these two types of real-time feedback information, the upcoming movements can be dynamically and online adjusted in the execution flow to compensate for execution deviations caused by environmental interference, hardware response delays, or music playback jitter, thereby improving the robustness, synchronization accuracy, and overall stability of the dance in real, non-ideal environments.

[0102] In practical implementation, the robot's joint angles, angular velocities, motor currents, and execution timestamps can be collected in real time as robot status feedback signals. Based on the current music beat point, beat interval, and rhythm intensity continuously output through real-time music analysis, the expected execution time window of the current target action is calculated. The expected execution time window of the target action is compared with the robot's actual execution progress. If the progress deviation exceeds a preset threshold, the start time of the subsequent action to be executed is compensated by translation, and the joint motion curve is time-scaled to realign with the music beat. If the robot status feedback signal indicates abnormal joint load, unstable posture, or insufficient power, an alternative action that matches the current music style, has lower mechanical wear, and is more stable to execute is retrieved from the dance action library, and the action replacement and posture transition are completed before the next beat point.

[0103] Here, by calculating the expected execution time window of the target action in real time and comparing it with the robot's actual execution progress, once a progress deviation is detected (e.g., the action starts late by 0.05 seconds), the start time of all subsequent actions to be executed can be shifted as a whole, and the joint motion curves can be stretched or compressed on the time axis. This is equivalent to applying a time warp correction to the entire subsequent dance sequence, so that the dance rhythm is accurately aligned with the music beat without interrupting the performance continuity.

[0104] When robot status feedback (such as a sudden increase in motor current or abnormal posture sensor data) indicates mechanical risks (joint overload, risk of imbalance) or system limitations (low battery), it can proactively perform motion replacement. Specifically, it can retrieve alternative movements from a dance movement library in real time that match the current music style and emotion but have lower mechanical wear and are more stable to execute. A smooth posture transition path is then planned to ensure a safe and seamless switch to the safer movement before the next musical beat. This achieves online motion safety replanning based on the robot's own state, effectively preventing mechanical damage and performance interruptions.

[0105] The robot dance motion control method provided in this disclosure can be used not only for controlling a single robot, but also for the collective control of multiple robots. For example, the optimized motion sequence may include a global formation transformation sequence for multiple robots. In this case, converting the optimized motion sequence into robot joint control commands and controlling the legged robot to perform dance motions through the robot joint control commands may include: assigning individual trajectories to each slave robot based on the global formation transformation sequence, obtaining slave robot trajectory data, and converting the slave robot trajectory data into robot joint control commands; sending the robot joint control commands corresponding to each slave robot to the corresponding slave robot through a local communication network, so as to execute the robot joint control commands on each slave robot, and using a distributed model predictive control algorithm to coordinate individual motion execution and collision avoidance with neighboring robots to achieve formation maintenance and collective dance execution.

[0106] When using a distributed model predictive control algorithm to coordinate individual action execution and collision avoidance with neighboring machines, the current state and trajectory target of the machine can be obtained, the state trajectory within a future time window can be predicted, and a predicted trajectory can be obtained; the position and predicted trajectory of the neighboring robot dog can be obtained, the minimum distance between the predicted trajectory of the machine and the predicted trajectory of the neighboring machine can be calculated, and it can be determined whether there is a collision risk; when there is a collision risk, the predicted trajectory of the machine can be adjusted under the premise of satisfying the formation maintenance constraint to generate a collision avoidance trajectory; based on the collision avoidance trajectory or the predicted trajectory, joint control commands can be calculated to control the robot dog to perform actions.

[0107] In this way, based on the precise dance of a single robot, collaborative control has enabled the safe and reliable choreography, coordination, autonomy, and on-site execution stability of a multi-robot collective dance.

[0108] like Figure 2 As shown, this embodiment of the disclosure provides a robot dance motion control device 200, including: Music parsing module 21 is used to acquire target music data and generate music feature vectors based on the target music data; the music feature vectors are used to indicate beat information, music style and music segment division information; The dance movement retrieval module 22 is used to retrieve matching dance movements from the dance movement library adapted to the legged robot based on the music feature vector, and generate an initial movement sequence; the dance movement library contains multi-dimensional annotation information of the movements, including emotional semantic tags, duration information and connection relationship information; The action sequence optimization module 23 is used to input the initial action sequence into the action sequence optimization model trained by the imitation learning stage and the reinforcement learning stage for optimization, and to determine the optimized action sequence. The reinforcement learning stage performs policy optimization based on a multi-dimensional reward function that includes beat alignment reward, smooth transition reward, style matching reward and mechanical feasibility reward. The optimized action sequence is transferred from the simulation environment to the real machine environment through the domain adaptation loss function to determine the adjusted action sequence that adapts to the mechanical response characteristics of the real machine. The control module 24 is used to control the legged robot to perform dance movements based on the adjusted action sequence.

[0109] In one embodiment, the robot dance motion control device 200 further includes: a dance motion library construction module 25, used to collect basic sub-motion data and composite motion data of the legged robot, and record joint angle change sequence information associated with each of the basic sub-motion data and composite motion data; calculate the similarity between different motions based on the joint angle change sequence information; filter motion versions according to the similarity between different motions, as well as the mechanical loss and execution stability information of different motions; wherein, the same motion version is associated with at least one of the basic sub-motions and / or composite motions; obtain multi-dimensional annotation information of the motion version, associate the multi-dimensional annotation information with the basic sub-motion data and / or composite motion data corresponding to the motion version and the joint angle change sequence information, and write it into the dance motion library.

[0110] In one implementation, the multidimensional annotation information includes emotional semantic tags, duration information, and connection relationship information; the emotional semantic tags include type tags corresponding to music styles after quantifying the joint movement rate and amplitude of the action; the duration information includes action execution duration information, start time information, and finish time information; wherein, for fixed-duration actions, the action execution duration information includes a fixed duration; for adjustable-duration actions, the action duration information includes a duration range and step interval; the start time information indicates the time taken for a joint to switch from an initial posture to an action posture, and the finish time... The information indicates the time taken for the joint to return to the transition posture after the action ends; the connection relationship information includes at least one preceding connection action information and at least one following connection action information that are adapted to the current action: the preceding and following connection action information are determined according to the posture compatibility requirements, emotional consistency requirements and duration matching requirements of adjacent actions, the posture compatibility requirements indicate that the joint angle difference between adjacent actions is less than a preset angle; the emotional consistency requirements indicate that actions with the same emotional label are prioritized for connection; the duration matching requirements indicate that the difference between the duration of the connection action and the duration of the original action is less than a preset duration.

[0111] In one implementation, the beat information includes the beat point position and the number of beats per unit time. When the music parsing module 21 generates a music feature vector based on the target music data, it is used to: preprocess the target music data to obtain a preprocessed audio signal, and extract frame-level audio features from the preprocessed audio signal; input the frame-level audio features into a beat detector to determine the beat point position; calculate the time interval between adjacent beat points based on the beat point position to determine the number of beats per unit time, and label the beat strength attribute; input the frame-level audio features into a music style classifier to determine the music style label; input the frame-level audio features into a music segment recognition module to identify and output the start and end times of the verse, chorus, and interlude segments, and generate music segment division information; and fuse the beat point position, the number of beats per unit time, the beat strength attribute, the music style label, and the music segment division information to generate the music feature vector.

[0112] In one implementation, when the dance movement retrieval module 22 retrieves matching dance movements from the dance movement library adapted to the legged robot based on the music feature vector and generates an initial movement sequence, it is used to: select an initial movement package with corresponding emotional tags from the dance movement library based on the music style indicated by the music feature vector; filter candidate movements with suitable movement durations from the initial movement package according to the number of beats per unit time indicated by the music feature vector; and generate an initial movement sequence based on the beat point position indicated by the music feature vector and the pre-annotated movement connection relationships in the dance movement library.

[0113] In one embodiment, the model further includes a model training module 26, configured to train the action sequence optimization model in the following manner: acquiring an expert demonstration dataset, training a basic action combination strategy through imitation learning, wherein the imitation learning is used to learn the action connection logic and beat alignment; based on the basic action combination strategy, constructing a reinforcement learning model including a state space, an action space, and a multi-dimensional reward function, and optimizing the action selection strategy through a joint iterative framework of policy optimization and value evaluation to obtain the action sequence optimization model.

[0114] In one implementation, the multidimensional reward function includes a weighted sum of beat alignment reward, smooth transition reward, style matching reward, and mechanical feasibility reward; the beat alignment reward is obtained by calculating the deviation between the start or end time of the movement and the beat point; the smooth transition reward is obtained by calculating the difference in joint angles between preceding and following movements and the duration matching degree; the style matching reward is obtained by comparing the consistency between the emotional semantic tags of movements in the dance movement library and the music style indicated by the music feature vector; and the mechanical feasibility reward is obtained by estimating the mechanical wear level and execution difficulty of the movement.

[0115] In one implementation, when the control module 24 controls the legged robot to perform dance movements based on the adjusted action sequence, it inputs the posture feature sequences corresponding to multiple sets of the adjusted action sequences and the music feature vector into a scoring network based on a convolutional neural network and a long short-term memory network, and outputs a quantized score; based on the quantized score, it selects the action sequence with the highest score and greater than a set score threshold from the multiple sets of the adjusted action sequences and determines it as the target action sequence; based on the target action sequence, it controls the legged robot to perform dance movements.

[0116] In one implementation, the dance movement library construction module 25 is further used to expand the dance movement library by: acquiring target music and style seeds; generating stylized movement features based on the target music and style seeds; performing stylized deformation and recombination based on the stylized movement features and basic movements in the dance movement library to generate a dance sequence with specific performer style characteristics; and adding the dance sequence with specific performer style characteristics to the dance movement library after multi-dimensional annotation.

[0117] In one implementation, the model training module 26 is further configured to: perform domain adaptation training on the action sequence optimization model based on the collected posture data of the legged robot performing actual actions and the domain adaptation loss function; The action sequence optimization module 23 performs migration processing from the simulation environment to the real machine environment on the optimized action sequence through the domain adaptation loss function. When determining the adjusted action sequence that adapts to the mechanical response characteristics of the real machine, it is used to: re-optimize the optimized action sequence based on the action sequence optimization model trained by domain adaptation to generate an adjusted action sequence that adapts to the action distribution of the real machine; verify the adjusted action sequence that adapts to the action distribution of the real machine in the simulation model, and output the adjusted action sequence after confirming that the smoothness, rhythm adaptation and mechanical execution safety meet the requirements.

[0118] In one implementation, when the control module 24 controls the legged robot to perform dance movements based on the adjusted action sequence, it is used to: perform dance movement execution tests on the legged robot in the order of single-movement debugging, segment debugging, and full-song debugging, and obtain test results of multiple consecutive executions of the adjusted action sequence; wherein, the test results indicate the consistency of movements, the accuracy of beat alignment, and the mechanical stability; The control module 24 is also used to: based on the real machine execution data and human feedback collected during the test, control the iterative optimization of the connection relationship information marked in the dance movement library, the parameters of the music feature extraction model, and the weight parameters of the movement sequence scoring network.

[0119] In one embodiment, when the control module 24 controls the legged robot to perform dance movements based on the adjusted action sequence, it is also used to perform dynamic beat compensation and movement adjustment during the execution process based on real-time music analysis and robot status feedback.

[0120] In one embodiment, the control module 24 is specifically used to: collect the robot's joint actual angles, angular velocities, motor currents, and execution timestamps in real time as robot status feedback signals; calculate the expected execution time window of the current target action based on the current music beat point, beat interval, and rhythm intensity continuously output through real-time music analysis; compare the expected execution time window of the target action with the robot's actual execution progress; if a progress deviation is detected to exceed a preset threshold, perform translation compensation on the start time of subsequent actions to be executed and perform time scaling on the joint motion curve to realign with the music beat; if the robot status feedback signal indicates abnormal joint load, unstable posture, or insufficient power, retrieve alternative actions from the dance action library that match the current music style, have lower mechanical wear, and are more stable to execute, and complete the action replacement and smooth posture transition before the next beat point.

[0121] For detailed implementation of each of the above modules, please refer to the description of the aforementioned methods; it will not be repeated here.

[0122] Reference Figure 3The diagram shown is a schematic representation of a device 300 according to an exemplary embodiment of this disclosure. The device 300 can be deployed on a legged robot, is the legged robot itself, a remote control device, or a server, etc. It may include a processor 310, a memory 320, and a bus 330. The memory 320 stores execution instructions and includes a main memory 321 and an external memory 322. The main memory 321, also called internal memory, is used to temporarily store computational data in the processor 310 and data exchanged with external memory 322 such as a hard disk. The processor 310 exchanges data with the external memory 322 through the main memory 321.

[0123] In this embodiment, the memory 320 is specifically used to store application code executing the scheme of this disclosure, and its execution is controlled by the processor 310. That is, when the electronic device 300 is running, the processor 310 communicates with the memory 320 through the bus 330, or the processor 310 communicates with the memory 320 through other means, so that the processor 310 executes the application code stored in the memory 320, thereby executing the steps of the robot dance motion control method described in any of the foregoing embodiments. The memory 320 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0124] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the robot dance motion control method described in any of the above embodiments. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as DVD-ROM, DVD-RAM, DVD-RW, DVD+RW, CD-ROM, CD-R (recordable) discs, CD-RW (rewritable) discs, MO (magneto-optical) storage media, etc.; and semiconductor storage media, such as flash memory, EEPROM, Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), etc.

[0125] The computer program can be written in various computer programming languages, including but not limited to C, C++, Python, and custom messages and services under the ROS framework. When the computer program is executed by the processor, it implements the various steps of the robot dance motion control method in the embodiments of this disclosure.

[0126] This disclosure also provides a computer program product storing a computer program. When executed by a processor, the computer program performs the steps of the robot dance motion control method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here. The computer program product can be implemented using hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and apparatuses described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for controlling robot dance movements, characterized in that, include: Acquire target music data, and generate a music feature vector based on the target music data; the music feature vector is used to indicate beat information, music style, and music segment division information. Based on the music feature vector, matching dance movements are retrieved from the dance movement library adapted to the legged robot to generate an initial movement sequence; the dance movement library contains multi-dimensional annotation information of the movements, including emotional semantic tags, duration information and connection relationship information; The initial action sequence is input into the action sequence optimization model trained through the imitation learning and reinforcement learning stages for optimization, and the optimized action sequence is determined. The reinforcement learning phase optimizes the strategy based on a multi-dimensional reward function that includes beat alignment reward, smooth transition reward, style matching reward, and mechanical feasibility reward. The optimized action sequence is transferred from the simulation environment to the real machine environment by using the domain adaptation loss function to determine the adjusted action sequence that adapts to the mechanical response characteristics of the real machine. Based on the adjusted action sequence, the legged robot is controlled to perform dance movements.

2. The method according to claim 1, characterized in that, The dance movement library is constructed through the following steps: Collect the basic sub-motion data and compound motion data of the legged robot, and record the joint angle change sequence information associated with each of the basic sub-motion data and compound motion data; Based on the joint angle change sequence information, the similarity between different actions is calculated; Action versions are filtered based on the similarity between the different actions, as well as the mechanical loss and execution stability information of the different actions; wherein, the same action version is associated with at least one of the basic sub-actions and / or composite actions; Obtain the multidimensional annotation information of the action version, associate the multidimensional annotation information with the basic sub-action data and / or compound action data corresponding to the action version, and the joint angle change sequence information, and write it into the dance action library.

3. The method according to claim 1 or 2, characterized in that, The emotional semantic tags include type tags corresponding to music styles, which quantify the joint movement rate and amplitude of the action. The duration information includes action execution duration information, start time information, and end time information; wherein, for fixed-duration actions, the action execution duration information includes a fixed duration; for adjustable-duration actions, the action duration information includes a duration range and a step interval; the start time information indicates the time taken for the joint to switch from the initial posture to the action posture, and the end time information indicates the time taken for the joint to return to the transition posture after the action ends; The connection relationship information includes at least one preceding connection action information and at least one following connection action information adapted to the current action: the preceding and following connection action information are determined based on the posture compatibility requirements, emotional consistency requirements, and duration matching requirements of adjacent actions. The posture compatibility requirements indicate that the joint angle difference between adjacent actions is less than a preset angle; the emotional consistency requirements indicate that actions with the same emotional tag are prioritized for connection; and the duration matching requirements indicate that the difference between the duration of the connection action and the duration of the original action is less than a preset duration.

4. The method according to claim 1, characterized in that, The beat information includes the beat point position and the number of beats per unit time; the generation of music feature vectors based on the target music data includes: The target music data is preprocessed to obtain a preprocessed audio signal, and frame-level audio features are extracted from the preprocessed audio signal. The frame-level audio features are input into the beat detector to determine the beat point position; Calculate the time interval between adjacent beat points based on the beat point positions, determine the number of beats per unit time, and label the beat strength attribute; The frame-level audio features are input into a music style classifier to determine the music style label; The frame-level audio features are input into the music segment recognition module, which identifies and outputs the start and end times of the verse, chorus, and interlude segments, generating music segment division information. The music feature vector is generated by fusing the beat point location, the number of beats per unit time, the beat intensity attribute, the music style tag, and the music segment division information.

5. The method according to claim 1, characterized in that, The step of retrieving matching dance movements from a dance movement library adapted to the legged robot based on the music feature vector and generating an initial movement sequence includes: Based on the music style and music segment division information indicated by the music feature vector, an initial motion package with corresponding emotional tags and adapted to each segment is selected from the dance motion library; Based on the number of beats per unit time indicated by the music feature vector, candidate actions with suitable action durations are selected from the initial action package; An initial movement sequence is generated based on the beat point position indicated by the music feature vector, the music segment division information, and the pre-annotated movement connection relationships in the dance movement library.

6. The method according to claim 1, characterized in that, The action sequence optimization model is trained using the following method: Obtain an expert demonstration dataset and train a basic action combination strategy through imitation learning. The imitation learning is used to learn the action connection logic and beat alignment method. Based on the aforementioned basic action combination strategy, a reinforcement learning model is constructed that includes a state space, an action space, and a multi-dimensional reward function. The action selection strategy is optimized through a joint iterative framework of policy optimization and value evaluation to obtain the action sequence optimization model.

7. The method according to claim 1, characterized in that, The step of controlling the legged robot to perform dance movements based on the adjusted action sequence includes: The posture feature sequences corresponding to the multiple sets of the adjusted action sequences and the music feature vectors are input into a scoring network based on convolutional neural networks and long short-term memory networks, and a quantized score is output. Based on the quantified score, the action sequence with the highest score that is greater than a set score threshold is selected from multiple sets of adjusted action sequences and determined as the target action sequence. Based on the target action sequence, the legged robot is controlled to perform dance movements.

8. The method according to claim 1, characterized in that, This also includes expanding the dance movement library in the following ways: Obtain the target music and style seed, and generate stylized action features based on the target music and style seed; Based on the stylized movement features and the basic movements in the dance movement library, stylized deformation and recombination are performed to generate dance sequences with specific performer style characteristics; The dance sequences with specific performer style characteristics are added to the dance movement library after being annotated in multiple dimensions.

9. The method according to claim 1, characterized in that, The optimized action sequence is transferred from the simulation environment to the real machine environment using a domain adaptation loss function to determine an adjusted action sequence that adapts to the mechanical response characteristics of the real machine, including: Based on the posture data of the legged robot performing actual actions and the domain adaptation loss function, the action sequence optimization model is trained for domain adaptation. Based on the action sequence optimization model trained by domain adaptation, the optimized action sequence is re-optimized to generate an adjusted action sequence that adapts to the action distribution of the real machine. The adjusted motion sequence adapted to the actual machine motion distribution is verified in the simulation model. After confirming that the smoothness, rhythm adaptation and mechanical execution safety meet the requirements, the adjusted motion sequence is output.

10. The method according to claim 1, characterized in that, The step of controlling the legged robot to perform dance movements based on the adjusted action sequence includes: The legged robot was tested for dance movements in the order of single-action debugging, segment debugging, and full-sequence debugging, and the test results of the adjusted movement sequence were obtained by executing it multiple times in succession; wherein, the test results indicate the consistency of movement, the accuracy of beat alignment, and the mechanical stability. The method further includes: Based on the real machine execution data and human feedback collected during the test, the connection relationship information marked in the dance movement library, the parameters of the music feature extraction model, and the weight parameters of the movement sequence scoring network are iteratively optimized.

11. The method according to claim 1, characterized in that, The step of controlling the legged robot to perform dance movements based on the adjusted action sequence includes: Based on the adjusted action sequence, the legged robot is controlled to perform dance movements, and dynamic beat compensation and movement adjustment are performed during the execution based on real-time music analysis and robot status feedback.

12. The method according to claim 11, characterized in that, The process of dynamically compensating for beats and adjusting movements based on real-time music analysis and robot status feedback during execution includes: The robot's joint angles, angular velocities, motor currents, and execution timestamps are collected in real time and used as robot status feedback signals. Based on the current music beat point, beat interval, and rhythm intensity continuously output through real-time music analysis, the expected execution time window of the current target action is calculated. The expected execution time window of the target action is compared with the actual execution progress of the robot. If the progress deviation is detected to exceed the preset threshold, the start time of the subsequent action to be executed is translated and compensated, and the joint motion curve is scaled in time to realign with the music beat. If the robot's status feedback signal indicates abnormal joint load, unstable posture, or insufficient power, it retrieves an alternative movement from the dance movement library that matches the current music style, has lower mechanical wear, and is more stable to execute, and completes the movement replacement and smooth posture transition before the next beat.

13. A robot dance motion control device, characterized in that, include: The music parsing module is used to acquire target music data and generate music feature vectors based on the target music data. The music feature vector is used to indicate beat information, music style, and music segment division information; The dance movement retrieval module is used to retrieve matching dance movements from the dance movement library adapted to the legged robot based on the music feature vector, and generate an initial movement sequence; the dance movement library contains multi-dimensional annotation information of the movements, including emotional semantic tags, duration information and connection relationship information; The action sequence optimization module is used to input the initial action sequence into the action sequence optimization model trained by the imitation learning stage and the reinforcement learning stage for optimization, and to determine the optimized action sequence. The reinforcement learning phase optimizes the strategy based on a multi-dimensional reward function that includes beat alignment reward, smooth transition reward, style matching reward, and mechanical feasibility reward; the optimized action sequence is then transferred from the simulation environment to the real machine environment using a domain adaptation loss function to determine an adjusted action sequence that adapts to the mechanical response characteristics of the real machine. The control module is used to control the legged robot to perform dance movements based on the adjusted action sequence.

14. A device, characterized in that, It includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions to perform the steps of the robot dance motion control method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the robot dance motion control method as described in any one of claims 1 to 12.

Images