A dance performance method and device of a mechanical arm, an electronic device and a storage medium

Abstract

The application provides a dance performance method and device of a mechanical arm, electronic equipment and a storage medium, and relates to the technical field of robot control. In the application, a motion parameter set of the mechanical arm at a current time is obtained. The motion parameter set can include at least one motion parameter corresponding to each joint of the mechanical arm at the current time. A plurality of motion parameters included in the motion parameter set are subjected to feature analysis to obtain at least one motion feature. Based on a preset corresponding relationship between the motion feature and music parameters, a plurality of music parameters corresponding to the at least one motion feature are determined, and background music corresponding to the motion parameter set is generated based on the plurality of music parameters. The mechanical arm is instructed to perform a dance action corresponding to the motion parameter set under the background music. Therefore, the matching accuracy and flexibility between the dance action of the mechanical arm and the background music are improved.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method, device, electronic device and storage medium for a robotic arm to perform dance. Background Technology

[0002] With the rapid development of robotics technology, robotic arms, as a common basic form of robot, are widely used in many areas of daily life. Among them, robotic arm dance, as a novel performance form, has also attracted widespread attention.

[0003] Currently, the robotic arm's dance performance can be made more visually appealing and artistic by pre-selecting background music that matches the dance moves it is about to perform, or by compiling the robotic arm's dance moves based on the selected background music.

[0004] However, the aforementioned method of using robotic arms for dance performance makes it difficult to ensure a precise match between the dance movements and the background music during the performance; furthermore, since the background music or the robotic arm's dance movements are predetermined, the flexibility of robotic arm dance performances is also relatively low. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for a robotic arm to perform dance, thereby improving the accuracy of matching between the robotic arm's dance movements and background music, and enhancing the flexibility of the robotic arm's dance performance.

[0006] In a first aspect, embodiments of this application provide a method for a robotic arm to perform a dance, the method comprising:

[0007] Obtain the set of motion parameters of the robotic arm at the current moment; the set of motion parameters includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment;

[0008] Feature analysis is performed on multiple motion parameters included in the set of motion parameters to obtain at least one motion feature;

[0009] Based on the pre-defined correspondence between motion features and background music parameters, multiple background music parameters corresponding to at least one motion feature are determined, and background music corresponding to the set of motion parameters is generated based on the multiple background music parameters.

[0010] The robotic arm is instructed to perform dance moves corresponding to a set of motion parameters to background music.

[0011] In one optional embodiment, the set of motion parameters includes at least one of the following motion parameters: joint motion action, joint motion speed, joint motion amplitude, and joint motion frequency; wherein, the motion action includes: joint posture information and motion trajectory.

[0012] In one optional embodiment, at least one motion feature includes any one or a combination of the robot arm's motion complexity, motion rhythm, and motion style; wherein, motion rhythm characterizes the variation pattern of the rotational angular velocity corresponding to each joint.

[0013] In one optional embodiment, feature analysis is performed on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, including:

[0014] The rotation angle, rotational angular velocity, and rotational angular acceleration of each joint at the current moment are determined based on multiple motion parameters.

[0015] The motion complexity of the robotic arm is determined based on the change in angle, angular velocity, and angular acceleration of each joint at the current moment. Each angle change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle of the previous historical moment adjacent to the current moment.

[0016] In one optional embodiment, feature analysis is performed on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, including:

[0017] The rotational angular velocity of each joint at the current moment is determined based on multiple motion parameters, and the rotational angular velocity of at least one historical moment adjacent to the current moment is obtained.

[0018] Based on the multiple rotational angular velocities corresponding to each joint, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0019] Based on the obtained autocorrelation coefficients, the movement rhythm of the robotic arm is determined.

[0020] In one optional embodiment, feature analysis is performed on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, including:

[0021] From multiple motion parameters, obtain the motion actions corresponding to each joint;

[0022] The movement movements corresponding to each joint are classified to determine the movement style of the robotic arm.

[0023] In one alternative embodiment, multiple music parameters include: the instruments, style, melody, rhythm, and chords used in the background music.

[0024] Secondly, embodiments of this application also provide a robotic arm dance performance device, the device comprising:

[0025] The data acquisition module is used to acquire the set of motion parameters of the robotic arm at the current moment; the set of motion parameters includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment;

[0026] The feature analysis module is used to perform feature analysis on multiple motion parameters included in the set of motion parameters to obtain at least one motion feature;

[0027] The music generation module is used to determine multiple music parameters corresponding to at least one motion feature based on the preset correspondence between motion features and music parameters, and to generate background music corresponding to the set of motion parameters based on the multiple music parameters.

[0028] The dance performance module is used to instruct the robotic arm to perform dance movements corresponding to a set of motion parameters to background music.

[0029] In one optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module is specifically used for:

[0030] The rotation angle, rotational angular velocity, and rotational angular acceleration of each joint at the current moment are determined based on multiple motion parameters.

[0031] The motion complexity of the robotic arm is determined based on the change in angle, angular velocity, and angular acceleration of each joint at the current moment. Each angle change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle of the previous historical moment adjacent to the current moment.

[0032] In one optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module is specifically used for:

[0033] The rotational angular velocity of each joint at the current moment is determined based on multiple motion parameters, and the rotational angular velocity of at least one historical moment adjacent to the current moment is obtained.

[0034] Based on the multiple rotational angular velocities corresponding to each joint, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0035] Based on the obtained autocorrelation coefficients, the movement rhythm of the robotic arm is determined.

[0036] In one optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module is specifically used for:

[0037] Obtain the motion actions corresponding to each joint from multiple motion parameters;

[0038] The movement movements corresponding to each joint are classified to determine the movement style of the robotic arm.

[0039] Thirdly, embodiments of this application also provide an electronic device, including:

[0040] Processor; and

[0041] Stored program memory,

[0042] The program includes instructions that, when executed by the processor, cause the processor to perform the robotic arm's dance performance method as described in the first aspect.

[0043] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the dance performance method of the robotic arm as described in the first aspect.

[0044] Fifthly, this application provides a computer program product that, when invoked by a computer, causes the computer to execute the steps of the robotic arm dance performance method as described in the first aspect.

[0045] The beneficial effects of this application are as follows:

[0046] In the robotic arm dance performance method provided in this application embodiment, after obtaining the set of motion parameters of the robotic arm at the current moment, feature analysis can be performed on at least one motion parameter corresponding to each joint of the robotic arm at the current moment, including the set of motion parameters, to obtain at least one motion feature. Then, based on the preset correspondence between motion features and background music parameters, multiple background music parameters corresponding to at least one motion feature can be determined, and background music corresponding to the set of motion parameters can be generated based on the multiple background music parameters. Finally, the robotic arm is instructed to perform the dance movements corresponding to the set of motion parameters to the background music. This approach improves upon the problems in related technologies where pre-selecting background music to match the dance movements to be performed by the robotic arm, or compiling the robotic arm's dance movements based on selected background music, leads to low matching accuracy between the robotic arm's dance movements and background music, and low flexibility in the robotic arm's dance performance. This method improves the matching accuracy between the robotic arm's dance movements and background music, and the flexibility in the robotic arm's dance performance.

[0047] Furthermore, other features and advantages of this application will be set forth in the following description and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described herein are used to provide a further understanding of this application, constitute a part of this application, and do not constitute an improper limitation of this application. In the accompanying drawings:

[0049] Figure 1 This is a schematic diagram illustrating an optional application scenario provided in an embodiment of this application;

[0050] Figure 2 A schematic diagram illustrating the implementation process of a robotic arm dance performance method provided in this application embodiment;

[0051] Figure 3 A logic diagram illustrating the determination of the motion complexity of a robotic arm, provided as an embodiment of this application;

[0052] Figure 4 A logic diagram illustrating the determination of the movement rhythm of a robotic arm, provided for an embodiment of this application;

[0053] Figure 5 A schematic diagram of an optional dance performance system provided for an embodiment of this application;

[0054] Figure 6 A schematic diagram of the structure of a robotic arm dance performance device provided in an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0057] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0058] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0059] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0060] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0061] First, the design concept of the embodiments of this application will be briefly introduced below:

[0062] With the rapid development of robotics technology, robots are gradually moving from traditional industrial applications to service, entertainment, and other fields that are closer to people's daily lives. Among these applications, the dance performances of robotic arms are attracting increasing attention.

[0063] In related technologies, the background music for the robotic arm's dance performance usually needs to be pre-selected and edited; or, the robotic arm's dance movements are compiled based on the selected background music.

[0064] However, using the above method makes it difficult to ensure a precise match between the dance movements and the background music during the robotic arm's dance performance; furthermore, the cost of compiling the robotic arm's dance movements based on the selected background music is high, and since the background music or the robotic arm's dance movements are predetermined, this also results in low flexibility for the robotic arm's dance performance.

[0065] In view of this, in order to improve or solve the above problems, this application proposes a method for a robotic arm to perform a dance. Specifically, it may include: acquiring a set of motion parameters of the robotic arm at the current moment; the set of motion parameters includes at least one motion parameter corresponding to each joint of the robotic arm at the current moment; then, performing feature analysis on the multiple motion parameters included in the set of motion parameters to obtain at least one motion feature; further, based on a preset correspondence between the motion features and background music parameters, determining multiple background music parameters corresponding to the at least one motion feature, and generating background music corresponding to the set of motion parameters based on the multiple background music parameters; finally, instructing the robotic arm to perform the dance movements corresponding to the set of motion parameters to the background music. Using this method, after determining the motion features of the robotic arm based on the motion parameters of each joint of the robotic arm at the current moment, multiple background music parameters matching the motion features of the robotic arm can be determined by combining the correspondence between the motion features and background music parameters. Thus, generating background music corresponding to the set of motion parameters based on the obtained multiple background music parameters, and then instructing the robotic arm to perform the dance movements corresponding to the set of motion parameters to the background music, not only improves the matching accuracy between the robotic arm's dance movements and the background music, but also improves the flexibility of the robotic arm's dance performance.

[0066] In particular, the preferred embodiments of this application will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments of this application and the features in the embodiments can be combined with each other without conflict.

[0067] See Figure 1 As shown, this is a schematic diagram of an optional application scenario provided by an embodiment of this application. The application scenario may include: a terminal device 101, a server 102, and a robotic arm 103. The terminal device 101 and the server 102 can all interact with the robotic arm 103 through a communication network. The communication network may employ wireless communication or wired communication.

[0068] For example, terminal device 101 can access the network via cellular mobile communication technology to communicate with server 102 and robotic arm 103. The cellular mobile communication technology may include, for example, 5G (5th generation mobile networks) or next-generation mobile communication technology. Optionally, terminal device 101 can access the network via short-range wireless communication to communicate with server 102 and robotic arm 103. The short-range wireless communication method may include, for example, Wi-Fi (wireless fidelity) technology.

[0069] This application embodiment does not limit the number of communication devices involved in the above application scenarios. For example, the above application scenarios may include more terminal devices, or may not include terminal devices, or may include other network devices. Figure 1 As shown, only terminal device 101, server 102 and robotic arm 103 are described as examples. The following is a brief introduction to each of the above communication devices and their respective functions.

[0070] Terminal device 101 is a device that can provide voice and / or data connectivity to a user, and may be a device that supports wired and / or wireless connections. For example, terminal device 101 may include, but is not limited to: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0071] Furthermore, a related client can be installed on the terminal device 101. This client can be software, such as an application (APP), browser, short video software, etc., or it can be a webpage, mini-program, etc. It should be noted that the terminal device 101 in this embodiment can enable the aforementioned client related to the robotic arm's dance performance to send the set of motion parameters of the robotic arm 103 to the server 102 for subsequent steps such as the robotic arm's dance performance.

[0072] It is understood that the motion data of the robotic arm 103 can be obtained by collecting multiple sensors deployed on the terminal device 101. Of course, the motion data of the robotic arm 103 can also be obtained by collecting multiple sensors deployed on the robotic arm 103. This application embodiment does not specifically limit this.

[0073] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0074] It is worth mentioning that, in this embodiment of the application, the server 102 can be used to obtain the set of motion parameters of the robotic arm at the current moment; then, feature analysis is performed on the multiple motion parameters included in the set of motion parameters to obtain at least one motion feature; further, based on the preset correspondence between the motion features and the background music parameters, multiple background music parameters corresponding to at least one motion feature are determined, and background music corresponding to the set of motion parameters is generated based on the multiple background music parameters; finally, the robotic arm is instructed to perform the dance movements corresponding to the set of motion parameters under the background music.

[0075] The robotic arm 103 is a mechatronic device that simulates the functions of a human arm and can perform various tasks such as grasping, handling, and manipulation. It typically consists of a mechanical structure, a drive system, a control system, and sensors. This application does not specifically limit the type of robotic arm 103, such as... Figure 1 The robotic arm 103 shown is only one example.

[0076] The following describes the robotic arm dance performance method provided by the exemplary embodiments of this application in conjunction with the above application scenarios and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0077] See Figure 2 The diagram shown illustrates the implementation flow of a robotic arm dance performance method provided in this application embodiment. Taking a server as an example, the specific implementation flow of this method is as follows:

[0078] S201: Obtain the set of motion parameters of the robotic arm at the current moment.

[0079] For example, during step S201, the terminal device can send a set of motion parameters of the robotic arm collected by multiple sensors at the current moment to the server. The aforementioned set of motion parameters may include at least one motion parameter corresponding to each joint of the robotic arm at the current moment.

[0080] It should be noted that among the aforementioned sensors, different types of sensors can be used to collect different types of motion parameters, and each motion parameter can correspond to one or more sensors. This application embodiment does not specifically limit this. Optionally, the sensor's collection of the robotic arm's motion parameters can be automatically triggered after the robotic arm begins a certain movement.

[0081] The aforementioned set of motion parameters includes at least one of the following motion parameters: joint movement, joint speed, joint range of motion, and joint frequency. The movement can include joint posture information and movement trajectory, while the joint range of motion is also known as the joint's range of motion, and the joint frequency is also known as the joint's movement frequency.

[0082] To improve the accuracy of subsequent motion feature extraction, the server can preprocess the acquired set of motion parameters (i.e., the motion data of the robotic arm collected by multiple sensors). Therefore, in one optional implementation, after acquiring the set of motion parameters of the robotic arm at the current moment, the server can also perform data preprocessing on the set of motion parameters to obtain a preprocessed set of motion parameters.

[0083] It is understood that the data preprocessing operations described above may include, but are not limited to, filtering and / or denoising. For example, the server may use a low-pass filter (e.g., a Butterworth filter) to filter the acquired set of motion parameters, removing high-frequency noise. As another example, the server may use wavelet transform or similar methods to remove random noise from the set of motion parameters.

[0084] Taking a robotic arm comprising n joints, and considering the rotation angle, angular velocity, and angular acceleration of each of the n joints at time t (i.e., the current time), determined by a set of motion parameters, as an example, the data matrix of the aforementioned rotation angle, angular velocity, and angular acceleration of the n joints at time t can be specifically represented as follows:

[0085]

[0086] Where D(t) represents the data matrix of rotation angle, rotational angular velocity, and rotational angular acceleration of n joints at time t without data preprocessing, θ i (t) represents the rotation angle of the i-th joint out of n joints at time t before data preprocessing, ω i (t) represents the rotational angular velocity of the i-th joint out of n joints at time t before data preprocessing, θ i (t) represents the rotational angular acceleration of the i-th joint among n joints at time t without data preprocessing, i = 1, 2, 3, ..., n.

[0087] Therefore, the data matrix of rotation angle, rotational angular velocity, and rotational angular acceleration corresponding to the n joints at time t, determined based on the preprocessed set of motion parameters, can be specifically represented as follows:

[0088]

[0089] Where D′(t) represents the preprocessed data matrix corresponding to data matrix D(t), and θ i ′ (t) represents the rotation angle θ i (t) corresponds to the rotation angle after data preprocessing, ω i ′ (t) represents the rotational angular velocity ω i (t) corresponds to the preprocessed rotational angular velocity, α i ′ (t) represents the angular acceleration α. i (t) corresponds to the rotational angular acceleration after data preprocessing.

[0090] S202: Perform feature analysis on multiple motion parameters included in the set of motion parameters to obtain at least one motion feature.

[0091] The aforementioned motion characteristics may include any one or a combination of the following: the motion complexity, motion rhythm, and motion style of the robotic arm. Motion rhythm can characterize the variation pattern of the rotational angular velocity corresponding to each joint in the robotic arm, while motion style refers to the stylistic characteristics of the robotic arm's movements (e.g., a smooth style).

[0092] Therefore, after obtaining the set of motion parameters of the robotic arm at the current moment, the server can extract and analyze the three feature dimensions of motion complexity, motion rhythm and motion style of the robotic arm based on the multiple motion parameters included in the set of motion parameters, thereby obtaining the motion complexity, motion rhythm and motion style of the robotic arm.

[0093] It is understood that at least one of the above-mentioned motion features may also include other types of motion features, and the embodiments of this application do not specifically limit them.

[0094] In one alternative implementation, see [link to relevant documentation]. Figure 3 As shown, when executing step S202, after obtaining the set of motion parameters, the server can determine the rotation angle, rotational angular velocity, and rotational angular acceleration of each joint at the current moment based on the multiple motion parameters included in the set of motion parameters. Thus, based on the change in angle, rotational angular velocity, and rotational angular acceleration of each joint at the current moment, the motion complexity of the robotic arm can be determined.

[0095] Each angle change can be determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle at the previous historical moment adjacent to the current moment. For example, the formula for calculating the angle change can be specifically expressed as follows:

[0096] Δθ i ′(t)=|θ i ′ (t)-θ i ′ (t-1)|

[0097] Where, Δθ i ′ (t) represents the change in angle between the rotation angle of the i-th joint at time t (i.e., the current time) and the rotation angle at the previous historical time (i.e., time t-1) adjacent to time t, θ i ′ (t) represents the rotation angle of the i-th joint at time t, θ i ′ (t-1) represents the rotation angle of the i-th joint at time t-1.

[0098] Since the motion complexity of the robotic arm can be quantified based on the angular change, angular velocity, and angular acceleration of each joint at the current moment, the aforementioned formula for quantifying the motion complexity of the robotic arm can be expressed as follows:

[0099]

[0100] Where C(t) represents the motion complexity of the robotic arm at time t (i.e., the current time), |θ i ′ (t)-θ i ′ (t-1)| represents the absolute value of the angular change between the rotation angle of the i-th joint at time t and the rotation angle at time t-1 adjacent to time t, |ω i ′ (t)| represents the absolute value of the rotational angular velocity of the i-th joint at time t, |α i ′ (t)| represents the absolute value of the rotational angular acceleration of the i-th joint at time t, and n represents the number of joints in the robotic arm.

[0101] In one alternative implementation, after obtaining the set of motion parameters, the server can further determine the motion rhythm of the robotic arm by analyzing the variation patterns of the rotational angular velocities of each joint, i.e., by performing autocorrelation analysis on the rotational angular velocities. For example, see [link to relevant documentation]. Figure 4 As shown, the server can determine the rotational angular velocity of each joint at the current moment based on multiple motion parameters included in the motion parameter set, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment. Based on the multiple rotational angular velocities corresponding to each joint, the server can obtain the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment, and then determine the motion rhythm of the robotic arm based on the obtained autocorrelation coefficients.

[0102] To reduce computational complexity without compromising the accuracy of motion rhythm determination, the robotic arm's motion rhythm can be determined based on one or more autocorrelation coefficients corresponding to any one of the aforementioned joints. Therefore, the quantification formula for the robotic arm's motion rhythm can be specifically expressed as follows:

[0103]

[0104] Where R(t) represents the movement rhythm of the robotic arm at time t (i.e., the current time), N represents the size of the autocorrelation window, that is, the number of historical moments adjacent to the current time, and ω o ′ (t) represents the rotational angular velocity of the i-th joint at time t, ω o ′ (tk) represents the rotational angular velocity of the i-th joint at time tk.

[0105] Based on the above method, the server extracts periodic information by calculating the autocorrelation coefficient of the joint rotational angular velocities during the process of determining the robotic arm based on rotational angular velocity. Furthermore, the server can also obtain the frequency domain information of the rotational angular velocity by performing a Fourier transform on the joint rotational angular velocity.

[0106] In one optional implementation, after obtaining the set of motion parameters, the server can also extract the motion actions corresponding to each joint from multiple motion parameters, thereby classifying the motion actions corresponding to each joint and determining the motion style of the robotic arm. Optionally, the motion style can be classified using machine learning algorithms, such as support vector machines (SVM) or random forests, to classify the robotic arm's motion actions and identify different motion styles, such as smooth, stiff, rotating, and swinging styles. For example, the motion style of the robotic arm at time t obtained through the classification algorithm can be represented as S(t).

[0107] S203: Based on the preset correspondence between motion features and background music parameters, determine multiple background music parameters corresponding to at least one motion feature, and generate background music corresponding to the set of motion parameters based on the multiple background music parameters.

[0108] For example, the aforementioned multiple background music parameters may include, but are not limited to, the instruments, style, melody, rhythm, and chords used in the background music. In this case, the correspondence between the aforementioned preset motion features and background music parameters is also the correspondence between the motion features and the instruments, style, melody, rhythm, and chords used in the background music.

[0109] Taking the motion characteristics of a robotic arm, including motion complexity, motion rhythm, and motion style, as an example, the server can select the appropriate instrument type based on the robotic arm's motion style S(t). For example, a smooth style can be selected from strings, piano, etc.; a rigid style from drums, bass, etc.; a spinning style from electronic synthesizers, etc.; and a swinging style from guitar, harmonica, etc.

[0110] The server can also select the style of background music (i.e., background music) based on the robotic arm's motion style S(t) and motion complexity C(t). For example, a lyrical style is selected for low motion complexity and a smooth motion style; a rock style is selected for high motion complexity and a rigid style; and an electronic dance music style is selected for medium motion complexity and a rotating style.

[0111] The server can also generate background music melodies based on the movement rhythm R(t) and action complexity C(t), combined with preset music melody generation methods (e.g., preset melody generation rules). For example, if the movement rhythm is fast, the melody has large pitch variations and a dense rhythm; if the movement rhythm is slow, the melody has small pitch variations and a sparse rhythm; if the action complexity is high, the melody is complex and contains more dissonant intervals; if the action complexity is low, the melody is simple and contains more consonant intervals. This application does not specifically limit this aspect.

[0112] For example, the server can also determine the rhythm of the background music based on the movement rhythm R(t) of the robotic arm. For instance, if the movement rhythm is fast, a higher tempo can be set, measured in beats per minute (BPM); if the movement rhythm is slow, a lower tempo can be set. Furthermore, the server can select an appropriate chord progression and generate chords based on the generated melody and the movement style S(t). For example, if the movement style S(t) is flowing, a major chord progression can be used; if the movement style S(t) is rigid, a minor chord progression can be used; and if the movement style S(t) is rotatable, a dissonant chord progression can be used.

[0113] Furthermore, after determining the multiple musical parameters corresponding to at least one of the aforementioned motion features, the server can generate background music corresponding to the set of motion parameters based on these multiple musical parameters. For example, the server can synthesize the final musical audio, i.e., the background music, based on the determined multiple musical parameters and pre-set music synthesis software or algorithms (such as musical instrument digital interface (MIDI) synthesizers, audio plugins, etc.).

[0114] S204: Instructs the robotic arm to perform dance movements corresponding to the set of motion parameters to background music.

[0115] For example, during step S204, the server can achieve synchronized output of the generated background music and the robotic arm's dance movements through timestamp synchronization or real-time synchronization. Timestamp synchronization involves assigning a timestamp to each music event during background music generation and matching it with the timestamp of the robotic arm's dance movements. Real-time synchronization utilizes a real-time operating system to achieve real-time synchronization between the background music and the robotic arm's dance movements.

[0116] In one alternative implementation, see [link to relevant documentation]. Figure 5 The diagram shows a system architecture schematic of an optional dance performance system provided in this application embodiment. The dance performance system may include: a robotic arm motion data acquisition module, a data preprocessing module, a motion feature extraction and analysis module, a music parameter generation module, a music synthesis module, and a synchronization output module. The robotic arm motion data acquisition module is responsible for real-time acquisition of motion data or parameters of each joint of the robotic arm, including joint angles, angular velocities, and angular accelerations. The data preprocessing module performs filtering, noise reduction, and other data preprocessing operations on the acquired robotic arm motion data to improve the accuracy and reliability of the data. The motion feature extraction and analysis module can extract key information that characterizes the motion features of the robotic arm from the preprocessed motion data, such as motion complexity, rhythm, and style. The music parameter generation module can generate corresponding music parameters based on the extracted robotic arm motion features, including instrument selection, style determination, melody generation, rhythm setting, and chord arrangement. The music synthesis module can synthesize the final music audio based on the generated music parameters. The synchronization output module can output the generated music audio synchronously with the robotic arm's dance movements.

[0117] It is understood that the embodiments of this application do not specifically limit the types and number of modules included in the dance performance system. For example... Figure 5 As shown, the dance performance system may also include a user interaction module and a system control module. The user interaction module provides an interface for users to interact with the system, allowing them to set up, control, and monitor the system. The system control module manages the entire operation of the dance performance system and coordinates the work between the various modules.

[0118] Therefore, based on the robotic arm dance performance method described in steps S201 to S204 above, the precise matching of music and robotic arm dance movements is achieved, enhancing the viewing experience and artistry of the robotic arm dance performance; by extracting and analyzing the features of the robotic arm motion parameters, diverse and varied background music can be generated according to different motion characteristics; by combining artificial intelligence technology with music generation technology, the intelligent and automated robotic arm dance performance is realized.

[0119] In summary, in the robotic arm dance performance method provided in this application embodiment, after obtaining the set of motion parameters of the robotic arm at the current moment, feature analysis can be performed on at least one motion parameter corresponding to each joint of the robotic arm at the current moment, including the set of motion parameters, to obtain at least one motion feature. Then, based on the preset correspondence between the motion features and background music parameters, multiple background music parameters corresponding to at least one motion feature can be determined, and background music corresponding to the set of motion parameters can be generated based on the multiple background music parameters. Finally, the robotic arm is instructed to perform the dance movements corresponding to the set of motion parameters to the background music. This approach improves upon the problems in related technologies where pre-selecting background music to match the dance movements to be performed by the robotic arm, or compiling the robotic arm's dance movements based on selected background music, leads to low matching accuracy between the robotic arm's dance movements and background music, and low flexibility in the robotic arm's dance performance. This method improves the matching accuracy between the robotic arm's dance movements and background music, and enhances the flexibility of the robotic arm's dance performance.

[0120] Furthermore, based on the same technical concept, embodiments of this application provide a robotic arm dance performance device, which is used to implement the above-described method flow of the embodiments of this application. For example, see [link to relevant documentation]. Figure 6 As shown, the robotic arm's dance performance device 600 may include: a data acquisition module 601, a feature analysis module 602, a music generation module 603, and a dance performance module 604, wherein:

[0121] The data acquisition module 601 is used to acquire the set of motion parameters of the robotic arm at the current moment; the set of motion parameters includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment;

[0122] The feature analysis module 602 is used to perform feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature;

[0123] The music generation module 603 is used to determine multiple music parameters corresponding to at least one motion feature based on the preset correspondence between motion features and music parameters, and to generate background music corresponding to the set of motion parameters based on the multiple music parameters.

[0124] The dance performance module 604 is used to instruct the robotic arm to perform dance movements corresponding to a set of motion parameters to background music.

[0125] In an optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module 602 is specifically used for:

[0126] The rotation angle, rotational angular velocity, and rotational angular acceleration of each joint at the current moment are determined based on multiple motion parameters.

[0127] The motion complexity of the robotic arm is determined based on the change in angle, angular velocity, and angular acceleration of each joint at the current moment. Each angle change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle of the previous historical moment adjacent to the current moment.

[0128] In an optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module 602 is specifically used for:

[0129] The rotational angular velocity of each joint at the current moment is determined based on multiple motion parameters, and the rotational angular velocity of at least one historical moment adjacent to the current moment is obtained.

[0130] Based on the multiple rotational angular velocities corresponding to each joint, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0131] Based on the obtained autocorrelation coefficients, the movement rhythm of the robotic arm is determined.

[0132] In an optional embodiment, when performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm, the feature analysis module 602 is specifically used for:

[0133] From multiple motion parameters, obtain the motion actions corresponding to each joint;

[0134] The movement movements corresponding to each joint are classified to determine the movement style of the robotic arm.

[0135] Based on the description of the method and apparatus embodiments above, an exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method according to an embodiment of the present invention.

[0136] This application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this application.

[0137] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this application.

[0138] See Figure 7 The diagram shown below illustrates the structure of an electronic device 700 that can serve as a server or client in this application, and is an example of a hardware device that can be applied to various aspects of this application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0139] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0140] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth devices, WiFi devices, worldwide interoperability for microwave access (WiMax) devices, cellular communication devices, and / or the like.

[0141] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the above-described robotic arm dance performance method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the above-described robotic arm dance performance method by any other suitable means (e.g., by means of firmware).

[0142] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM) or flash memory, optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device, PLD) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0147] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0148] Furthermore, it should be understood that the above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of this invention are still within the scope of this application.

Claims

1. A method of dance performance of a robot arm, characterized by, The method comprises: acquiring a motion parameter set of a mechanical arm at a current time; the motion parameter set comprises at least one motion parameter corresponding to each joint of the mechanical arm at the current time; performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature; the at least one motion feature comprises any one or combination of motion complexity, motion rhythm and motion style of the mechanical arm; wherein the motion rhythm represents a variation law of a rotation angular velocity of each joint corresponding to the current time; the feature analysis on the plurality of motion parameters included in the motion parameter set to obtain the at least one motion feature of the mechanical arm comprises: determining the rotation angular velocity of each joint at the current time based on the plurality of motion parameters, and acquiring the rotation angular velocity at at least one historical time adjacent to the current time; obtaining the autocorrelation coefficient of the rotation angular velocity of each joint at the current time based on the plurality of rotation angular velocities corresponding to each joint; and determining the motion rhythm of the mechanical arm based on the obtained autocorrelation coefficients of each joint; determining a plurality of music parameters corresponding to the at least one motion feature based on a preset corresponding relationship between motion features and music parameters, and generating background music corresponding to the motion parameter set based on the plurality of music parameters; instructing the mechanical arm to perform a dance action corresponding to the motion parameter set under the background music.

2. The method of claim 1, wherein, The motion parameter set comprises at least one of the following motion parameters: motion action of a joint, motion speed of a joint, motion amplitude of a joint, and motion frequency of a joint; wherein the motion action comprises posture information and a motion trajectory of a joint.

3. The method of claim 1, wherein, The feature analysis on the plurality of motion parameters included in the motion parameter set to obtain the at least one motion feature of the mechanical arm further comprises: determining the rotation angle, rotation angular velocity and rotation angular acceleration of each joint corresponding to the current time based on the plurality of motion parameters; determining the motion complexity of the mechanical arm based on the angle change amount, rotation angular velocity and rotation angular acceleration of the rotation angle of each joint corresponding to the current time; wherein each angle change amount is determined according to the rotation angle of the corresponding joint at the current time and the rotation angle at the last historical time adjacent to the current time.

4. The method of claim 1, wherein, The feature analysis on the plurality of motion parameters included in the motion parameter set to obtain the at least one motion feature of the mechanical arm further comprises: acquiring the motion action corresponding to each joint from the plurality of motion parameters; performing action classification on the motion action corresponding to each joint to determine the motion style of the mechanical arm.

5. The method of claim 1 or 2, wherein, The plurality of music parameters comprise: an instrument, style, melody, rhythm and chord of the background music.

6. A dancing performance device of a robot arm, characterized by, The method comprises: a data acquisition module for acquiring a motion parameter set of a mechanical arm at a current time; the motion parameter set comprises at least one motion parameter corresponding to each joint of the mechanical arm at the current time; The feature analysis module is configured to perform feature analysis on the plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the mechanical arm performing the dance performance at the current time. The at least one motion feature includes any one or combination of motion complexity, motion rhythm, and motion style of the mechanical arm. The motion rhythm represents a variation law of the rotation angular velocity of each joint. The feature analysis on the plurality of motion parameters included in the motion parameter set to obtain the at least one motion feature of the mechanical arm includes determining the rotation angular velocity of each joint at the current time based on the plurality of motion parameters, and obtaining the rotation angular velocity at at least one historical time adjacent to the current time. The motion rhythm is obtained based on the plurality of rotation angular velocities of each joint, and the autocorrelation coefficient of the rotation angular velocity of each joint at the current time is obtained based on the plurality of rotation angular velocities of each joint. The motion rhythm of the mechanical arm is determined based on the obtained autocorrelation coefficients. The music generation module is configured to determine a plurality of music parameters corresponding to the at least one motion feature based on a preset corresponding relationship between the motion feature and the music parameters, and generate background music corresponding to the motion parameter set based on the plurality of music parameters. The dance performance module is configured to instruct the mechanical arm to perform a dance action corresponding to the motion parameter set under the background music.

7. An electronic device comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-5.

8. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method of any one of claims 1-5.

Images