Dance performance method and device of mechanical arm, electronic equipment and storage medium

Abstract

The invention provides a dance performance method and device of a mechanical arm, electronic equipment and a storage medium, and relates to the technical field of robot control. According to the method, a motion parameter set of the mechanical arm at the current moment is obtained; the motion parameter set can comprise at least one motion parameter corresponding to each joint of the mechanical arm at the current moment; performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature; on the basis of a preset corresponding relation between the motion features and the incidental music parameters, multiple incidental music parameters corresponding to the at least one motion feature are determined, and background music corresponding to the motion parameter set is generated on the basis of the multiple incidental music parameters; and the mechanical arm is indicated to perform the dance movement corresponding to the movement parameter set under the background music, so that the matching accuracy and flexibility between the dance movement of the mechanical arm and the background music are improved.

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and in particular to a dance performance method, device, electronic equipment and storage medium for a robot arm. Background Art

[0002] With the rapid development of robotics technology, robotic arms, as a common basic form of robots, have been widely used in many fields of daily life. Among them, robotic arm dance, as a novel performance form, has also attracted widespread attention.

[0003] Currently, the dance performance of the robotic arm can be made more ornamental and artistic by pre-selecting background music that matches the dance moves that the robotic arm is about to perform, or by compiling the dance moves of the robotic arm according to the selected background music.

[0004] However, using the above-mentioned dance performance method of the robotic arm, it is difficult to ensure the precise match between the dance movements and the background music during the dance performance of the robotic arm; and, since the background music or the dance movements of the robotic arm are predetermined, the flexibility of the robotic arm dance performance is also low. Summary of the invention

[0005] The embodiments of the present application provide a dance performance method, device, electronic device and storage medium for a robotic arm, which are used to improve the matching accuracy between the dance movements of the robotic arm and background music, and to improve the flexibility of the dance performance of the robotic arm.

[0006] In a first aspect, an embodiment of the present application provides a dance performance method of a robotic arm, the method comprising:

[0007] Acquire a motion parameter set of the robotic arm at the current moment; the motion parameter set includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment;

[0008] Performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature;

[0009] Based on the correspondence between the preset motion features and the music parameters, determining a plurality of music parameters corresponding to at least one motion feature, and generating background music corresponding to the motion parameter set based on the plurality of music parameters;

[0010] Instruct the robot arm to perform dance movements corresponding to the motion parameter set under background music.

[0011] In an optional embodiment, the motion parameter set includes at least one of the following motion parameters: the motion action of the joint, the motion speed of the joint, the motion amplitude of the joint, and the motion frequency of the joint; wherein the motion action includes: the posture information and motion trajectory of the joint.

[0012] In an optional embodiment, at least one motion feature includes: any one or a combination of the robot arm's movement complexity, motion rhythm, and motion style; wherein the motion rhythm represents: the change pattern of the rotational angular velocity corresponding to each joint.

[0013] In an optional embodiment, a plurality of motion parameters included in the motion parameter set are subjected to feature analysis to obtain at least one motion feature of the robot arm, including:

[0014] Determine the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment based on multiple motion parameters;

[0015] The motion complexity of the robotic arm is determined based on the angular change, angular velocity and angular acceleration of the rotation angles corresponding to each joint at the current moment; wherein each angular change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle at the previous historical moment adjacent to the current moment.

[0016] In an optional embodiment, a plurality of motion parameters included in the motion parameter set are subjected to feature analysis to obtain at least one motion feature of the robot arm, including:

[0017] Determine the rotational angular velocity of each joint at a current moment based on a plurality of motion parameters, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment;

[0018] Based on the multiple rotational angular velocities corresponding to the joints, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0019] Based on the obtained autocorrelation coefficients, the movement rhythm of the robot arm is determined.

[0020] In an optional embodiment, a plurality of motion parameters included in the motion parameter set are subjected to feature analysis to obtain at least one motion feature of the robot arm, including:

[0021] From multiple motion parameters, obtain the motion actions corresponding to each joint;

[0022] Classify the motions corresponding to each joint and determine the motion style of the robotic arm.

[0023] In an optional embodiment, the plurality of music parameters include: instruments, music style, melody, rhythm and chords selected for the background music.

[0024] In a second aspect, an embodiment of the present application further provides a dance performance device of a robotic arm, the device comprising:

[0025] A data acquisition module is used to acquire a set of motion parameters of the robot arm at the current moment; the set of motion parameters includes: at least one motion parameter corresponding to each joint of the robot arm at the current moment;

[0026] A feature analysis module, used to perform feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature;

[0027] A music generation module, used to determine a plurality of music parameters corresponding to at least one motion feature based on a correspondence between preset motion features and music parameters, and generate background music corresponding to a set of motion parameters based on the plurality of music parameters;

[0028] The dance performance module is used to instruct the robot arm to perform dance movements corresponding to the motion parameter set under background music.

[0029] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module is specifically used to:

[0030] Determine the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment based on multiple motion parameters;

[0031] The motion complexity of the robotic arm is determined based on the angular change, angular velocity and angular acceleration of the rotation angles corresponding to each joint at the current moment; wherein each angular change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle at the previous historical moment adjacent to the current moment.

[0032] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module is specifically used to:

[0033] Determine the rotational angular velocity of each joint at a current moment based on a plurality of motion parameters, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment;

[0034] Based on the multiple rotational angular velocities corresponding to the joints, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0035] Based on the obtained autocorrelation coefficients, the movement rhythm of the robot arm is determined.

[0036] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module is specifically used to:

[0037] From multiple motion parameters, obtain the motion actions corresponding to each joint;

[0038] Classify the motions corresponding to each joint and determine the motion style of the robotic arm.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0040] Processor; and

[0041] Memory for storing programs,

[0042] The program includes instructions, which, when executed by a processor, cause the processor to execute the dance performance method of the robotic arm as described in the first aspect.

[0043] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the dance performance method of the robotic arm as described in the first aspect.

[0044] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the dance performance method of the robotic arm as described in the first aspect.

[0045] The beneficial effects of this application are as follows:

[0046] In the dance performance method of the robot arm provided in the embodiment of the present application, after obtaining the motion parameter set of the robot arm at the current moment, at least one motion parameter corresponding to each joint of the robot arm included in the motion parameter set at the current moment can be subjected to feature analysis to obtain at least one motion feature; then, based on the correspondence between the preset motion feature and the soundtrack parameter, multiple soundtrack parameters corresponding to the at least one motion feature can be determined, and background music corresponding to the motion parameter set can be generated based on the multiple soundtrack parameters; finally, the robot arm is instructed to perform the dance movements corresponding to the motion parameter set under the background music. In this way, the problem of low matching accuracy between the dance movements of the robot arm and the background music and the flexibility of the dance performance of the robot arm, which is caused by pre-selecting background music that matches the dance movements to be performed by the robot arm, or compiling the dance movements of the robot arm according to the selected background music, is improved in the related art, and the matching accuracy between the dance movements of the robot arm and the background music and the flexibility of the dance performance of the robot arm are improved.

[0047] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 A schematic diagram of an optional application scenario provided for an embodiment of the present application;

[0050] Figure 2 A schematic diagram of an implementation flow of a dance performance method for a robotic arm provided in an embodiment of the present application;

[0051] Figure 3 A logic diagram for determining the complexity of a robot arm's movements provided in an embodiment of the present application;

[0052] Figure 4 A logic diagram for determining the motion rhythm of a robotic arm provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the system architecture of an optional dance performance system provided in an embodiment of the present application;

[0054] Figure 6 A schematic structural diagram of a dance performance device of a robotic arm provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0057] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0058] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0059] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0060] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0061] First, the design concept of the embodiment of the present application is briefly introduced below:

[0062] With the rapid development of robotics technology, robots are gradually moving from traditional industrial applications to areas closer to people's daily lives, such as services and entertainment. Among them, the dance performances of robotic arms are also attracting more and more attention.

[0063] In the related art, the background music that matches the dance performance of the robot arm usually needs to be pre-selected and edited; or, the dance movements of the robot arm are compiled according to the selected background music.

[0064] However, using the above method, it is difficult to ensure the precise match between the dance movements and the background music during the robot arm dance performance; and the cost of compiling the dance movements of the robot arm according to the selected background music is high. Since the background music or the dance movements of the robot arm are predetermined, the flexibility of the robot arm dance performance is also low.

[0065] In view of this, in order to improve or solve the above problems, the embodiment of the present application proposes a dance performance method for a robot arm, which may specifically include: obtaining a motion parameter set of the robot arm at the current moment; the motion parameter set includes: at least one motion parameter corresponding to each joint of the robot arm at the current moment; then, performing feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature; further, based on the correspondence between the preset motion feature and the soundtrack parameter, determining multiple soundtrack parameters corresponding to at least one motion feature, and generating background music corresponding to the motion parameter set based on the multiple soundtrack parameters; finally, instructing the robot arm to perform dance movements corresponding to the motion parameter set under the background music. In this way, after determining the motion feature of the robot arm according to the motion parameters of each joint of the robot arm at the current moment, multiple soundtrack parameters matching the motion feature of the robot arm can be determined in combination with the correspondence between the motion feature and the soundtrack parameter. In this way, the background music corresponding to the motion parameter set is generated according to the obtained multiple soundtrack parameters, and then the robot arm is instructed to perform dance movements corresponding to the motion parameter set under the background music, which not only improves the matching accuracy between the dance movements of the robot arm and the background music, but also improves the flexibility of the robot arm dance performance.

[0066] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0067] See also Figure 1 As shown, it is a schematic diagram of an optional application scenario provided by an embodiment of the present application, and the application scenario may include: a terminal device 101, a server 102, and a robotic arm 103. The terminal device 101, the server 102, and the robotic arm 103 may all exchange information through a communication network, wherein the communication mode adopted by the aforementioned communication network may include: a wireless communication mode and a wired communication mode.

[0068] Exemplarily, the terminal device 101 can access the network through cellular mobile communication technology and communicate with the server 102 and the robotic arm 103. The cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology. Optionally, the terminal device 101 can access the network through short-range wireless communication mode and communicate with the server 102 and the robotic arm 103. The short-range wireless communication mode, for example, includes wireless fidelity (wireless fidelity, Wi-Fi) technology.

[0069] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above application scenario. For example, the above application scenario may include more terminal devices, or may not include terminal devices, or may include other network devices. Figure 1 As shown, only the terminal device 101, the server 102 and the robotic arm 103 are described as examples, and the above-mentioned communication devices and their respective functions are briefly introduced below.

[0070] The terminal device 101 is a device that can provide voice and / or data connectivity to the user, and can be a device that supports wired and / or wireless connection. Exemplarily, the terminal device 101 can include, but is not limited to: a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal device in industrial control, a wireless terminal device in unmanned driving, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.

[0071] In addition, the terminal device 101 may be installed with a relevant client, which may be software, such as an application (APP), a browser, a short video software, etc., and of course, may also be a web page, a small program, etc. It should be noted that the terminal device 101 in the embodiment of the present application can enable the above-mentioned client related to the dance performance of the robot arm to send a set of motion parameters of the robot arm 103 to the server 102, so as to perform the subsequent method steps such as the dance performance of the robot arm.

[0072] It can be understood that the motion data of the robotic arm 103 can be obtained by collecting multiple sensors deployed on the terminal device 101. Of course, the motion data of the robotic arm 103 can also be obtained by collecting multiple sensors deployed on the robotic arm 103. The embodiment of the present application does not make any specific limitations on this.

[0073] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0074] It is worth mentioning that in an embodiment of the present application, the server 102 can be used to obtain a set of motion parameters of the robotic arm at the current moment; then, perform feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature; further, based on the correspondence between preset motion features and music parameters, determine multiple music parameters corresponding to at least one motion feature, and generate background music corresponding to the motion parameter set based on the multiple music parameters; finally, instruct the robotic arm to perform dance moves corresponding to the motion parameter set under the background music.

[0075] The robot arm 103 is a mechatronic device that simulates the functions of a human arm and can perform a variety of tasks such as grasping, carrying, and operating. It is usually composed of a mechanical structure, a drive system, a control system, and a sensor. The embodiment of the present application does not specifically limit the type of the robot arm 103. Figure 1 The robot arm 103 shown is only an example.

[0076] The following describes the dance performance method of the robotic arm provided by the exemplary embodiment of the present application in combination with the above-mentioned application scenarios and with reference to the accompanying drawings. It should be noted that the above-mentioned application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0077] See also Figure 2 As shown, it is a schematic diagram of the implementation process of a dance performance method of a robotic arm provided in an embodiment of the present application. The execution subject takes a server as an example. The specific implementation process of the method is as follows:

[0078] S201: Obtain a set of motion parameters of the robot arm at the current moment.

[0079] Exemplarily, when executing step S201, the terminal device may send a set of motion parameters of the robot arm collected by multiple sensors at the current moment to the server, wherein the aforementioned set of motion parameters may include: at least one motion parameter corresponding to each joint of the robot arm at the current moment.

[0080] It should be noted that among the above-mentioned multiple sensors, different types of sensors can be used to collect different types of motion parameters, and each motion parameter can correspond to one or more sensors, which is not specifically limited in the embodiments of the present application. Optionally, the sensor can automatically trigger the collection of the motion parameters of the robot arm after the robot arm starts a movement.

[0081] The above-mentioned motion parameter set includes at least one of the following motion parameters: joint motion, joint motion speed, joint motion range, and joint motion frequency. The motion may include: joint posture information and motion trajectory, while the joint motion range is also the joint motion range, and the joint motion frequency is also the joint motion frequency.

[0082] In order to improve the accuracy of subsequent motion feature extraction, the server can perform data preprocessing on the acquired motion parameter set (i.e., the motion data of the robotic arm collected by multiple sensors). Therefore, in an optional implementation, after acquiring the motion parameter set of the robotic arm at the current moment, the server can also perform data preprocessing on the motion parameter set to obtain the motion parameter set after data preprocessing.

[0083] It is understandable that the above data preprocessing operation may include but is not limited to filtering and / or denoising. For example, the server may use a low-pass filter (e.g., Butterworth filter) to filter the collected motion parameter set to remove high-frequency noise in the motion parameter set. For another example, the server may use methods such as wavelet transform to remove random noise in the motion parameter set.

[0084] Take the example of a robot arm including n joints, and the rotation angles, rotation angular velocities and rotation angular accelerations corresponding to the n joints at time t (i.e., the current time) determined according to the motion parameter set. Exemplarily, the data matrix of the rotation angles, rotation angular velocities and rotation angular accelerations corresponding to the aforementioned n joints at time t can be specifically expressed as:

[0085]

[0086] Where D(t) represents the data matrix of the rotation angle, rotation angular velocity and rotation angular acceleration of n joints at time t without data preprocessing, θ i (t) represents the rotation angle of the i-th joint among n joints at time t without data preprocessing, ω i (t) represents the rotation angular velocity of the i-th joint among the n joints at time t without data preprocessing, θ i (t) represents the rotational angular acceleration of the i-th joint among n joints at time t without data preprocessing, i = 1, 2, 3, ..., n.

[0087] Then, the data matrix of the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to the n joints at time t determined according to the motion parameter set after data preprocessing can be specifically expressed as:

[0088]

[0089] Where D′(t) represents the data matrix after data preprocessing corresponding to the data matrix D(t), θ i ′ (t) represents the rotation angle θ i (t) corresponds to the rotation angle after data preprocessing, ω i ′ (t) represents the angular velocity ω i (t) corresponds to the rotation angular velocity after data preprocessing, α i ′ (t) represents the rotational angular acceleration α i (t) Corresponding rotational angular acceleration after data preprocessing.

[0090] S202: Perform feature analysis on multiple motion parameters included in the motion parameter set to obtain at least one motion feature.

[0091] The at least one motion feature mentioned above may include: any one or a combination of: the motion complexity, motion rhythm and motion style of the robot arm. The motion rhythm may represent: the change law of the rotation angular velocity corresponding to each joint included in the robot arm, and the motion style is the style feature of the robot arm motion (e.g., smooth style).

[0092] Therefore, after obtaining the motion parameter set of the robotic arm at the current moment, the server can extract and analyze features of the three characteristic dimensions of motion complexity, motion rhythm and motion style of the robotic arm based on the multiple motion parameters included in the motion parameter set, thereby obtaining the motion complexity, motion rhythm and motion style of the robotic arm.

[0093] It is understandable that the at least one motion feature mentioned above may also include other types of motion features, which is not specifically limited in the embodiments of the present application.

[0094] In an alternative implementation, see Figure 3 As shown, when executing step S202, after obtaining the motion parameter set, the server can determine the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment based on the multiple motion parameters included in the motion parameter set, thereby determining the motion complexity of the robotic arm based on the angular change of the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment.

[0095] Each angle change can be determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle of the previous historical moment adjacent to the current moment. Exemplarily, the calculation formula of the angle change can be specifically expressed as follows:

[0096] Δθ i ′(t)=|θ i ′ (t)-θ i ′ (t-1)|

[0097] Among them, Δθ i ′ (t) represents the angle change between the rotation angle of the i-th joint at time t (i.e., the current time) and the rotation angle at the previous historical time adjacent to time t (i.e., time t-1), θ i ′ (t) represents the rotation angle of the i-th joint at time t, θ i ′ (t-1) represents the rotation angle of the i-th joint at time t-1.

[0098] Since the angular change, angular velocity and angular acceleration of the rotation angles corresponding to each joint at the current moment can be used to quantify the motion complexity of the robot arm, the quantification formula of the aforementioned motion complexity of the robot arm can be specifically expressed as follows:

[0099]

[0100] Among them, C(t) represents the motion complexity of the robot at time t (i.e., the current time), |θ i ′ (t)-θ i ′ (t-1)| represents the absolute value of the angle change between the rotation angle of the i-th joint at time t and the rotation angle at time t-1 adjacent to time t, |ω i ′ (t)| represents the absolute value of the rotational angular velocity of the i-th joint at time t, |α i ′ (t)| represents the absolute value of the rotational angular acceleration of the i-th joint at time t, and n represents the number of joints included in the robotic arm.

[0101] In an optional implementation, after obtaining the motion parameter set, the server may also determine the motion rhythm of the robot arm by analyzing the variation pattern of the rotation angular velocity of each joint, that is, performing autocorrelation analysis on the rotation angular velocity. Figure 4 As shown, the server can determine the rotational angular velocity of each joint at the current moment based on multiple motion parameters included in the motion parameter set, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment, thereby obtaining the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment based on the multiple rotational angular velocities corresponding to each joint, and then determine the movement rhythm of the robotic arm based on the obtained autocorrelation coefficients.

[0102] In order to reduce the complexity of calculation, without affecting the accuracy of the motion rhythm determination, the motion rhythm of the robot arm can be determined according to one or more autocorrelation coefficients corresponding to any one of the above-mentioned joints. Therefore, the quantification formula of the motion rhythm of the above-mentioned robot arm can be specifically expressed as follows:

[0103]

[0104] Among them, R(t) represents the movement rhythm of the robot at time t (i.e., the current moment), N represents the size of the autocorrelation window, that is, the number of historical moments adjacent to the current moment, ω o ′ (t) represents the rotational angular velocity of the i-th joint at time t, ω o ′ (tk) represents the rotational angular velocity of the i-th joint at time tk.

[0105] Based on the above method, in the process of determining the mechanical arm based on the rotational angular velocity, the server extracts periodic information by calculating the autocorrelation coefficient of the rotational angular velocity of the joint. In addition, the server can also obtain the frequency domain information of the rotational angular velocity by Fourier transforming the rotational angular velocity of the joint.

[0106] In an optional implementation, after obtaining the set of motion parameters, the server can also obtain the motion actions corresponding to each joint from multiple motion parameters, thereby classifying the motion actions corresponding to each joint and determining the motion style of the robot. Optionally, the motion style can be classified by a machine learning algorithm, such as using a support vector machine (SVM), random forest or other algorithms to classify the motion of the robot, and identify different motion styles, such as smooth, rigid, rotating, swinging, etc. Exemplarily, the motion style of the robot at time t obtained by the classification algorithm can be expressed as S(t).

[0107] S203: Based on the correspondence between the preset motion features and the music parameters, determine a plurality of music parameters corresponding to at least one motion feature, and generate background music corresponding to the motion parameter set based on the plurality of music parameters.

[0108] For example, the above-mentioned multiple soundtrack parameters may include but are not limited to: the instrument, style, melody, rhythm and chords selected for the background music. At this time, the correspondence between the above-mentioned preset motion features and the soundtrack parameters is also the correspondence between the motion features and the instrument, style, melody, rhythm and chords selected for the background music.

[0109] Taking the motion characteristics of the robot arm, including motion complexity, motion rhythm, and motion style, as an example, the server can select the appropriate instrument type according to the motion style S(t) of the robot arm. For example, string instruments and pianos are selected for smooth styles; drums and bass are selected for hard styles; electronic synthesizers are selected for rotation styles; and guitars and harmonicas are selected for swing styles.

[0110] The server can also select the style of background music (i.e., soundtrack) according to the robot's motion style S(t) and motion complexity C(t). For example, if the motion complexity is low and the flow style is smooth, a lyrical style is selected; if the motion complexity is high and the style is rigid, a rock style is selected; if the motion complexity is medium and the style is rotating, an electronic dance style is selected.

[0111] The server can also generate the melody of the background music according to the movement rhythm R(t) and the action complexity C(t), and in combination with a preset music melody generation method (e.g., a preset melody generation rule). For example, if the movement rhythm is fast, the melody pitch changes greatly and the rhythm is dense; if the movement rhythm is slow, the melody pitch changes little and the rhythm is sparse; if the action complexity is high, the melody complexity is high and contains more dissonant intervals; if the action complexity is low, the melody is simple and contains more harmonious intervals. The embodiments of the present application do not specifically limit this.

[0112] For another example, the server can also determine the rhythm of the background music according to the movement rhythm R(t) of the robot arm. For example, if the movement rhythm is faster, a higher rhythm speed can be set, and the unit of the rhythm speed can be: beats per minute (BPM); if the movement rhythm is slower, a lower rhythm speed can be set. In addition, the server can also select a suitable chord progression and generate chords based on the generated melody result and the movement style S(t). For example, if the movement style S(t) is a smooth style, a major chord progression can be used; if the movement style S(t) is a rigid style, a minor chord progression can be used; if the movement style S(t) is a rotational style, a dissonant chord progression can be used.

[0113] Furthermore, after determining the multiple soundtrack parameters corresponding to the at least one motion feature, the server can generate background music corresponding to the motion parameter set based on the multiple soundtrack parameters. For example, the server can synthesize the final soundtrack audio, i.e., the background music, based on the determined multiple soundtrack parameters and pre-set music synthesis software or algorithm (e.g., a musical instrument digital interface (MIDI) synthesizer, an audio plug-in, etc.).

[0114] S204: Instructing the robot arm to perform dance movements corresponding to the motion parameter set under background music.

[0115] Exemplarily, when executing step S204, the server can achieve synchronous output of the generated background music and the dance movements of the robot arm through timestamp synchronization or real-time synchronization. Timestamp synchronization means that when generating background music, each music event is timestamped and matched with the timestamp of the dance movement of the robot arm. Real-time synchronization means that the real-time synchronization of the background music and the dance movement of the robot arm is achieved by using a real-time operating system.

[0116] In an alternative implementation, see Figure 5 As shown, it is a schematic diagram of the system architecture of an optional dance performance system provided by an embodiment of the present application. The dance performance system may include: a mechanical arm motion data acquisition module, a data preprocessing module, a motion feature extraction and analysis module, a soundtrack parameter generation module, a music synthesis module and a synchronous output module. Among them, the mechanical arm motion data acquisition module is responsible for real-time acquisition of motion data or motion parameters of each joint of the mechanical arm, including joint angle, angular velocity, angular acceleration, etc. The data preprocessing module can perform data preprocessing operations such as filtering and denoising on the collected mechanical arm motion data to improve the accuracy and reliability of the data. The motion feature extraction and analysis module can extract key information that can characterize its motion characteristics from the mechanical arm motion data after data preprocessing, such as movement complexity, movement rhythm, movement style, etc. The soundtrack parameter generation module can generate corresponding soundtrack parameters based on the extracted mechanical arm motion features, such as instrument selection, style determination, melody generation, rhythm setting, chord arrangement, etc. The music synthesis module can synthesize the final soundtrack audio based on the generated soundtrack parameters. The synchronous output module can synchronize the generated soundtrack audio with the dance movements of the mechanical arm.

[0117] It is understandable that the embodiment of the present application does not specifically limit the module types and the number of modules included in the dance performance system. Figure 5 As shown, the dance performance system may also include a user interaction module and a system control module. The user interaction module may provide an interface for the user to interact with the system, and the user may set, control and monitor the system through the module. The system control module may control the operation process of the entire dance performance system and coordinate the work between various modules.

[0118] It can be seen that the dance performance method of the robotic arm recorded in the above steps S201 to S204 achieves accurate matching of music and robotic arm dance movements, thereby improving the viewing and artistry of the robotic arm dance performance; by extracting and analyzing the features of the robotic arm motion parameters, it is possible to generate music with diverse styles and rich changes according to different movement characteristics; and by combining artificial intelligence technology with music generation technology, the intelligent and automated robotic arm dance performance is achieved.

[0119] In summary, in the dance performance method of the robot arm provided in the embodiment of the present application, after obtaining the motion parameter set of the robot arm at the current moment, at least one motion parameter corresponding to each joint of the robot arm included in the motion parameter set at the current moment can be subjected to feature analysis to obtain at least one motion feature; then, based on the correspondence between the preset motion feature and the soundtrack parameter, multiple soundtrack parameters corresponding to the at least one motion feature can be determined, and background music corresponding to the motion parameter set can be generated based on the multiple soundtrack parameters; finally, the robot arm is instructed to perform the dance movements corresponding to the motion parameter set under the background music. In this way, the problem of low matching accuracy between the dance movements of the robot arm and the background music and the flexibility of the dance performance of the robot arm, which is caused by pre-selecting background music that matches the dance movements to be performed by the robot arm, or compiling the dance movements of the robot arm according to the selected background music, is improved in the related art, and the matching accuracy between the dance movements of the robot arm and the background music and the flexibility of the dance performance of the robot arm are improved.

[0120] Further, based on the same technical concept, the present application embodiment provides a dance performance device of a robot arm, and the dance performance device of the robot arm is used to implement the above method flow of the present application embodiment. Figure 6 As shown, the dance performance device 600 of the robot arm may include: a data acquisition module 601, a feature analysis module 602, a music generation module 603 and a dance performance module 604, wherein:

[0121] The data acquisition module 601 is used to acquire a set of motion parameters of the robot arm at the current moment; the set of motion parameters includes: at least one motion parameter corresponding to each joint of the robot arm at the current moment;

[0122] A feature analysis module 602, configured to perform feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature;

[0123] The music generation module 603 is used to determine a plurality of music parameters corresponding to at least one motion feature based on a preset correspondence relationship between the motion feature and the music parameters, and generate background music corresponding to the motion parameter set based on the plurality of music parameters;

[0124] The dance performance module 604 is used to instruct the robot arm to perform dance movements corresponding to the motion parameter set under background music.

[0125] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module 602 is specifically used to:

[0126] Determine the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment based on multiple motion parameters;

[0127] The motion complexity of the robotic arm is determined based on the angular change, angular velocity and angular acceleration of the rotation angles corresponding to each joint at the current moment; wherein each angular change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle at the previous historical moment adjacent to the current moment.

[0128] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module 602 is specifically used to:

[0129] Determine the rotational angular velocity of each joint at a current moment based on a plurality of motion parameters, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment;

[0130] Based on the multiple rotational angular velocities corresponding to the joints, the autocorrelation coefficient of the rotational angular velocity of each joint at the current moment is obtained;

[0131] Based on the obtained autocorrelation coefficients, the movement rhythm of the robot arm is determined.

[0132] In an optional embodiment, when performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robot arm, the feature analysis module 602 is specifically used to:

[0133] From multiple motion parameters, obtain the motion actions corresponding to each joint;

[0134] Classify the motions corresponding to each joint and determine the motion style of the robotic arm.

[0135] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0136] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0137] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0138] See also Figure 7 As shown, the structured block diagram of the electronic device 700 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0139] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0140] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 may be any type of device capable of inputting information to the electronic device 700, and the input unit 706 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 707 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 may include but is not limited to a disk, an optical disk. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0141] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned dance performance method of the robotic arm may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 may be configured to perform the above-mentioned dance performance method of the robotic arm by any other appropriate means (e.g., by means of firmware).

[0142] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0143] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0147] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0148] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A dance performance method for a robotic arm, characterized in that: include: Get the motion parameter set of the robot arm at the current moment; The motion parameter set includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment; Performing feature analysis on a plurality of motion parameters included in the motion parameter set to obtain at least one motion feature; Based on the correspondence between the preset motion features and the music parameters, determining a plurality of music parameters corresponding to the at least one motion feature, and generating background music corresponding to the motion parameter set based on the plurality of music parameters; The robotic arm is instructed to perform a dance movement corresponding to the motion parameter set under the background music.

2. The method according to claim 1, characterized in that The motion parameter set includes at least one of the following motion parameters: the motion action of the joint, the motion speed of the joint, the motion amplitude of the joint, and the motion frequency of the joint; wherein the motion action includes: the posture information and motion trajectory of the joint.

3. The method according to claim 1 or 2, characterized in that The at least one motion feature includes: any one or a combination of the movement complexity, movement rhythm and movement style of the robotic arm; wherein the movement rhythm represents: the change pattern of the rotational angular velocity corresponding to each of the joints.

4. The method according to claim 3, characterized in that The performing feature analysis on the plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm comprises: Determine the rotation angle, rotation angular velocity and rotation angular acceleration corresponding to each joint at the current moment based on the multiple motion parameters; The motion complexity of the robotic arm is determined based on the angular change of the rotation angle, the angular velocity and the angular acceleration of the rotation angle corresponding to each joint at the current moment; wherein each angular change is determined based on the rotation angle of the corresponding joint at the current moment and the rotation angle at the previous historical moment adjacent to the current moment.

5. The method according to claim 3, characterized in that The performing feature analysis on the plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm comprises: Determine the rotational angular velocity of each joint at the current moment based on the multiple motion parameters, and obtain the rotational angular velocity of at least one historical moment adjacent to the current moment; Based on the multiple rotational angular velocities corresponding to the respective joints, obtaining the autocorrelation coefficient of the rotational angular velocity of the respective joints at the current moment; Based on the obtained autocorrelation coefficients, the movement rhythm of the robot arm is determined.

6. The method according to claim 3, characterized in that The performing feature analysis on the plurality of motion parameters included in the motion parameter set to obtain at least one motion feature of the robotic arm comprises: From the plurality of motion parameters, obtaining motion actions corresponding to the respective joints; The motion actions corresponding to the respective joints are classified to determine the motion style of the robotic arm.

7. The method according to claim 1 or 2, characterized in that: The multiple music parameters include: the instrument, style, melody, rhythm and chords selected for the background music.

8. A dance performance device for a robotic arm, characterized in that: include: A data acquisition module is used to obtain a set of motion parameters of the robot arm at the current moment; The motion parameter set includes: at least one motion parameter corresponding to each joint of the robotic arm at the current moment; A feature analysis module, configured to perform feature analysis on a plurality of motion parameters included in the motion parameter set, and obtain at least one motion feature of the dance performance performed by the robotic arm at the current moment; A music generation module, configured to determine a plurality of music parameters corresponding to the at least one motion feature based on a correspondence between preset motion features and music parameters, and generate background music corresponding to the motion parameter set based on the plurality of music parameters; The dance performance module is used to instruct the robot arm to perform dance movements corresponding to the motion parameter set under the background music.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images