Dance editing methods, devices, and dance machines

Abstract

This application relates to a dance editing method, apparatus, dance machine, computer device, and computer-readable storage medium. The method includes: acquiring choreography music and identifying each beat point of the choreography music; playing the choreography music to a user to assist in choreography; during user choreography, collecting key direction signals generated by foot pedals at each beat point position, and acquiring human motion data at the corresponding beat point position through a motion capture camera; jointly determining the valid stepping keys corresponding to each beat point based on the key direction signals and the human motion data; and generating choreography footwork based on the choreography music and the valid stepping keys corresponding to each beat point. This technical solution enables accurate choreography on a dance machine, reduces the difficulty and complexity of choreography, lowers the barrier to entry for ordinary players, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of entertainment equipment technology, and in particular to a dance editing method, apparatus, dance machine, computer equipment, and computer-readable storage medium. Background Technology

[0002] With the popularization of dance machines, people's entertainment options have been greatly enriched, and more and more players are starting to get into game dance. Even those without a dance background can move to the rhythm of the music.

[0003] However, most dance machine products on the market only provide choreography for songs in the song library. Even if local songs can be added, the generated choreography is difficult to reach an artistic level. The accuracy between the music and the dance is relatively low. More and more dance game players are keen on better dance expression, and the dance on dance machines can no longer meet the requirements of diversification and artistry.

[0004] The existing dance machine choreography methods require professional choreographers, who need a high level of personal artistry and professional skills, as well as a keen sense of music to create corresponding choreography. The choreography is difficult and the operation is complex, which makes the choreography threshold high for ordinary players and seriously restricts the application functions of dance machines. Summary of the Invention

[0005] To address one of the aforementioned technical deficiencies, this application provides a dance editing method, apparatus, dance machine, computer equipment, and computer-readable storage medium, thereby lowering the choreography threshold for dance machines and enhancing their application functionality.

[0006] First aspect:

[0007] This application provides a dance editing method, including:

[0008] Acquire choreography music and identify each beat point of the choreography music, and play the choreography music to the user to assist in choreography;

[0009] When users are choreographing, the key direction signals generated by the foot pedals are collected at each beat point, and the human motion data at the corresponding beat point is obtained through the motion capture camera.

[0010] The effective pedaling keys are determined by jointly judging the pedaling keys corresponding to each beat point based on the key direction signal and the human motion data.

[0011] The choreography footwork is generated based on the choreography music and the valid step keys corresponding to each beat point.

[0012] In one embodiment, the effective pedaling keys are determined by jointly judging the pedaling keys corresponding to each beat point based on the key direction signal and the human motion data, including:

[0013] When the collected key direction signal reaches the effective pedaling pressure threshold, the key corresponding to the key direction signal is output as the effective pedaling key.

[0014] If the collected key direction signal does not reach the effective stepping pressure threshold, the relative distance and relative angle between each limb are calculated based on the human motion data.

[0015] If both the relative distance and the relative angle reach the determination threshold, then the key corresponding to the key direction signal is a valid step key.

[0016] In one embodiment, after generating the choreography footwork based on the valid footwork keys corresponding to each beat point, the method further includes:

[0017] The choreography footwork is divided into several segmented footwork patterns;

[0018] Calculate the mean local deviation between each segment of the score and the beat line; where the beat line is an auxiliary line for the beat point.

[0019] The segmented footnotes are initially corrected based on the mean local deviation of each segmented footnote;

[0020] Calculate the global mean deviation between the choreography footwork and the beat line;

[0021] The choreography footwork is then revised a second time based on the global mean deviation.

[0022] In one embodiment, generating choreography footwork based on the choreography music and the valid footwork keys corresponding to each beat point includes:

[0023] Obtain the beat lines corresponding to each beat point of the choreography music; wherein, the beat lines are beat point auxiliary lines;

[0024] Align the valid step keys corresponding to each beat point with the beat line, and generate choreography footwork by combining it with the choreography music.

[0025] In one embodiment, the dance editing method further includes:

[0026] Based on the human motion data, obtain the hand position information corresponding to each beat point;

[0027] Generate hand gestures corresponding to each beat point based on the hand position information;

[0028] A choreographed dance is generated based on the gestures and choreography footwork.

[0029] In one embodiment, the dance editing method further includes:

[0030] A virtual dance video corresponding to the choreographed dance is generated using 3D rendering technology and based on the human motion data.

[0031] The virtual dance video is stored on a server for synchronized demonstration of animated characters when the choreography is played.

[0032] In one embodiment, the dance editing method further includes, during user choreography:

[0033] The video image flashes at each beat point;

[0034] Prompt the user to prepare to perform a step button or gesture action;

[0035] After the user steps on a key, the key is displayed to the user via video or a 3D gesture is demonstrated simultaneously.

[0036] In one embodiment, the choreographed dance includes: a single-key pedal dance, a single-key pedal dance and hand gesture dance, a multi-key pedal dance, or a multi-key pedal dance and hand gesture dance.

[0037] In one embodiment, acquiring choreographic music and identifying the beat points of the choreographic music includes:

[0038] Acquire at least one piece of music from the signal source;

[0039] Choreography music is created by splicing together at least one segment from each piece of music.

[0040] STFT spectrum analysis is performed on the choreography music to obtain the beat points corresponding to the time of the choreography music, and a beat line aligned with the beat points is generated.

[0041] In one embodiment, acquiring the key direction signal generated by the foot pedal at each beat point position includes:

[0042] Each beat point is used as a timestamp, and a signal capture window is determined according to a set time range. Within the capture window, the changing signal output when the foot pedal is stepped on is continuously detected.

[0043] In one embodiment, the dance editing method further includes:

[0044] Obtain the rhythm modifications made by the user to the choreography music;

[0045] The choreography footwork is then edited a second time based on the modified choreography music, and the footwork is then refined based on the second editing.

[0046] In one embodiment, the dance editing method further includes:

[0047] Obtain the choreography footwork for the dance;

[0048] Calculate the similarity between the choreography footwork and the pre-stored footwork in the dance music library, and evaluate the choreography based on the similarity to obtain a score.

[0049] The choreographed dance will be recommended to other users based on the rating score.

[0050] In one embodiment, the dance editing method further includes:

[0051] Obtain evaluation data for the recommended choreography;

[0052] The weights of each evaluation indicator were determined using the analytic hierarchy process (AHP).

[0053] The popularity score of the choreographed dance is calculated based on the evaluation data and weights.

[0054] The recommended choreography dances are sorted according to the popularity value.

[0055] In one embodiment, the dance editing method further includes:

[0056] Matching the pre-stored footwork scores from the dance music library according to the beat line and song style of the choreography music, the corresponding footwork score fragments are obtained; based on the character movement data corresponding to the footwork score fragments, the virtual limb dance of the simulated 3D animated character is performed.

[0057] In one embodiment, after generating the choreography footwork based on the valid footwork keys corresponding to each beat point, the method further includes:

[0058] Request a list of beat icon and stomp effect from the server's skin library;

[0059] The footwork editing toolbar displays a list of beat point icons and a list of stomping effect special effects;

[0060] Get the target beat point icon style and target stomp effect style selected by the user from the beat point icon list and stomp effect list;

[0061] Replace the beat point icon style in the editing area with the target beat point icon style, and control the beat line to demonstrate the target stomping effect style.

[0062] The second aspect:

[0063] This application provides a dance editing device, comprising:

[0064] The music processing module is used to acquire choreography music and identify the beat points of the choreography music, and play the choreography music to the user to assist in choreography.

[0065] The information acquisition module is used to collect the key direction signals generated by the foot pedal at each beat point when the user is choreographing, and to acquire human motion data at the corresponding beat point position through the motion capture camera.

[0066] The joint determination module is used to jointly determine the valid pedaling keys corresponding to each beat point based on the key direction signal and the human motion data.

[0067] The footwork generation module is used to generate choreography footwork based on the choreography music and the valid step keys corresponding to each beat point.

[0068] Third aspect:

[0069] This application provides a dance machine, including: a main unit, and a display device, a motion capture camera, and a foot pedal connected to the main unit; wherein, the main unit is also connected to a server;

[0070] The host is configured to perform the steps of the dance editing method described above;

[0071] The motion capture camera is used to capture image data of the choreography and output human motion data;

[0072] The foot pedal is used to output a key direction signal that the key area is being pressed;

[0073] The display device is used to display screen information.

[0074] Fourth aspect:

[0075] This application provides a computer device, which includes:

[0076] One or more processors;

[0077] Memory;

[0078] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to: perform the dance editing method described above.

[0079] Fifth aspect:

[0080] This application provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded by the processor and executed to perform the dance editing method described above.

[0081] The aforementioned dance editing method and apparatus, dance machine, computer equipment, and computer-readable storage medium acquire choreography music and identify each beat point of the choreography music. During user choreography, key direction signals generated by foot pedals are collected at each beat point, and human motion data is acquired through a motion capture camera. The key direction signals and human motion data are used to jointly determine the effective stepping keys corresponding to each beat point. Then, choreography footwork is generated based on the choreography music and the effective stepping keys corresponding to each beat point. This technical solution enables accurate choreography on a dance machine, reduces the difficulty and complexity of choreography, lowers the barrier to entry for ordinary players, and enhances the user experience.

[0082] Furthermore, by expanding the acquisition and capture window for key direction signals, an ultra-wide capture window can be used for the two seconds before and after the beat of the choreography music, thereby ensuring that the user's key-stepping area can be detected and guaranteeing the accuracy of the choreography.

[0083] Furthermore, by using 3D rendering to create virtual dance videos, when users request the choreography, the animated characters can dance in sync, providing users with video references for dance games and enhancing their dance game experience.

[0084] Furthermore, by calculating the degree of deviation between the footwork and the beat line, and then correcting each segment by moving it up or down, a combination of segmented correction and global correction is used to automatically make subtle corrections to the user's footwork, thereby obtaining a choreography footwork that is completely consistent with the beat line, ensuring the accuracy of the beat point and the position of the footwork.

[0085] Furthermore, after the initial generation of the choreography footnotes, users can pause, replay, slow down, and fast-forward the choreography footnotes multiple times. Based on the secondary editing operations, the choreography footnotes can be refined and corrected to obtain suitable personalized choreography footnotes.

[0086] Furthermore, scoring and sorting functions have been designed to enhance players' creative enthusiasm, especially for advanced players who can create highly challenging racing footwork patterns and highly artistic freestyle footwork patterns based on their own abilities.

[0087] Additional aspects and advantages of this application will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description

[0088] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0089] Figure 1 This is a structural framework diagram of an example dance editing technology solution;

[0090] Figure 2 This is a flowchart of a dance editing method according to one embodiment;

[0091] Figure 3 This is a diagram illustrating the music processing workflow;

[0092] Figure 4 This is a diagram illustrating the user login process;

[0093] Figure 5 It is a custom skin flowchart;

[0094] Figure 6 This is a screenshot of the custom skin interface;

[0095] Figure 7 This is a flowchart illustrating the secondary editing process;

[0096] Figure 8 This is a flowchart illustrating the classification process of choreography modes;

[0097] Figure 9 This is a schematic diagram of the process of sorting choreography footwork.

[0098] Figure 10 This is a schematic diagram of the structure of a dance editing device according to one embodiment. Detailed Implementation

[0099] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0100] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the word “comprising” as used in this application’s specification means the presence of the stated feature, integer, step, or operation, but does not preclude the presence or addition of one or more other features, integers, steps, or operations.

[0101] The dance editing solution in this application can be applied to dance machines, such as... Figure 1 As shown, Figure 1This is a system architecture diagram of an example dance machine. The main unit connects to a display device, a motion capture camera, and foot pedals, and connects to a server via a network. Generally, the foot pedals are foot motion capture devices, also known as dance mats, dance pedals, or pedals. The device has five key areas (upper left, lower left, upper right, lower right, and center), each marked with a directional arrow that illuminates during dancing to indicate the key area the user is stepping on. The motion capture camera is typically an AI camera, capable of capturing image data of the user dancing and analyzing it to output human motion data. Alternatively, if the motion capture camera outputs image data, the main unit can process and output human motion data. The main unit can import music data and play it through a sound system. Users can choreograph dances based on the music. During choreography, the main unit records the footwork by detecting the key direction signals of the foot pedals, ultimately generating the choreography score, which is stored locally or on a server. This score can also be shared with other users or used for later playback. The display device can display the information output by the main unit, such as the music's beat lines and video animations.

[0102] The following example uses a dance machine to illustrate dance editing methods. Figure 2 As shown, Figure 2 This is a flowchart of a dance editing method according to an embodiment, which mainly includes the following steps:

[0103] Step S10: Obtain the choreography music and identify each beat point of the choreography music, and play the choreography music to the user to assist in choreography.

[0104] In this step, at least one piece of music can be obtained from the signal source, and then at least one segment of each piece of music can be spliced ​​together to obtain the choreography music; then, STFT (Short-Time Fourier Transform) spectrum analysis is performed on the choreography music to obtain the beat points of the corresponding time of the choreography music, and a beat line aligned with the beat points is generated, which serves as an auxiliary line for the beat points.

[0105] For example, refer to Figure 3 , Figure 3 This is a diagram of the music processing flow. Music files can be sent to a server for processing or processed locally on the dance machine. Users can transfer their favorite music according to their personal preferences via Bluetooth, local transfer (CD, Blu-ray, DVD, USB flash drive), cloud storage, Wi-Fi (MIRACAST / AIRPLAY / DLNA), coaxial, optical, FM and other input sources to the dance machine.

[0106] After importing music, users can edit the music files. By playing the music files multiple times, they can select sections with rich musical elements for choreography, or select multiple sections to splice together to create choreography music, thereby increasing the entertainment and challenge of the game. After editing the music files, STFT spectrum analysis can be performed on the server or locally to obtain the beat points of the song at the corresponding time. The beat lines aligned with the choreography music can be generated on the score editor for users to use as a reference for choreography.

[0107] When playing choreography music to users, the music can be set to play in slow motion (one-half, one-third, one-quarter, or one-eighth of the speed), which gives users enough time to accurately and complete the stepping movements.

[0108] In addition, to ensure that users can access their own choreography anytime, anywhere, this embodiment can provide personalized registration services for users, see reference. Figure 4 , Figure 4 This is a user login process diagram; before use, users can log in to their personal accounts through methods such as facial recognition, fingerprint recognition, mobile phone number and dynamic code, mini-program QR code, account password, etc., and then they can choreograph and dance games.

[0109] Step S20: When the user is choreographing, collect the key direction signals generated by the foot pedal at each beat point, and acquire human motion data at the corresponding beat point position through the motion capture camera.

[0110] In this step, the beat points in the choreography music are used as timestamps. Based on these timestamps, the key direction signals of the foot pedals are collected and combined with the human motion data output by the motion capture camera to make a comprehensive judgment on the stepping area, thereby ensuring the accuracy of the choreography footwork. The human motion data mainly includes data on key parts such as arms, shoulders, head, body, leg joints, and feet.

[0111] Specifically, with the aid of choreography music, users can choreograph according to the beat lines, such as... Figure 1 In this process, the user steps on the key areas of the foot pedal according to the beat, and the foot pedal outputs key direction signals to record the key areas stepped on by the user; at the same time, the motion capture camera can capture image data of the user's choreography in real time, and output human motion data based on the image data; through human motion data, the trajectory of human limb movement can be obtained, thereby obtaining the user's limb movements in real time.

[0112] In one embodiment, considering that it is difficult for a user to perfectly match the beat when pressing the pedal, in order to reduce the difficulty, when collecting the key direction signal generated by the pedal at each beat point, each beat point can be used as a timestamp, and a signal capture window can be determined according to a set time range. Within the capture window, the change signal output when the pedal is pressed can be continuously detected.

[0113] The technical solution of the above embodiment expands the acquisition and capture window of the key direction signal. It can use the two-second time before and after the beat of the choreography music as an ultra-wide capture window, thereby ensuring that the user's stepping on the key area can be detected and guaranteeing the accuracy of the choreography.

[0114] Step S30: Based on the key direction signal and the human motion data, the pedal key corresponding to each beat point is jointly determined to obtain the valid pedal key.

[0115] In one embodiment, the technical solution for jointly determining the valid pedal position may include the following:

[0116] (1) When the collected key direction signal reaches the effective stepping pressure threshold and the relative distance and relative angle of each limb calculated according to the human skeletal motion data reach the threshold and the direction is consistent, the key corresponding to the key direction signal is output as the effective stepping key.

[0117] (2) When the collected key direction signal does not reach the effective stepping pressure threshold, but reaches the minimum set threshold, the relative distance and relative angle between each limb are calculated based on the human motion data; if the relative distance and relative angle both reach the judgment threshold, the key corresponding to the key direction signal is output as an effective stepping key. At this time, the user can choose to pause, slow down or fast up to generate the key. Otherwise, the key direction signal is a misjudgment signal and is discarded.

[0118] Because different users apply different pressure and direction when pressing the pedals, incorrect key direction signals or failure to output correct key direction signals can easily occur after pressing. The system acquires the user's three-dimensional human motion data through a motion capture camera, calculates the human motion trajectory, calibrates the three-dimensional coordinate system through normalization and regularization, calculates the relative distance using Euclidean distance, calculates the relative angle using trigonometric cosine, and finally combines the key direction signals output by the pedals to jointly determine the user's valid key press.

[0119] The above technical solution ensures the accuracy of the choreography by collecting the key direction signals of the foot pedals and combining them with human motion data from the motion capture camera to comprehensively determine the stepping area.

[0120] Step S40: Generate choreography footwork based on the choreography music and the valid step keys corresponding to each beat point.

[0121] In this step, the beat lines corresponding to each beat point of the choreography music can be obtained; then, the valid step keys corresponding to each beat point are aligned with the beat lines, and the choreography footwork is generated in combination with the choreography music.

[0122] The technical solution of the above embodiment acquires choreography music and identifies each beat point of the choreography music. When the user is choreographing, the key direction signals generated by the foot pedals are collected at each beat point position, and human motion data is acquired through a motion capture camera. The key direction signals and human motion data are used to jointly determine the effective stepping keys corresponding to each beat point. Then, a choreography footnote is generated based on the choreography music and the effective stepping keys corresponding to each beat point. This technical solution enables accurate choreography on dance machines, reduces the difficulty and complexity of choreography, lowers the threshold for ordinary players to learn choreography, and improves the user experience. In specific implementation, the dance machine can recommend suitable choreography music based on the user's age and gender. The choreography footnote edited by the user is automatically saved to the user's account. The next time the user logs in to any device, they can recall the edited choreography footnote to play the dance game.

[0123] To make the technical solution of this application clearer, it will be described below with reference to more embodiments.

[0124] In one embodiment, in order to increase the difficulty of the game and enhance the players' creative enthusiasm, the technical solution of this application can also add choreography for hand gestures.

[0125] Accordingly, the dance editing method of this application can also obtain hand position information corresponding to each beat point based on the human motion data; generate hand gestures corresponding to each beat point based on the hand position information; and generate choreographed dances based on the hand gestures and choreography footwork.

[0126] Furthermore, 3D rendering technology is used to generate a virtual dance video corresponding to the choreography based on the human motion data; the virtual dance video is stored in the server for synchronous demonstration of animated characters when the choreography is played.

[0127] The technical solution of the above embodiment generates choreographed dances by using choreography footwork and gestures, and uses 3D rendering to create virtual dance videos. The choreographed dances can be saved to a personal account. The next time a user requests the choreographed dance, the animated characters can dance in sync, providing users with video references for dance games and enhancing their dance game experience.

[0128] In one embodiment, to improve the accuracy of the user stepping on the pedals, when the user is choreographing in step S20, the user can select a song to play fast or slow beforehand to choreograph, and can choose to pause during the process. Then, the video screen flashes at each beat point to prompt the user to prepare to step on the key or make a gesture. After the user steps on the key, the key is displayed to the user through the video screen or a 3D gesture is demonstrated simultaneously.

[0129] For example, based on the beat line prompt, the video screen flashes for 0.5 seconds to prompt the user to prepare to step on the key or make a gesture. The key that the user stepped on is displayed on the display device. This technical solution makes it easy for the user to know in a timely manner whether the current key area has been correctly stepped on.

[0130] In one embodiment, to enhance personalization, custom beat point designs can be added to the editor. Players can personalize the placement and visual effects of these beat points. Based on this, after generating the choreography footwork according to the valid footwork keys corresponding to each beat point, the editor also includes:

[0131] Request a list of beatpoint icons and a list of stomping effects from the server's skin library; display the list of beatpoint icons and stomping effects in the footwork editing toolbar; obtain the target beatpoint icon style and target stomping effect style selected by the user from the list of beatpoint icons and stomping effects; replace the beatpoint icon style in the editing area with the target beatpoint icon style, and control the beat line to demonstrate with the target stomping effect style.

[0132] For example, refer to Figure 5 As shown, Figure 5 This is a custom skin flowchart; the server has a skin library, corresponding to a personalized beat point icon library and a stomping effect library in the editor. When editing the footwork, players can select the target beat point icon style (i.e., the "arrow" style in the image) and the target stomping effect style (i.e., the "effect" style in the image) from the list and apply them to each beat point; (Reference) Figure 6 As shown, Figure 6 This is a schematic diagram of the custom skin interface. The list of beat point icons and stomping effect effects downloaded from the skin library is displayed in the toolbar. Users can select the target beat point icon style (i.e., the "arrow" style in the image) and the target stomping effect style (i.e., the "effect" style in the image) from the list on the right, and then apply them to the footwork at the beat line position in the editing area on the left.

[0133] In one embodiment, due to the large differences in users' dance skills and the different sensitivities of different players to music, the timing of each user's footsteps and movements during choreography is not entirely consistent with the beat point position. Accordingly, this application also provides an automated beat correction technology.

[0134] Accordingly, after generating the choreography footwork in step S40, the following may also be included:

[0135] (a) Divide the choreography footwork into several segmented footworks; specifically, the division can be based on the rhythm information of the choreography music.

[0136] (b) Calculate the mean local deviation between each segment of the musical score and the beat line; and make an initial correction to the segmented musical score based on the mean local deviation of each segment.

[0137] (c) Calculate the global mean deviation between the choreography footwork and the beat line; and make a second correction to the choreography footwork after the first correction based on the global mean deviation.

[0138] The technical solution of the above embodiment calculates the degree of deviation between the footwork and the beat line, and then corrects each segment by moving it up or down. By combining segmented correction and global correction, it automatically makes minor corrections to the user's footwork, thereby obtaining a choreography footwork that is completely consistent with the beat line, ensuring the accuracy of the beat point and the position of the footwork.

[0139] In one embodiment, after the initial generation of the choreography footnotes, users can edit the choreography footnotes multiple times to obtain suitable personalized choreography footnotes.

[0140] Accordingly, the dance editing method of this application can also obtain the rhythm modification made by the user to the choreography music; perform a secondary editing operation on the choreography footwork according to the modified choreography music, and use replay, pause, fast forward, slow motion functions during the secondary editing, and perform fine-grained footwork correction on the choreography footwork according to the secondary editing operation.

[0141] For details, please refer to Figure 7 As shown, Figure 7This is a flowchart illustrating the secondary editing process. Users can repeatedly modify their personalized choreography scores using a mouse, keyboard, remote control, or touchscreen. For example, users can replay the score to make changes, or choose to pause, slow (half, quarter, or eighth of a second slow playback without changing the rhythm), fast (1x, 2x, 8x, 16x, or 32x speed without changing the rhythm), or change the speed (directly changing the speed without changing the pitch, modifying the song's rhythm). During secondary editing, the choreography music can be magnified on the display device, facilitating detailed corrections to the choreography score. The initial editing also includes pause, slow, fast playback, and speed adjustment options, providing users with multiple editing methods.

[0142] In addition, users can modify the generated choreography footwork according to the difficulty level (such as 1 star, 2 stars, 3 stars, 4 stars, 5 stars, etc.), and appropriately set the key positions for stepping. They can also save each type of choreography footwork to the server and store it in their personal account. They can also save it according to different difficulty IDs. The same choreography footwork can support saving multiple choreography footwork footwork of different difficulties.

[0143] The technical solution of the above embodiments provides replay and editing functions. Through the replay process, users can choreograph footwork at any time and beat point, and can also redo the movements, modify the step keys or redo the gestures to form a new choreography.

[0144] In one embodiment, to meet different types of user needs, the technical solution of this application can provide a technical solution that allows users to choose between automatically generating footnotes or customizing footnote designs. Accordingly, as an embodiment, refer to Figure 8 As shown, Figure 8 This is a flowchart illustrating the choreography mode classification process. There are three choreography modes for users to choose from: automatic choreography mode, speed choreography mode, and personalized choreography mode.

[0145] (I) Automatic Choreography Mode:

[0146] In this mode, the dance machine automatically generates choreography footwork based on song style, rhythm, and other information at the corresponding BPM (beats per minute). This mode is suitable for users who are not good at editing footwork footwork, providing quick choreography footwork generation and secondary editing functions for the choreography footwork footwork.

[0147] As an example, the dance editing method in this mode can match the pre-stored footwork scores from the dance music library according to the beat line and song style of the choreography music to obtain footwork score fragments with the corresponding beat; and simulate the virtual limb dance of the 3D animated character based on the character movement data corresponding to the footwork score fragments.

[0148] (II) Racing Footwork Mode:

[0149] In this mode, the pedaling keys are related to the beat points, rhythm details, lyrics, etc., requiring the pedaling keys to be on the beat points and small beat details.

[0150] (III) Personalized Choreography Mode:

[0151] In this mode, users can edit various fancy footwork patterns, making it easy for players to freely create choreography patterns. The footwork patterns can be edited according to the style of the choreography music. After editing, the beat points of the choreography pattern will be automatically corrected to ensure that the stepping keys on the choreography pattern are completely consistent with the beat points.

[0152] As described in the above embodiments, the personalized choreography mode can provide different players with a free creative space. Therefore, in order to meet the choreography needs of dances of different difficulty, this application provides the following types of choreography based on the level of difficulty:

[0153] A, Foot Pedal Single-Key Footwork Dance; Specifically, Foot Pedal Single-Key Footwork refers to determining the optimal key position for each beat based solely on the timing of the beat and the motion capture camera.

[0154] B, Foot Pedal Single Key Footwork and Gesture Dance; Specifically, in this choreography, the foot pedal single key footwork involves stepping on only one key at each beat point, and includes gesture movements.

[0155] C, Pedal Multi-Key Footwork Dance; Specifically, pedal multi-key footwork refers to multiple key information for each beat point based on the pressure threshold on the time window and the direction of human skeletal movement obtained by the motion capture camera being consistent with the key direction.

[0156] D, a dance with multiple key pedals and hand gestures; specifically, the multi-key pedals in this choreography involve stepping on multiple keys at each beat and includes hand gestures.

[0157] For example, the difficulty levels of single-key pedal dance (A), single-key pedal dance with hand gestures (B), multi-key pedal dance (C), and multi-key pedal dance with hand gestures (D) can be categorized into 1 star, 2 stars, 3 stars, and 4 stars. In type A, only one key with a specific pedal direction is allowed per beat point, and each beat point has one arrow on the pedal. The recognition rule combines the key direction signal and human motion data to identify which key the user is currently pressing. In type B... The system simultaneously identifies the key positions in the direction of a footstep and the direction of the user's hand gestures. In type C, multiple arrows in the direction of a footstep can be placed simultaneously within a beat point, and the system comprehensively determines the multiple key positions to be stepped on based on the key position direction signals and human motion data. In type D, while recognizing multiple arrows on the foot pedals, the system also recognizes the user's hand movements based on the key position direction signals and human motion data. Through the above technical solutions, the game difficulty can be gradually increased to achieve choreography diversification, meet the needs of players at different levels, and improve the user experience.

[0158] In one embodiment, in order to continuously improve the quality of users' choreography, this application also provides a technical solution for scoring and ranking, which adopts a similarity calculation scoring and a multi-dimensional feature joint evaluation strategy.

[0159] Based on this, refer to Figure 9 As shown, Figure 9 This is a flowchart illustrating the sorting process of choreography footnotes. The dance editing method of this application can also acquire the choreography footnotes of a choreographed dance; calculate the similarity between the automatically choreographed choreography footnotes and pre-stored footnotes in a dance music library; evaluate the choreography based on the similarity to obtain a score; and recommend the choreography to other users based on the score. Furthermore, after recommending the choreography, the method can also acquire evaluation data of the recommended choreography; determine the weights of each evaluation indicator using the analytic hierarchy process; calculate the popularity value of the choreography based on the evaluation data and weights; and sort the choreography to be recommended based on the popularity value.

[0160] Based on the above embodiments, for example, the music (song id), footwork (feet id), 3D character animation (action id), and 3D character motion data (3D move id) of the choreography can be set. Then, the choreography can be uploaded to the server for scoring and ranking, and the points can be calculated and credited to the user's account. According to the user's personal music style, music of the corresponding style and ranked at the top can be recommended to the user on a regular basis for the user to use.

[0161] Specifically, the server uses cosine similarity to compare the choreography sheet music with sheet music in the dance music library. The lower the similarity, the higher the creativity, which is used to score the user's level of innovation. Then, based on the choreography sheet music uploaded by users, and according to its style type, the server periodically calculates the popularity value of new choreography sheet music. The analytic hierarchy process (AHP) is used to determine the weight of each indicator (such as likes, views, comment quality, shares, upvotes / downvotes, remote PK value, fluency, and coherence), thereby calculating the ranking of the corresponding styles. The calculation formula is as follows:

[0162]

[0163] In the formula, H represents coherence, A represents likes, B represents views, C represents comment quality, D represents shares, E represents upvotes and downvotes, F represents remote PK value, and G represents fluency. is a coefficient.

[0164] I. Regarding the incentive mechanism indicators, the higher the number of likes, comments, reposts, and views, the higher the weight given to its ranking calculation.

[0165] II. The number of upvotes and downvotes reflects how much other players like the choreography. The higher the number of upvotes, the higher its weight in the ranking calculation. However, the higher the number of downvotes, the higher its weight in the ranking penalty, which may lower the ranking of the choreography or even delete it and store it only in the personal account.

[0166] III. Remote PK function: This function allows players to engage in online PK battles with players from all over the world through an online platform. PK battles will be ranked, and players who rank higher and have more participants will receive higher ranking scores.

[0167] IV. Smoothness and continuity scoring: Players can judge the continuity and smoothness of the choreography based on their own gaming experience. If the smoothness is low but the other parameters are ranked high, the choreography can be recommended by professional choreographers and advanced speedrunners.

[0168] The technical solutions described above can enhance players' creative enthusiasm, especially for advanced players who can create highly challenging racing footwork patterns and highly artistic freestyle footwork patterns according to their own abilities.

[0169] The following describes an embodiment of the dance editing device.

[0170] refer to Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of a dance editing device according to one embodiment, including:

[0171] The music processing module 10 is used to acquire choreography music and identify each beat point of the choreography music, and play the choreography music to the user to assist in choreography.

[0172] The information acquisition module 20 is used to collect the key direction signals generated by the foot pedal at each beat point when the user is choreographing, and to acquire human motion data at the corresponding beat point position through the motion capture camera.

[0173] The joint determination module 30 is used to jointly determine the pedaling key corresponding to each beat point based on the key direction signal and the human motion data to obtain the valid pedaling key.

[0174] The footwork generation module 40 is used to generate choreography footwork based on the choreography music and the valid step keys corresponding to each beat point.

[0175] The dance editing device of this embodiment can execute a dance editing method provided in the embodiments of this disclosure. The implementation principle is similar. The actions performed by each module in the dance editing device in each embodiment of this disclosure correspond to the steps in the dance editing method in each embodiment of this disclosure. For detailed functional descriptions of each module of the dance editing device, please refer to the descriptions in the corresponding dance editing methods shown above. They will not be repeated here.

[0176] The following describes an example of a dance machine.

[0177] The dance machine provided in this application includes: a main unit, and a display device, a motion capture camera, and foot pedals connected to the main unit; wherein the main unit is also connected to a server.

[0178] In use, the host is configured to perform the steps of the dance editing method of any embodiment; the motion capture camera is used to capture image data of the choreography and output human motion data; the foot pedal is used to output key position direction signals of the key area being stepped on; and the display device is used to display screen information.

[0179] The following describes an embodiment of the computer device of this application, which includes:

[0180] One or more processors;

[0181] Memory;

[0182] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to: perform a dance editing method according to any of the above embodiments.

[0183] The following describes an embodiment of the computer-readable storage medium of this application, wherein the storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded by the processor and executes the dance editing method of any of the above embodiments.

[0184] The aforementioned dance editing device, dance machine, computer equipment, and computer-readable storage medium enable accurate choreography on the dance machine, reducing the difficulty and complexity of choreography, lowering the barrier to entry for ordinary players, and improving the user experience.

[0185] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A dance editing method, characterized in that, include: Acquire choreography music and identify each beat point of the choreography music, and play the choreography music to the user to assist in choreography; When users are choreographing, the beat points in the choreography music are used as timestamps. Based on these timestamps, the key direction signals generated by the foot pedals are collected at each beat point, and human motion data at the corresponding beat point is obtained through motion capture cameras. The effective pedaling keys are determined by jointly judging the pedaling keys corresponding to each beat point based on the key direction signal and the human motion data. This includes: when the collected key direction signal reaches the effective pedaling pressure threshold, the key corresponding to the key direction signal is output as the effective pedaling key; when the collected key direction signal does not reach the effective pedaling pressure threshold, the relative distance and relative angle between each limb are calculated based on the human motion data; if both the relative distance and relative angle reach the judgment threshold, the key corresponding to the key direction signal is output as the effective pedaling key. The choreography footwork is generated based on the choreography music and the valid step keys corresponding to each beat point.

2. The dance editing method according to claim 1, characterized in that, After generating the choreography footwork based on the valid footwork keys corresponding to each beat point, it also includes: The choreography footwork is divided into several segmented footwork patterns; Calculate the mean local deviation between each segment of the score and the beat line; where the beat line is an auxiliary line for the beat point. The segmented footnotes are initially corrected based on the mean local deviation of each segmented footnote; Calculate the global mean deviation between the choreography footwork and the beat line; The choreography footwork is then revised a second time based on the global mean deviation.

3. The dance editing method according to claim 1, characterized in that, Based on the choreography music and the valid footwork keys corresponding to each beat point, a choreography footwork notation is generated, including: Obtain the beat lines corresponding to each beat point of the choreography music; wherein, the beat lines are beat point auxiliary lines; Align the valid step keys corresponding to each beat point with the beat line, and generate choreography footwork by combining it with the choreography music.

4. The dance editing method according to claim 1, characterized in that, Also includes: Based on the human motion data, obtain the hand position information corresponding to each beat point; Generate hand gestures corresponding to each beat point based on the hand position information; A choreographed dance is generated based on the gestures and choreography footwork.

5. The dance editing method according to claim 4, characterized in that, Also includes: A virtual dance video corresponding to the choreographed dance is generated using 3D rendering technology and based on the human motion data. The virtual dance video is stored on a server for synchronized demonstration of animated characters when the choreography is played.

6. The dance editing method according to claim 4, characterized in that, When users choreograph, it also includes: The video image flashes at each beat point; Prompt the user to prepare to perform a step button or gesture action; After the user steps on a key, the key is displayed to the user via video or a 3D gesture is demonstrated simultaneously.

7. The dance editing method according to claim 4, characterized in that, The choreography includes: single-key footwork dance, single-key footwork dance and hand gesture dance, multi-key footwork dance, or multi-key footwork dance and hand gesture dance. The single-key footwork refers to pressing only one key at each beat, while the multi-key footwork refers to pressing multiple keys at each beat.

8. The dance editing method according to claim 3, characterized in that, Acquire the choreography music and identify the various beat points of the choreography music, including: Acquire at least one piece of music from the signal source; Choreography music is created by splicing together at least one segment from each piece of music. STFT spectrum analysis is performed on the choreography music to obtain the beat points corresponding to the time of the choreography music, and a beat line aligned with the beat points is generated.

9. The dance editing method according to claim 1, characterized in that, The key direction signals generated by the foot pedal are collected at each beat point, including: Each beat point is used as a timestamp, and a signal capture window is determined according to a set time range. Within the capture window, the changing signal output when the foot pedal is stepped on is continuously detected.

10. The dance editing method according to claim 1, characterized in that, Also includes: Obtain the rhythm modifications made by the user to the choreography music; The choreography footwork is then edited a second time based on the modified choreography music, and the footwork is then refined based on the second editing operation.

11. The dance editing method according to claim 4, characterized in that, Also includes: Obtain the choreography footwork for the dance; Calculate the similarity between the choreography footwork and the pre-stored footwork in the dance music library, and evaluate the choreography based on the similarity to obtain a score. The choreographed dance will be recommended to other users based on the rating score.

12. The dance editing method according to claim 11, characterized in that, Also includes: Obtain evaluation data for the recommended choreography; The weights of each evaluation indicator were determined using the analytic hierarchy process (AHP). The popularity score of the choreographed dance is calculated based on the evaluation data and weights. The recommended choreography dances are sorted according to the popularity value.

13. The dance editing method according to claim 11, characterized in that, Also includes: Matching the pre-stored footwork scores from the dance music library according to the beat line and song style of the choreography music, and obtaining the footwork score fragments corresponding to the beat; Based on the character movement data corresponding to the footwork fragments, a virtual limb dance of a simulated 3D animated character is created.

14. The dance editing method according to claim 2, characterized in that, After generating the choreography footwork based on the valid footwork keys corresponding to each beat point, it also includes: Request a list of beat icon and stomp effect from the server's skin library; The footwork editing toolbar displays a list of beat point icons and a list of stomping effect special effects; Get the target beat point icon style and target stomp effect style selected by the user from the beat point icon list and stomp effect list; Replace the beat point icon style in the editing area with the target beat point icon style, and control the beat line to demonstrate the target stomping effect style.

15. A dance editing device, characterized in that, include: The music processing module is used to acquire choreography music and identify the beat points of the choreography music, and play the choreography music to the user to assist in choreography. The information acquisition module is used to collect key direction signals generated by the foot pedal at each beat point position based on the beat points in the choreography music when the user is choreographing, and to acquire human motion data at the corresponding beat point position through the motion capture camera. The joint determination module is used to jointly determine the valid pedaling keys corresponding to each beat point based on the key direction signal and the human motion data; including: when the collected key direction signal reaches the effective pedaling pressure threshold, the key corresponding to the key direction signal is output as the valid pedaling key; when the collected key direction signal does not reach the effective pedaling pressure threshold, the relative distance and relative angle between each limb are calculated based on the human motion data; if both the relative distance and relative angle reach the determination threshold, the key corresponding to the key direction signal is output as the valid pedaling key. The footwork generation module is used to generate choreography footwork based on the choreography music and the valid step keys corresponding to each beat point.

16. A dancing machine, characterized in that, include: The host computer, and a display device, motion capture camera and foot pedal connected to the host computer; wherein the host computer is also connected to a server; The host computer is configured to perform the steps of the dance editing method according to any one of claims 1-14; The motion capture camera is used to capture image data of the choreography and output human motion data; The foot pedal is used to output a key direction signal that the key area is being pressed; The display device is used to display screen information.

Images