Virtual dance processing method and device, electronic equipment and storage medium

Abstract

The invention provides a virtual dance processing method and device, electronic equipment and a storage medium. The method comprises the steps that a virtual scene is displayed, and the virtual scene comprises a virtual object; in response to a dance editing trigger operation, playing a target audio, and in a process of playing the target audio, obtaining action data of a target object, and controlling the virtual object to execute a dance action corresponding to the action data; and in response to completion of the dance editing operation, dance content is generated, and the dance content comprises the dance action and the target audio. According to the method and the device, the dance arrangement cost in the game can be reduced, meanwhile, a user can be helped to improve dance skills, and brand-new dance experience is provided for the user.

Description

Technical Field

[0001] The present application relates to the technical field of computer human-computer interaction, and in particular to a virtual dance processing method, device, electronic device and storage medium. Background Art

[0002] Display technology based on graphics processing hardware has expanded the channels for perceiving the environment and obtaining information, especially the display technology of virtual scenes. It can realize diversified interactions between virtual images controlled by users (or players) or artificial intelligence according to actual application needs, and has various typical application scenarios. For example, in virtual scenes such as games, it can simulate the real battle process between virtual images.

[0003] Taking dancing games as an example, in the related technology, all dances and arrangements need to be preset in the game, and a large amount of R&D manpower is needed to develop dances and arrangements, which leads to a great limitation on the number of dances in the game. At the same time, users cannot customize their own virtual images, which is not very friendly to some relatively shy users, resulting in a poor gaming experience for users. Summary of the invention

[0004] The embodiments of the present application provide a virtual dance processing method, device, electronic device, computer-readable storage medium and computer program product, which can reduce the cost of arranging dances in games, and at the same time can help users improve their dancing skills, providing users with a new dancing experience.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application embodiment provides a virtual dance processing method, including:

[0007] Displaying a virtual scene, wherein the virtual scene includes a virtual object;

[0008] In response to a choreography trigger operation, playing target audio, and

[0009] During the process of playing the target audio, obtaining the motion data of the target object, and controlling the virtual object to perform dance movements corresponding to the motion data;

[0010] In response to completing the choreography operation, dance content is generated, wherein the dance content includes the dance movements and the target audio.

[0011] The embodiment of the present application provides a virtual dance processing device, including:

[0012] A display module, used for displaying a virtual scene, wherein the virtual scene includes a virtual object;

[0013] A playing module, used for playing target audio in response to a choreography trigger operation;

[0014] An acquisition module, used for acquiring the motion data of the target object during the process of playing the target audio;

[0015] A control module, used for controlling the virtual object to perform dance movements corresponding to the movement data;

[0016] A generating module is used for generating dance content in response to completing the choreography operation, wherein the dance content includes the dance movements and the target audio.

[0017] An embodiment of the present application provides an electronic device, including:

[0018] A memory for storing executable instructions;

[0019] The processor is used to implement the virtual dance processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0020] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the virtual dance processing method provided in the embodiment of the present application when executed by a processor.

[0021] An embodiment of the present application provides a computer program product, including a computer program or a computer executable instruction, which is used to implement the virtual dance processing method provided in the embodiment of the present application when executed by a processor.

[0022] The embodiments of the present application have the following beneficial effects:

[0023] During the choreography process, users only need to follow the target audio to choreograph dance moves to obtain the corresponding dance content, thereby reducing the difficulty and cost of choreography. At the same time, through the mapping of virtual objects, on the one hand, it can protect the user's personal privacy. On the other hand, users can observe their own dance moves through virtual objects and adjust their dance moves in time, so as to continuously improve their dance skills, providing users with a brand new dance experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of the architecture of a virtual dance processing system 100 provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application;

[0026] Figure 3 is a flowchart of a method for processing a virtual dance provided in an embodiment of the present application;

[0027] Figure 4 is a flowchart of a method for processing a virtual dance provided in an embodiment of the present application;

[0028] Figure 5A and Figure 5B is a flowchart of a method for processing a virtual dance provided in an embodiment of the present application;

[0029] Figure 6 is a process diagram of a virtual dance processing method provided in an embodiment of the present application;

[0030] FIG. 7A to FIG. 7C It is a schematic diagram of an application scenario of a virtual dance processing method provided in an embodiment of the present application;

[0031] Figure 8 It is a flowchart of a method for processing virtual dance provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0033] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] It is understandable that in the embodiments of the present application, when user information and other related data (such as user dance data) are involved, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0035] In the following description, the terms "first\second\..." are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0037] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0038] 1) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed may be executed.

[0039] 2) Virtual scene: It is the scene displayed (or provided) when the application is running on the terminal device. The scene can be a simulation environment of the real world, a semi-simulation and semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene. For example, the virtual scene may include the sky, land, ocean, etc. The land may include environmental elements such as deserts and cities, and the user can control the virtual image to move in the virtual scene.

[0040] 3) Virtual objects: images of various people and objects that can interact in a virtual scene, or movable objects in a virtual scene. The movable objects can be virtual characters, virtual animals, cartoon characters, etc., such as characters and animals displayed in a virtual scene. A virtual image can be a virtual image in a virtual scene that represents a user. A virtual scene can include multiple virtual images, each of which has its own shape and volume in the virtual scene and occupies a part of the space in the virtual scene. A virtual image can also be a game character controlled by a user (or player).

[0041] 4) Scene data: characteristic data representing a virtual scene, such as the area of ​​the construction area in the virtual scene, the current architectural style of the virtual scene, etc.; it may also include the location of the virtual building in the virtual scene, and the area occupied by the virtual building, etc.

[0042] 5) Cloud gaming: Also known as gaming on demand, it means deploying a game program on a server and running an instance of the game program (referred to as a game instance). The game instance sends the game data output during the running process to the browser page of the user terminal. The page calls the media component of the browser to decode the game data and renders the real-time game screen during the game according to the decoding result. When the page detects the operation performed by the user on the game screen, it will report it to the game instance running on the server. When the game data generated by the game instance in response to the operation is received, the decoding and rendering process will be repeated, so that the game screen changes according to the user's operation are presented on the page.

[0043] In other words, cloud gaming is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the user terminal (such as the player's game terminal), but on the cloud server, which renders the game scene into an audio and video stream and transmits it to the user terminal over the network. In this way, the user terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0044] In the process of implementing the embodiments of the present application, the applicant discovered that the TV dancing game provided by the related art has the following disadvantages: 1) All dances and arrangements need to be preset in the game, which requires a large amount of R&D manpower to develop dances and arrangements, resulting in a large limitation on the number of dances in the game; 2) There are restrictions on music copyright, which leads to restrictions on dance creation; 3) Users cannot customize their own virtual images, which makes it less friendly to some relatively shy users.

[0045] In view of this, the embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for processing virtual dance, which can reduce the cost of arranging dances in games, and at the same time can help users improve their dancing skills, providing users with a new dancing experience. The electronic device provided by the embodiments of the present application is described below. The electronic device provided by the embodiments of the present application can be implemented as a terminal device, or can be implemented by a terminal device and a server in collaboration. The following is an example of a method for processing virtual dance provided by the embodiments of the present application in collaboration with a terminal device and a server.

[0046] In the right Figure 1Before the introduction, the game mode involved in the embodiment of the present application is first introduced. The scheme for the collaborative implementation of the terminal device and the server mainly involves two game modes, namely the local game mode and the cloud game mode, wherein the local game mode refers to the terminal device and the server jointly running the game processing logic, the operation instructions entered by the player in the terminal device, part of which is processed by the terminal device running the game logic, and the other part is processed by the server running the game logic, and the game logic processing run by the server is often more complicated and requires more computing power; the cloud game mode refers to the game logic processing run entirely by the server (such as a cloud server), and the cloud server renders the game scene data into an audio and video stream, which is then transmitted to the terminal device through the network for display. In other words, the terminal device only needs to have basic streaming media playback capabilities and the ability to obtain the player's operation instructions and send them to the server.

[0047] For example, see Figure 1 , Figure 1 1 is a schematic diagram of the architecture of a virtual dance processing system 100 provided in an embodiment of the present application, which is used to support reducing the cost of arranging dances in games, and at the same time can help users improve their dancing skills, providing users with a new dancing experience. Figure 1 As shown, the processing system 100 of virtual dance includes: a server 200, a network 300 and a terminal device 400 (i.e., a first electronic device), wherein the server 200 is a background server of a client 410 running on the terminal device 400, the network 300 can be a local area network or a wide area network, or a combination of the two, the terminal device 400 is a terminal device associated with a player, and a client 410 runs on the terminal device 400. The client 410 can be various types of clients, such as a dance game client, a competitive game client, a browser, etc.

[0048] In some embodiments, taking the target object as player 1 as an example, Figure 1As shown, a virtual scene (e.g., a dance scene selected by player 1) can be displayed in the human-computer interaction interface of client 410, and a virtual object created by player 1 can be displayed in the virtual scene. Then, client 410 can respond to the choreography trigger operation, for example, client 410 receives a click operation of player 1 on the recording button, plays the target audio (e.g., the music uploaded by player 1 to server 200 via mobile phone), and in the process of playing the target audio, obtains the action data of the dance action choreographed by player 1 through the built-in camera of terminal device 400, and controls the virtual object displayed in the virtual scene to synchronously perform the dance action corresponding to the action data, that is, the dance action choreographed by player 1 along with the playing of the target audio can be displayed through the virtual object in the virtual scene. Finally, when client 410 detects that the target audio has been played (i.e., the choreography is completed), dance content can be generated, such as a dance video, or a game level produced according to the dance action and the target audio, wherein the dance content can include the dance action performed by the virtual object following player 1 and the music uploaded by player 1. In this way, players only need to choreograph dance moves in front of the terminal device 400 to obtain the corresponding dance content in real time, which reduces the difficulty and cost of choreography in dancing games. In addition, through the mapping of virtual objects, on the one hand, the privacy of players can be protected. On the other hand, players can also observe their own dance moves through virtual objects and adjust their dance moves in time, so as to continuously improve their dancing skills, providing players with a brand new dancing experience.

[0049] In other embodiments, the embodiments of the present application can also be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing.

[0050] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0051] For example, Figure 1The server 200 in the example may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms. The terminal device 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, a car terminal, etc., but is not limited thereto. The terminal device 400 and the server 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0052] In some embodiments, the terminal device or server can also implement the virtual dance processing method provided in the embodiments of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a native application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a dance game APP; it can also be a small program that can be embedded in any APP, that is, a program that only needs to be downloaded to a browser environment to run. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0053] The structure of the electronic device provided in the embodiment of the present application is further described below. Taking the electronic device as a terminal device as an example, see Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application, Figure 2 The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 2 Various buses are labeled as bus system 540 .

[0054] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0055] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0056] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 510.

[0057] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0058] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0059] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0060] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0061] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0062] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

[0063] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The processing device 555 of the virtual dance stored in the memory 550 is shown, which can be software in the form of programs and plug-ins, including the following software modules: display module 5551, playback module 5552, acquisition module 5553, control module 5554, generation module 5555, download module 5556, addition module 5557, adjustment module 5558, detection module 5559 and sending module 55510. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. It should be pointed out that in Figure 2 For the sake of convenience, all the above modules are shown at once, but it should not be considered that the virtual dance processing device 555 excludes the implementation of only including the display module 5551, the playback module 5552, the acquisition module 5553, the control module 5554 and the generation module 5555. The functions of each module will be explained below.

[0064] The following will specifically describe the method for processing virtual dance provided in the embodiment of the present application in combination with the exemplary application and implementation of the first electronic device (such as a smart TV) provided in the embodiment of the present application.

[0065] See also Figure 3 , Figure 3 is a flowchart of a method for processing virtual dance provided in an embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0066] It should be noted that Figure 3 The method shown can be executed by various forms of computer programs running on the first electronic device, and is not limited to the client. For example, it can also be the operating system, software module, script, and applet described above, so the example of the client below should not be regarded as a limitation on the embodiments of the present application. In addition, for the sake of convenience, the first electronic device and the client running on the first electronic device are not specifically distinguished below.

[0067] In step 101, a virtual scene is displayed.

[0068] Here, the virtual scene may include virtual objects, such as a virtual object (or virtual image) created by a target object (such as player 1), or a virtual object pre-existing in the virtual scene. For example, a virtual object selected by player 1 from multiple candidate virtual objects pre-existing in the virtual scene may be used as the virtual object corresponding to player 1.

[0069] In some embodiments, the first electronic device (e.g., a smart TV) may implement step 101 in the following manner: displaying a scene selection control (e.g., a scene selection button); in response to a trigger operation on the scene selection control, displaying multiple scene identifiers (e.g., thumbnails of virtual scenes or names of virtual scenes, etc.) corresponding one to one to multiple candidate virtual scenes; in response to a selection operation on multiple scene identifiers, displaying a virtual scene corresponding to the selected scene identifier; in response to a virtual object creation operation, displaying a virtual object created by a target object (e.g., user 1) in the virtual scene corresponding to the selected scene identifier.

[0070] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1), a dancing game APP is installed on the smart TV. When receiving a click operation of player 1 on the icon of the dancing game APP, the homepage of the dancing game APP can be displayed, and a scene selection button can be displayed on the homepage for player 1 to select a favorite dancing scene. For example, when receiving a click operation of player 1 on the scene selection button, multiple scene identifiers corresponding to multiple candidate dancing scenes can be displayed on the screen of the smart TV. For example, 5 thumbnails corresponding to 5 candidate dancing scenes can be displayed, which are thumbnails 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 50, 51, 52, 53, 54, 59, 51, 52, 53, 59, 50, 51, 52, 53, 54, 59, 50, 51, 52, 53, 59, 50, 51, 52, 53, 59, 50, 50, 50, 50, 51 Figure 1 To thumbnail 5. Assume that player 1 receives a request for thumbnail Figure 2 Click to display the thumbnail Figure 2 Finally, player 1 can also select the dance scene (i.e. the abbreviated dance scene) Figure 2 For example, after player 1 selects a dance scene, multiple candidate virtual images can be displayed in the dance scene selected by player 1, and the virtual image selected by player 1 from the multiple candidate virtual images can be used as the virtual image created by player 1.

[0071] It should be noted that the virtual image can also be created through the avatar uploaded by Player 1. For example, after Player 1 uploads his or her avatar, the game server can generate a corresponding virtual image based on the avatar uploaded by Player 1. The embodiment of the present application does not specifically limit the method of creating the virtual image.

[0072] In other embodiments, the first electronic device may also implement the above step 101 in any of the following ways: displaying the most frequently used virtual scene, displaying the last selected virtual scene, or displaying the highest-rated virtual scene.

[0073] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1), a dancing game APP is installed on the smart TV. When receiving a click operation of player 1 on the icon of the dancing game APP, the virtual scene last selected by player 1 can be displayed first, or the virtual scene with the highest score can be displayed, or the virtual scene selected by most players can be displayed. When player 1 is not satisfied with the currently displayed virtual scene, he can click the scene selection button to switch, so that the player can save time in selecting virtual scenes.

[0074] In step 102, in response to a choreography trigger operation, a target audio is played.

[0075] Here, the target audio refers to the audio associated with the virtual scene, for example, it can be music uploaded by the player by scanning a code, or music selected by the player from multiple music pre-stored on the first electronic device (such as a smart TV) to be used as background music for the player's subsequent choreography.

[0076] In some embodiments, after the target object (e.g., player 1) selects a virtual scene and creates his or her own virtual image in the virtual scene, the target object can start choreography. For example, the first electronic device can respond to a choreography trigger operation, such as receiving a click operation of a recording button by the target object, and play target audio, which may be music uploaded by the target object by scanning a code, or music selected from multiple candidate music.

[0077] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1), after player 1 selects a dance scene and creates his own virtual image in the dance scene, he can start recording. For example, when player 1 clicks on the recording button displayed in the dance scene, music associated with the dance scene can start playing. For example, it can be music uploaded by player 1 by scanning a QR code displayed on the smart TV. The details are explained below.

[0078] In some embodiments, the above-mentioned virtual scene may also include a graphic code (such as a QR code or a barcode), wherein the graphic code is used for a second electronic device (such as a mobile phone of user 1) to perform the following processing: in response to a scanning operation on the graphic code, multiple candidate audios (such as multiple candidate music files) are displayed in the human-computer interaction interface of the second electronic device; in response to a selection operation on multiple candidate audios, the selected audio from the multiple candidate audios is sent to the server.

[0079] For example, the first electronic device is a smart TV, and the second electronic device is a mobile phone. A dance scene selected by a target object (such as player 1) is displayed in the human-computer interaction interface of the smart TV. A QR code can be displayed in the dance scene for player 1 to upload his favorite music. For example, when player 1 scans the QR code displayed on the smart TV through a mobile phone, player 1's mobile phone will jump to the music selection interface, and multiple candidate music files are displayed in the music selection interface. For example, assuming that 5 music files are displayed, namely music files 1 to music files 5, player 1 can select his favorite music file from these 5 music files as background music. For example, assuming that a click operation of player 1 on music file 2 is received, music file 2 can be used as the music file selected by player 1 and uploaded to the game server. In other words, players can expand their horizons by selecting different dance scenes and music. Players can try different dance types, such as modern dance, street dance, etc., or explore different music styles, such as pop music, electronic music, etc. This process allows players to better understand different art forms and increase their aesthetic vision.

[0080] In some other embodiments, following the above examples, see Figure 4 , Figure 4 is a flow chart of a method for processing a virtual dance provided in an embodiment of the present application, such as Figure 4 As shown, the first electronic device executes Figure 3 Before step 102 shown, you can also perform Figure 4 Step 105 shown combines Figure 4 The steps shown are explained.

[0081] In step 105, in response to receiving the upload completion instruction sent by the second electronic device, the selected audio is downloaded from the server as the target audio.

[0082] In some embodiments, taking the first electronic device as a smart TV associated with the target object (e.g., player 1) and the second electronic device as a mobile phone associated with the target object as an example, player 1 can use the mobile phone to scan the QR code displayed on the smart TV and select his favorite music to upload to the game server. After successfully uploading the music selected by player 1 to the game server, player 1's mobile phone can send an upload completion instruction to the smart TV. After receiving the upload completion instruction sent by player 1's mobile phone, the smart TV can download the music just uploaded by player 1 from the game server as the background music (i.e., target audio) for the dance scene.

[0083] In other embodiments, after downloading the audio selected by the target object from the server, the first electronic device may also perform the following processing: displaying virtual decorations (such as virtual flowers, virtual pets, etc.) associated with the selected audio at a set position in the virtual scene, wherein the set position includes one of the following: the position selected by the target object in the virtual scene, the position of the virtual image in the virtual scene.

[0084] For example, taking the first electronic device as a smart TV as an example, after the smart TV downloads the music just uploaded by the player from the game server, the game server can automatically recommend virtual decorations associated with the music to the player, so that the player can use the virtual decorations to decorate the virtual scene. For example, after the smart TV downloads the music just uploaded by the player from the game server, the game server can send virtual decorations matching the type of the music to the smart TV. After receiving the virtual decorations sent by the game server, the smart TV can automatically display them at the location where the virtual image is located, for example, the virtual decorations sent by the game server can be automatically displayed on the head or hand of the virtual image. Of course, the virtual decorations sent by the game server can also be displayed at the location selected by the player. For example, after the smart TV downloads the music just uploaded by the player from the game server, it can display a prompt message to prompt the player to select the location to be decorated, and then the virtual decorations sent by the game server can be displayed at the location selected by the player in the virtual scene. In this way, the player can better exert his imagination and increase the player's creative thinking.

[0085] It should be noted that, in addition to being uploaded via a graphic code, the target audio may also be pre-stored in the first electronic device. For example, after the player selects a virtual scene, multiple candidate audios may be further displayed, wherein the multiple candidate audios are pre-stored in the first electronic device. Subsequently, the audio selected by the player from the multiple candidate audios may be used as the target audio. The embodiments of the present application do not specifically limit this.

[0086] In addition, it should be noted that the order in which players select virtual scenes and audio can be reversed, that is, players can first select audio and then select virtual scenes. For example, after players upload music by scanning with their mobile phones, multiple candidate virtual scenes associated with the music uploaded by the players can be displayed on the smart TV, such as multiple candidate virtual scenes that match the type of music uploaded by the players. Subsequently, the virtual scene selected by the player from the multiple candidate virtual scenes can be used as the final virtual scene (i.e., the dancing scene) and displayed on the smart TV.

[0087] In some embodiments, in order to better personalize their own dance video, the player can also choose to add special effects in the virtual scene, and the first electronic device will execute Figure 3Before step 102 shown, the following processing can also be performed: in response to an editing trigger operation (such as receiving a player's click operation on an edit button), display at least one of the timeline of the target audio and the rhythm graph of the target audio in a virtual scene; in response to a playback progress setting operation on the timeline, use the time point corresponding to the set playback progress as the set time point, and add special effects (such as flash light effects, rainbow effects, etc.) at the set time point; display prompt information at key rhythm points in the rhythm graph (such as the point where the main song switches to the chorus), and in response to a trigger operation on the prompt information, add special effects at the key rhythm points.

[0088] For example, taking the smart TV associated with the first electronic device as the target object (such as player 1) as an example, after the smart TV downloads the music file just uploaded by player 1 from the game server, it can read the music file and display the time progress bar (i.e., timeline) and the rhythm graph of the music. For example, in the editing interface, the top is a virtual scene, and the bottom is the rhythm graph and time progress bar of the music. Player 1 can pull the time progress bar to add special effects at the corresponding music time point (such as the 20th second); or, a prompt message will appear at the key rhythm point in the music rhythm graph, and player 1 can directly add special effects at the key rhythm point by clicking the prompt message, so that players can not only better personalize their works, but also increase the creativity of players. Players can freely choose different special effects, such as flash light effects, rainbow effects, etc., or they can create new special effects by themselves. This process allows players to better use their imagination and increase their creative thinking.

[0089] It should be noted that the above-mentioned special effects may be pre-stored in the first electronic device, for example, they may be special effects previously downloaded by the first electronic device from a server, or they may be special effects newly pushed by the server, or they may be special effects uploaded by the player himself. The embodiment of the present application does not specifically limit the source of the special effects.

[0090] In some embodiments, following the above example, the first electronic device may also perform at least one of the following processing while playing the target audio: displaying special effects in a virtual scene in response to the playback progress of the target audio reaching a set time point; displaying special effects in a virtual scene in response to the playback progress of the target audio reaching a key rhythm point.

[0091] For example, taking the smart TV associated with the first electronic device as the target object (such as player 1), when the smart TV receives the click operation of player 1 on the recording button, it starts to play the music uploaded by player 1. When it detects that the playing progress of the music reaches the set time point (such as the 20th second), the rainbow special effect pre-added by player 1 can be displayed in the virtual scene; or, when it detects that the playing progress of the music reaches the key rhythm point, the flash light special effect pre-added by player 1 is displayed in the virtual scene. In this way, not only can players better personalize their dance works, but also increase the creativity of players. Players can freely choose different special effects, such as rainbow special effects, flash light special effects, etc., or they can create new special effects by themselves. This process allows players to better use their imagination and increase their creative thinking.

[0092] It should be noted that the special effects can be displayed in full screen or non-full screen in the virtual scene. When the special effects are displayed in a non-full screen manner in the virtual scene, the special effects can be displayed at a set position in the virtual scene. For example, when the player sets the display time point of the special effects, he can also further set the display position of the special effects in the virtual scene. Then, when it is detected that the playback progress of the music reaches the set time point, the special effects can be displayed at the position set by the player. The embodiments of the present application do not specifically limit this.

[0093] In step 103, during the process of playing the target audio, the motion data of the target object is obtained, and the virtual object is controlled to perform dance movements corresponding to the motion data.

[0094] In some embodiments, the motion data of the target object can be obtained in the following ways: using an image acquisition device (such as a camera) to acquire image data of the target object when choreographing dance movements; analyzing the skeletal motion trajectory of the target object from the image data; and determining the three-dimensional skeletal data of the target object when choreographing dance movements based on the skeletal motion trajectory as the motion data of the target object.

[0095] For example, taking the first electronic device as the target object (such as player 1) associated with the smart TV as an example, the smart TV can also identify the dance movements choreographed by player 1 in front of the smart TV by calling the built-in (or external) camera of the smart TV during the process of playing the music uploaded by player 1. For example, the smart TV can capture the real-time image data of the dance movements choreographed by player 1 through the camera, wherein the dance movements can be single movements or several groups of serial movements. For the camera, a depth camera or a wide-angle camera, etc. can be used. Player 1 needs to perform choreography and dance within the shooting range of the camera. For example, when the smart TV detects that player 1 is outside the shooting range of the camera, a prompt message can be displayed to prompt the player to enter the shooting range of the camera to dance. After the real-time image data of player 1 is obtained by shooting, the image data can be analyzed based on 3D modeling technology to obtain the skeletal motion trajectory of player 1 during the dancing process, and then the three-dimensional skeletal data of player 1 when choreographing the dance movements can be formed according to the skeletal motion trajectory, wherein the three-dimensional skeletal data can describe the skeletal key points of each dance movement.

[0096] In other embodiments, the motion data of specific parts of the target object (such as hands or feet) collected by the motion capture device can also be combined to form three-dimensional skeleton data of the target object when choreographing dance movements. The above-mentioned determination of the three-dimensional skeleton data of the target object when choreographing dance movements based on the skeleton motion trajectory can be achieved in the following way: the motion data of the target part (such as hands or feet) of the target object when choreographing dance movements is collected by the motion capture device; the three-dimensional skeleton data of the target object when choreographing dance movements is determined based on the skeleton motion trajectory and the target part motion data.

[0097] For example, taking the first electronic device as a target object (e.g., player 1) associated with a smart TV as an example, while the smart TV captures the real-time image data of the dance movements of player 1 through a camera, it can also collect the specific part movement data of player 1 when dancing (e.g., including foot movement data and hand movement data, etc.) through a connected motion capture device (e.g., including a dance pedal, a dance mat, a sensor bracelet, a sensor handle, etc.). For example, the footsteps data of player 1 when dancing can be captured through a dance pedal, a dance mat, etc. connected to the smart TV, and the hand movement data of player 1 when dancing can be detected through a sensor bracelet, a sensor handle, etc. Subsequently, after analyzing the image data captured by the camera based on 3D modeling technology and obtaining the skeletal motion trajectory of player 1 during the dancing process, the skeletal motion trajectory and the motion data captured by the motion capture device can be combined to form the three-dimensional skeletal data of player 1 when dancing, so that the dance movements choreographed by the target object during the audio playback process can be captured more accurately.

[0098] In other embodiments, based on the above example, the above-mentioned control of the virtual object to perform the dance movements corresponding to the action data can be achieved in the following manner: the virtual image is controlled to synchronously perform the dance movements choreographed by the target object according to the three-dimensional skeleton data.

[0099] For example, a smart TV associated with a first electronic device as a target object (e.g., player 1) is used as an example. After the three-dimensional skeleton data of player 1 when dancing is formed by combining the skeleton motion trajectory obtained by analyzing the image data and the motion data collected by the motion capture device, the virtual image can be controlled to synchronously execute the dance movements choreographed by player 1 during the playing of the music according to the three-dimensional skeleton data. In other words, the smart TV can render the three-dimensional skeleton data into a corresponding virtual dance animation according to the virtual image created by player 1, so that the dance movements choreographed by player 1 can be displayed through the virtual image, that is, through the mapping of the virtual image, player 1 can see the action demonstration of his dancing in real time through the smart TV. In other words, when the player clicks the start recording button, the music starts playing, and the camera of the smart TV starts to recognize the dance movements choreographed by the player and records them, and displays the virtual image of the corresponding action on the smart TV. This process allows players to better feel their own dance skills and also increases the self-confidence of players. Players can try different dance moves, such as jumping, turning, etc., and can also continuously improve their dance moves during the recording process to improve their dance skills.

[0100] In step 104, in response to the completion of the choreography operation, dance content is generated.

[0101] Here, the dance content may include dance movements and target audio, wherein the type of the dance content may be a dance video, or a game level produced according to the dance movements and target audio.

[0102] For example, taking the dance content as a dance video, after the target object (such as player 1) completes the choreography, the game server can generate a corresponding dance video based on the dance moves performed by the virtual object and the music uploaded by player 1, and send the dance video to a first electronic device (such as a smart TV) so that the smart TV generates a link to the dance video. Player 1 can then share the link to the dance video on a social platform or share it with friends, so that other players can view the dance video made by player 1 by clicking on the link.

[0103] For example, taking the game level made by players in a dance game as an example, after completing the choreography, player 1 can send the dance movements choreographed by player 1 (i.e., the dance movements performed by the virtual object according to the movement data) and the target audio (e.g., the music uploaded by player 1) to the game server, so that the game server can generate the corresponding game level according to the dance movements and the target audio, that is, the game level made by player 1 can be added to the dance game, so that other players can experience the game level made by player 1 in the dance game after logging into the dance game. For example, in the game level made by player 1, other players need to follow the music uploaded by player 1 to perform the dance movements choreographed by player 1, that is, other players can deeply experience the process of player 1's choreography, which further enriches the game levels of the dance game and improves the game experience of the players.

[0104] It should be noted that, in the case where the dance content is a game level made by player 1 in a dance game, player 1 can also share it on a social platform or with his friends. For example, when a sharing operation triggered by player 1 is received, a corresponding notification message can be displayed on the selected social platform to inform other players that player 1 has made a game level to attract other players to experience it.

[0105] In some embodiments, taking the smart TV associated with the first electronic device as the target object (e.g., player 1) as an example, when the smart TV detects that the music has finished playing (i.e., the dance has finished recording), it can automatically generate a link to the dance video (or a scannable picture), so that player 1 can share his or her dance video on a social platform or with his or her friends through the link.

[0106] It should be noted that the link to the dance video can also be generated after receiving the player 1's click operation on the confirmation generation button. For example, taking the dance content as a dance video, when the smart TV detects that the music has finished playing, it can display a prompt message on whether to generate a link to the dance video, where the prompt information includes a confirmation generation button and a cancel button. When the player's click operation on the confirmation generation button is received, the smart TV can generate a link to the dance video.

[0107] In other embodiments, taking the dance content as a dance video as an example, after the dance video is recorded, the player can also modify some non-standard dance moves in the dance video. Before generating a link to the dance video, the first electronic device can also perform the following processing: display the timeline of the target audio, and multiple skeletal action graphs corresponding to the multiple time points included in the timeline; in response to an adjustment operation on the target skeletal action graph, adjust the dance moves performed by the virtual image at the time point corresponding to the target skeletal action graph, wherein the target skeletal action graph is a skeletal action graph selected from the multiple skeletal action graphs.

[0108] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1), when the smart TV detects that the music uploaded by player 1 has finished playing, that is, after player 1 has finished recording the dance moves, it can display a time progress bar for the music and multiple skeletal action graphs corresponding to the multiple time nodes on the time progress bar, wherein each skeletal action graph includes the dance moves choreographed by player 1 at the corresponding time node. Player 1 can discover his or her own non-standard dance moves by observing the multiple skeletal action graphs, and modify the dance moves of the virtual image by adjusting the skeletal nodes in the skeletal action graphs. For example, taking the multiple skeletal action graphs as 10 skeletal action graphs, which are skeletal action graphs respectively. Figure 1 To the skeleton action diagram 10, when the player 1 finds that the skeleton action shown in the skeleton action diagram 5 is not standard, the player 1 can select the skeleton action diagram 5 and adjust the skeleton nodes of the skeleton action shown in the skeleton action diagram 5, so that the dance action performed by the virtual image at the time node corresponding to the skeleton action diagram 5 (for example, assuming the 50th second) can be adjusted, so that the player can better improve his dance work and enhance his patience and perseverance. The player can repeatedly replay his dance video (i.e., work) to find his own shortcomings and continuously improve his dance video.

[0109] It should be noted that the above-mentioned multiple time points can be obtained by dividing the audio time axis according to a fixed duration (for example, 30 seconds); or they can be obtained by dividing the audio time axis according to a non-fixed duration, for example, the audio time axis can be divided according to the speed of the audio rhythm to obtain multiple time points; of course, the above-mentioned multiple time points can also be obtained by dividing the dance movements choreographed for the target object, for example, the time point when the target object switches from one dance movement to another dance movement can be taken as a corresponding time point, and the embodiments of the present application do not make specific limitations on this.

[0110] In some embodiments, following the above example, the first electronic device may further perform the following processing before responding to an adjustment operation on a target skeletal action graph: in response to a selection operation on multiple skeletal action graphs, the selected skeletal action graph is used as a target skeletal action graph, and the target skeletal action graph is enlarged.

[0111] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1) as an example, when the smart TV detects that the music uploaded by player 1 has finished playing, that is, after player 1 has finished recording the dance moves, the time progress bar of the music and multiple skeletal action graphs corresponding to multiple time nodes on the time progress bar can be displayed, for example, including 10 skeletal action graphs corresponding to 10 time nodes, assuming that they are skeletal actions Figure 1 To the skeleton action diagram 10. When the player 1 finds that the skeleton action included in the skeleton action diagram 5 is not standard, the player 1 can select the skeleton action diagram 5. For example, when the player 1 clicks on the skeleton action diagram 5, the skeleton action diagram 5 can be enlarged, so as to facilitate the player to adjust the skeleton nodes and improve the player's use experience.

[0112] In some embodiments, taking the dance content as a dance video as an example, after the dance video is recorded, the player can also modify some non-standard dance moves in the dance video. Before generating a link to the dance video, the first electronic device can also perform the following processing: obtain multiple skeletal action graphs that correspond one to one to multiple time points included in the timeline of the target audio; for each skeletal action graph, call a machine learning model to detect the skeletal actions included in the skeletal action graph to obtain a detection result, wherein the types of detection results include standard and non-standard; and display the skeletal action graph with a non-standard detection result in a virtual scene.

[0113] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1) as an example, when the smart TV detects that the music uploaded by player 1 has finished playing, that is, the dance video has finished recording, the smart TV can further obtain multiple skeletal action graphs (e.g., 10 skeletal action graphs, each of which is a skeletal action graph corresponding to multiple time nodes on the time progress bar of the music) corresponding to each other. Figure 1 To the skeleton action diagram 10), then for each skeleton action diagram, the smart TV can call the trained machine learning model to detect the posture of the skeleton action included in the skeleton action diagram to obtain the detection result, wherein the type of the detection result can include standard and non-standard. For example, suppose the smart TV calls the machine learning model for detection and determines the skeleton action Figure 4 and skeletal actions Figure 8 If the detection result is not standard, the skeleton action can be displayed in the virtual scene Figure 4 and skeletal actions Figure 8 , so that Player 1 can modify the non-standard dance moves.

[0114] It should be noted that the machine learning model in the embodiments of the present application can be a neural network model (such as a convolutional neural network, a deep convolutional neural network, or a fully connected neural network), a decision tree model, a gradient boosting tree, a multilayer perceptron, and a support vector machine, etc. The embodiments of the present application do not specifically limit the type of machine learning model.

[0115] The following will continue to explain the training process of the above machine learning model.

[0116] In some embodiments, before calling the machine learning model to detect and process the skeletal motion graph, the machine learning model can also be trained in the following way: obtain a sample skeletal motion graph; call the initialized machine learning model to detect the skeletal motion in the sample skeletal motion graph to obtain a detection result; determine the difference between the detection result and the label, and perform back propagation based on the difference, so that the parameters of the machine learning model can be updated layer by layer during the back propagation process, wherein the label type includes "0" and "1", wherein when the label type is "0", it indicates that the skeletal motion in the sample skeletal motion graph is not standard, and when the label type is "1", it indicates that the skeletal motion in the sample skeletal motion graph is standard.

[0117] In other embodiments, the first electronic device may also perform the following processing in response to completing a choreography operation (e.g., detecting that the target audio has finished playing): obtaining motion parameters of the dance movement, wherein the motion parameters include at least one of the following: motion amplitude, frequency, speed; based on the motion parameters, determining the score of the target object, and displaying the score in the virtual scene.

[0118] For example, taking the smart TV associated with the first electronic device as the target object (e.g., player 1), when the smart TV detects that the music uploaded by player 1 has finished playing, that is, the dance video of player 1 has finished recording, it can further obtain the action parameters of the dance movements choreographed by player 1, such as the movement amplitude, frequency, speed, etc. of the dance movements. Subsequently, the smart TV can score the dance movements choreographed by player 1 according to the obtained action parameters, thereby obtaining the final score of player 1. Finally, the smart TV can display the final score of player 1 in the virtual scene selected by player 1.

[0119] In some embodiments, taking the dance content as a dance video as an example, after generating the dance video, a link to the dance video can be further generated, and the player can share the link to a selected social platform (such as a dance community). Figure 3 After step 104 shown, you can also perform Figure 5ASteps 106A and 107A shown in FIG. Figure 5A The steps shown are explained.

[0120] In step 106A, in response to a sharing trigger operation, a plurality of candidate social platforms are displayed.

[0121] In some embodiments, taking the smart TV associated with the first electronic device as the target object (e.g., player 1) as an example, after generating a link to the dance video, the smart TV may further display prompt information on the screen asking whether to share, wherein the prompt information includes a confirm share button and a cancel button. When receiving a click operation of player 1 on the confirm share button, multiple candidate social platforms may be displayed, for example, three different types of candidate social platforms may be displayed.

[0122] In step 107A, in response to a selection operation on a plurality of candidate social platforms, the link shared by the target object is displayed in the selected social platform.

[0123] In some embodiments, following the above example, after the smart TV displays multiple candidate social platforms, player 1 can select from multiple candidate social platforms (for example, three social platforms, namely social platform 1, social platform 2 and social platform 3). For example, assuming that a click operation from player 1 on social platform 2 is received, a link to the dance content shared by player 1 can be displayed on social platform 2 (for example, a dance community) for other players to click and view the dance content posted by player 1.

[0124] In other embodiments, taking the dance content as a dance video as an example, after generating the dance video, a link to the dance video can be further generated, and the player can also share the link with his friends. Figure 3 After step 104 shown, you can also perform Figure 5B Steps 106B and 107B shown in FIG. Figure 5B The steps shown are explained.

[0125] In step 106B, in response to the sharing trigger operation, a plurality of candidate objects are displayed.

[0126] In some embodiments, taking the smart TV associated with the first electronic device as the target object (e.g., player 1) as an example, after generating a link to the dance video, the smart TV may further display prompt information on the screen asking whether to share, wherein the prompt information includes a confirm share button and a cancel button. When receiving a click operation of player 1 on the confirm share button, multiple other players who have a friend relationship with player 1 may be displayed, and player 1 may select a player to be shared from multiple candidate players.

[0127] In step 107B, in response to a selection operation on a plurality of candidate objects, a link is sent to a third electronic device associated with the selected object.

[0128] In some embodiments, following the above example, after the smart TV displays multiple other players who have a friend relationship with player 1 (for example, three players, namely player 2, player 3 and player 4), player 1 can select the player who needs to share from these three players. For example, assuming that a click operation is received from player 1 on the identifier of player 2 (such as an account or nickname), a link to a dance video can be sent to a third electronic device associated with player 2 (such as player 2's mobile phone). For example, the link to the dance video sent by player 1 can be displayed on player 2's mobile phone.

[0129] In other embodiments, taking the dance content as a dance video as an example, the above-mentioned link can also be used for a third electronic device associated with the selected object (such as the mobile phone of player 2) to perform the following processing: display the link in the human-computer interaction interface of the third electronic device; in response to the trigger operation on the link, play the dance video; in response to the completion of the dance video playing, jump to the download page, wherein the download page includes a download control; in response to the trigger operation on the download control, download the client (such as a dance game APP) from the server.

[0130] For example, taking the third electronic device as the mobile phone of player 2, after receiving the link of the dance video sent by the smart TV (i.e., the first electronic device) associated with player 1, the mobile phone of player 2 can display it in the human-computer interaction interface, for example, it can be displayed in the conversation interface with player 1. Subsequently, when a click operation of player 2 on the link is received, the dance video can be played. When it is detected that the dance video has been played, it can automatically jump to the download page, and a download button is displayed on the download page. When a click operation of player 2 on the download button is received, the mobile phone of player 2 will send a download request for the dance game APP to the game server to download the dance game APP from the server.

[0131] The virtual dance processing method provided in the embodiment of the present application is that during the choreography process, the user only needs to choreograph the dance movements facing the first electronic device to obtain the corresponding dance video, thereby reducing the difficulty and cost of choreography. At the same time, through the mapping of virtual objects, on the one hand, the user's personal privacy can be protected, and on the other hand, the user can observe his or her own dance movements while dancing and adjust his or her dance movements in time, so as to continuously improve his or her dance skills, providing the user with a brand new dance experience.

[0132] Below, taking a dancing game as an example, an exemplary application of an embodiment of the present application in an actual application scenario is explained.

[0133] The embodiment of the present application provides an interactive solution for innovative gameplay of dancing games on TV (TeleVision). The solution provides a novel gameplay. A user (e.g., user 1) can create his or her own avatar on the TV. When the TV camera recognizes human body movements, the avatar will be mapped to the TV, and the movements of the avatar will correspond to the movements of the user when dancing. In the dancing game, users can independently select the dancing scene and scan the QR code on the TV to upload music as background music for the dance.

[0134] When the user clicks to start choreography, the music starts playing and the user can start creating his or her own dance. The TV camera will recognize human movements and map the movements of the avatar to the TV. After the user completes the choreography, a shareable link, picture or video can be generated. Taking the link as an example, the user can share the link to a social platform, and other users (such as user 2) can open the link to view the dance video and install the TV version of the dance game applet through the guided tutorial.

[0135] After the installation is complete, user 2 can try to dance the dance that user 1 just created, or create his own virtual image and dance. The technical solution provided by the embodiment of the present application provides a new dancing experience, allowing users to better integrate into dancing and enjoy the fun of dancing. Users can share their creations with friends, so that more people can understand this game and join in the dance together. The embodiment of the present application also provides different scenes and music options, and users can choose the dancing method that suits them according to their preferences.

[0136] In addition, the technical solution provided by the embodiment of the present application can also help users improve their dancing skills. For example, through the mapping of the virtual image, users can observe their own dancing movements, adjust their movements in time, and continuously improve their dancing skills. Users can also learn other users' dancing skills by watching other users' dancing videos, and continuously explore and improve their own dancing level.

[0137] In short, the embodiment of the present application provides a good dancing platform, which can not only allow users to enjoy the fun of dancing, but also help users improve their dancing skills, making users more confident and beautiful. In front of the TV at home, users can dance to their heart's content, which can not only relax their body and mind, but also improve their dancing skills and enjoy the fun of dancing. In addition, the embodiment of the present application also provides a good social platform. In this game, users can communicate with other users, share experiences and skills, and dance and create together. This can not only allow users to have more fun, but also allow users to make more friends.

[0138] The following is a detailed description of the virtual dance processing method provided in the embodiment of the present application.

[0139] In some embodiments, see Figure 6 , Figure 6 is a process diagram of a virtual dance processing method provided in an embodiment of the present application, such as Figure 6 As shown, users can create their own avatars on the TV side. When the TV camera recognizes human movements, the avatar will be mapped to the TV, and the avatar's movements will correspond to the user's dancing movements. In the dancing game, users can choose the dancing scene by themselves, and upload music as the background music of the dance by scanning the QR code on the TV. When the user clicks to start choreography, the music starts playing, and the user can start to create his own dance. The TV camera will recognize human movements and map the avatar's movements to the TV. After the user completes the choreography, a shareable link, picture, or video can be generated. Taking the link as an example, the user can share the link to a social platform, and other users can open the link to view the dancing video and install the TV version of the dancing game applet through the guided tutorial.

[0140] After the installation is complete, other users can try to dance the dance just created, or create their own virtual image and dance. The embodiment of the present application provides a new dance experience, allowing users to better integrate into dancing and enjoy the fun of dancing. Users can share their creations with friends, so that more people can understand this game and join in the dance together. The embodiment of the present application also provides different scenes and music options, and users can choose the dance method that suits them according to their preferences. In addition, the embodiment of the present application can also help users improve their dancing skills. Through the mapping of the virtual image, users can observe their own dancing movements, adjust their movements in time, and continuously improve their dancing skills. Users can also learn other users' dancing skills by watching other users' dancing videos, and constantly explore and improve their dancing level. Other users can also create their own dances and share them, inviting more new users to download the TV version of the dance game applet to form a closed loop of links used by users.

[0141] In some embodiments, see Fig. 7A , Fig. 7A is a schematic diagram of an application scenario of the virtual dance processing method provided in an embodiment of the present application, such as Fig. 7AAs shown, the user can create his own virtual image 701 on the TV side. After creating the virtual image 701, the user can also select a suitable dance scene (i.e., a virtual scene) on the TV side. For example, a "select scene" button 702 (corresponding to the above-mentioned scene selection control) can also be displayed on the TV side. When a user clicks on the "select scene" button 702, thumbnails corresponding to multiple candidate virtual scenes can be displayed. When a user clicks on the thumbnail 703 corresponding to the virtual scene 1, the virtual scene 1 can be used as the virtual scene selected by the user, that is, the dance scene selected by the user for displaying the virtual image 701.

[0142] In other embodiments, after the user uploads music, the game server can automatically recommend virtual scenes and virtual decorations associated with the music to the user, and the user can use the virtual decorations to decorate the virtual scenes. For example, assuming that the type of music uploaded by the user is pop music, the game server can automatically send virtual scenes (such as space scenes) and virtual decorations associated with pop music to the TV end, so that the user can use the virtual decorations to decorate the virtual scene to obtain a unique dance scene, further improving the user's gaming experience.

[0143] In other embodiments, see Figure 7B , Figure 7B is a schematic diagram of an application scenario of the virtual dance processing method provided in an embodiment of the present application, such as Figure 7B As shown, the user can also add special effects in the virtual scene. For example, in the editing interface, the top is the virtual scene 704, and the bottom is the music rhythm graph 705 and the music time progress bar 706. The user can pull the music time progress bar 706 to add special effects at the corresponding music time point. For example, after the user pulls the music time progress bar 706, the user can select the special effect to be added from multiple candidate special effects. For example, assuming that the user selects special effect 708 from multiple candidate special effects, the special effect 708 can be added at the corresponding music time point; or, a prompt message 707 is displayed at a key rhythm point in the music rhythm graph 705. After the user clicks the prompt message 707, the special effect will be directly added at the key rhythm point, for example, adding the special effect 708 selected by the user from multiple candidate special effects at the key rhythm point.

[0144] In some embodiments, see Figure 7C , Figure 7C is a schematic diagram of an application scenario of the virtual dance processing method provided in an embodiment of the present application, such as Figure 7CAs shown, after the user has recorded the dance movements (for example, after the music has finished playing), multiple skeletal action diagrams corresponding to multiple time nodes on the time progress bar 706 of the music can also be displayed on the TV end. If there are non-standard ones, the user can adjust the skeletal nodes to modify the movements of the avatar. For example, when the user finds that the skeletal movements in the skeletal action diagram 709 are not standard, the user can select the skeletal action diagram 709 and modify the skeletal nodes of the skeletal movements displayed in the skeletal action diagram 709 to modify the dance movements performed by the avatar at the corresponding time nodes. In this way, the user can better improve his or her work, discover his or her own shortcomings, and continuously improve his or her work.

[0145] It should be noted that when the user no longer modifies the dance video, the smart TV can generate a link to the dance video or a scannable picture. The user can share his or her dance video to the dance community or square through the link or picture, and other users can view the dance video by clicking the link or scanning the code.

[0146] Next, continue to combine Figure 8 The processing method of virtual dance provided in the embodiment of the present application is described.

[0147] For example, see Figure 8 , Figure 8 is a flowchart of a method for processing virtual dance provided in an embodiment of the present application, which will be combined with Figure 8 The steps shown are explained.

[0148] In step 201, the user TV terminal creates a virtual image in response to a virtual image creation instruction.

[0149] In step 202, the user TV terminal responds to the scene selection operation and uses the selected virtual scene as the dance scene.

[0150] In some embodiments, the user can create a virtual image on the TV side and select a dance scene (i.e., a virtual scene) for which music needs to be recorded. This process can not only improve the user's physical coordination, but also enhance the user's sense of dynamic rhythm. In addition, the user can expand his or her horizons by selecting different dance scenes and music. The user can try different dance types, such as modern dance, street dance, etc., or explore different music styles, such as pop music, electronic music, etc. This process can allow users to better understand different art forms and increase the user's aesthetic vision.

[0151] In step 203, the user's mobile phone responds to the scan trigger operation and scans the QR code displayed on the screen of the user's TV.

[0152] In step 204, the user's mobile phone uploads the music selected by the user to the game server.

[0153] In step 205, the user's mobile phone sends an upload completion instruction to the user's TV.

[0154] In step 206, the user's TV terminal downloads the music just uploaded by the user from the game server and loads it.

[0155] In step 207, the user TV terminal displays a music progress bar and decorations.

[0156] In some embodiments, the user uses a mobile phone to scan the QR code for uploading music on the TV screen, and selects the music file to be uploaded on the mobile phone. The user's mobile phone uploads the selected music file to the game server. After the upload is completed, the user's mobile phone sends an upload completion instruction to the TV. The TV downloads and reads the music file from the game server and displays a music progress bar.

[0157] In step 208, the user TV adds decorations or special effects at the music progress selected by the user.

[0158] In some embodiments, after the user selects the music progress bar, they can choose to add special effects. This process not only allows the user to better personalize their work, but also increases the user's creativity. The user can freely choose different special effects, such as flashing lights, rainbows, etc., or create new special effects by themselves. This process allows the user to better use his imagination and also increases the user's creative thinking.

[0159] In step 209, the user TV receives a click operation on the recording button by the user.

[0160] In step 210, the user's TV starts playing music and calls the camera to start recognizing user actions.

[0161] In some embodiments, the user clicks the start recording button, the music starts playing, the camera starts to recognize the user's movements and record them, and the TV displays a virtual image of the corresponding movements. This process allows the user to better feel their dance skills and increase the user's self-confidence. The user can try different movements, such as jumping, turning, etc., and can also continuously improve their movements during the recording process to improve their dancing skills.

[0162] In step 211, the user TV terminal re-records the action selected by the user in response to the re-recording instruction.

[0163] In some embodiments, when the recording is completed, the TV starts to play back. The user can pause and select an action to re-record. This process allows the user to better improve his or her work and also enhance the user's patience and perseverance. The user can repeatedly play back his or her work to discover his or her shortcomings and continuously improve his or her work.

[0164] In step 212, the user TV generates a shareable link.

[0165] In step 213, the user TV terminal sends the link to other user mobile terminals.

[0166] In some embodiments, the TV terminal can generate a shareable link or a scannable picture. Users can share their works with other users through social platforms. This process allows users to better share their works and increase their social experience. Users can share their works with other users, watch other users' works, learn about different dance and music styles, and make more friends.

[0167] In step 214, the mobile phones of other users receive click operations on the links by other users.

[0168] In step 215, other users download the TV dancing game applet on their mobile phones.

[0169] In some embodiments, after other users click on the link, they are guided to download the TV-side dance game applet. After logging in, they can experience the previously recorded dance game. This process allows users to continue to experience their own works better, and can also increase the user's interest and enthusiasm for the game. Users can constantly challenge themselves on the TV side, create better works, and compare and communicate with other users.

[0170] In summary, the technical solution provided by the embodiments of the present application has the following beneficial effects:

[0171] 1) Provides a new dancing experience, allowing users to better integrate into dancing;

[0172] 2) It can help users improve their dancing skills. Through the mapping of virtual images, users can observe their own dancing movements, adjust their movements in time, and continuously improve their dancing skills;

[0173] 3) Different scenes and music options are provided. Users can choose the dance method that suits them according to their preferences and create their own dances, breaking the limit on the number of dances in the dance game;

[0174] 4) It provides a good social platform to let more users know about the game and join in the dance;

[0175] 5) It provides users with a good way of entertainment. Users can dance to their heart's content in front of the TV at home, which can not only relax their body and mind, but also improve their dancing skills and enjoy the fun of dancing.

[0176] The following is a description of an exemplary structure of a virtual dance processing device 555 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the virtual dance processing device 555 of the memory 550 may include: a display module 5551 , a play module 5552 , an acquisition module 5553 , a control module 5554 and a generation module 5555 .

[0177] Display module 5551 is used to display a virtual scene, wherein the virtual scene includes a virtual object; playback module 5552 is used to play a target audio in response to a choreography trigger operation; acquisition module 5553 is used to acquire action data of a target object during the process of playing the target audio; control module 5554 is used to control the virtual object to perform dance movements corresponding to the action data; generation module 5555 is used to generate dance content in response to completing a choreography operation, wherein the dance content includes dance movements and target audio.

[0178] In some embodiments, the virtual scene also includes a graphic code, wherein the target audio is an audio selected from a plurality of candidate audios sent to the server by the second electronic device by scanning the graphic code.

[0179] In some embodiments, the virtual dance processing device 555 also includes a download module 5556, which is used for the playback module 5552 to perform the following processing before playing the target audio in response to the choreography trigger operation: in response to receiving an upload completion instruction sent by the second electronic device, downloading the selected audio from the server as the target audio.

[0180] In some embodiments, after the download module 5556 downloads the selected audio from the server, the display module 5551 is also used to display virtual decorations associated with the selected audio at a set position in the virtual scene, wherein the set position includes one of the following: the position selected by the target object, the position of the virtual image.

[0181] In some embodiments, the display module 5551 is also used to display a scene selection control; in response to a trigger operation on the scene selection control, multiple scene identifiers corresponding to multiple candidate virtual scenes are displayed one by one; in response to a selection operation on multiple scene identifiers, a virtual scene corresponding to the selected scene identifier is displayed; in response to a virtual object creation operation, a virtual object created by the target object is displayed in the virtual scene corresponding to the selected scene identifier.

[0182] In some embodiments, the display module 5551 is also used to perform at least one of the following processing: in response to the playback progress of the target audio reaching a set time point, displaying special effects in the virtual scene; in response to the playback progress of the target audio reaching a key rhythm point, displaying special effects in the virtual scene.

[0183] In some embodiments, before the playback module 5552 responds to a choreography trigger operation and plays the target audio, the display module 5551 is also used to display at least one of the timeline of the target audio and the rhythm graph of the target audio in a virtual scene in response to an editing trigger operation; the virtual dance processing device 555 also includes an adding module 5557, which is used to respond to a playback progress setting operation on the timeline, use the time point corresponding to the set playback progress as the set time point, and add special effects at the set time point; the display module 5551 is also used to display prompt information at key rhythm points in the rhythm graph; the adding module 5557 is also used to add special effects at key rhythm points in response to a trigger operation on the prompt information.

[0184] In some embodiments, the acquisition module 5553 is also used to collect image data of the target object when choreographing dance movements through an image acquisition device; analyze the skeletal motion trajectory of the target object from the image data; determine the three-dimensional skeletal data of the target object when choreographing dance movements based on the skeletal motion trajectory; and the control module 5554 is also used to control the virtual image to synchronously execute the dance movements choreographed by the target object based on the three-dimensional skeletal data.

[0185] In some embodiments, the acquisition module 5553 is also used to collect the target part motion data of the target object when choreographing dance movements through a motion capture device; and determine the three-dimensional bone data of the target object when choreographing dance movements based on the bone motion trajectory and the target part motion data.

[0186] In some embodiments, before the generation module 5555 generates a link to the dance content, the display module 5551 is also used to display the timeline of the target audio and multiple skeletal action graphs corresponding one by one to the multiple time points included in the timeline; the virtual dance processing device 555 also includes an adjustment module 5558, which is used to adjust the dance movements performed by the virtual image at the time points corresponding to the target skeletal action graph in response to the adjustment operation on the target skeletal action graph, wherein the target skeletal action graph is a skeletal action graph selected from the multiple skeletal action graphs.

[0187] In some embodiments, before the adjustment module 5558 responds to the adjustment operation on the target skeletal action graph, the display module 5551 is also used to respond to the selection operation on multiple skeletal action graphs, take the selected skeletal action graph as the target skeletal action graph, and enlarge the target skeletal action graph.

[0188] In some embodiments, before the generation module 5555 generates the dance content, the acquisition module 5553 is also used to obtain multiple skeletal action graphs that correspond one to one to multiple time points included in the time axis of the target audio; the virtual dance processing device 555 also includes a detection module 5559, which is used to call a machine learning model to detect the skeletal actions included in the skeletal action graph for each skeletal action graph, and obtain a detection result, wherein the types of the detection results include standard and non-standard; the display module 5551 is also used to display the skeletal action graph whose detection result is non-standard in the virtual scene.

[0189] In some embodiments, the acquisition module 5553 is further used to obtain motion parameters of the dance movement in response to completing the choreography operation, wherein the motion parameters include at least one of the following: motion amplitude, frequency, and speed; the display module 5551 is further used to determine the score of the target object based on the motion parameters and display the score in the virtual scene.

[0190] In some embodiments, the dance content includes a dance video; the generation module 5555 is also used to generate a link to the dance video; the display module 5551 is also used to display multiple candidate social platforms in response to a sharing trigger operation; and to display the link shared by the target object in the selected social platform in response to a selection operation for multiple candidate social platforms.

[0191] In some embodiments, the dance content includes a dance video, and the generation module 5555 is also used to generate a link to the dance video. The display module 5551 is also used to display multiple candidate objects in response to a sharing trigger operation; the virtual dance processing device 555 also includes a sending module 55510, which is used to send a link to a third electronic device associated with the selected object in response to a selection operation for multiple candidate objects.

[0192] In some embodiments, the first electronic device runs a client, the dance content includes a dance video, and the link is used for the third electronic device to perform the following processing: display the link in the human-computer interaction interface of the third electronic device; play the dance video in response to a trigger operation on the link; jump to a download page in response to the completion of the dance video, wherein the download page includes a download control; download the client from the server in response to a trigger operation on the download control.

[0193] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figure 3 , Figure 4 , Figure 5A ,or Figure 5B The present invention should be understood by referring to the description of any one of the accompanying drawings.

[0194] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer readable storage medium. The processor of the computer device reads the computer executable instruction from the computer readable storage medium, and the processor executes the computer executable instruction, so that the computer device executes the virtual dance processing method described in the embodiment of the present application.

[0195] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will execute the virtual dance processing method provided by the present application embodiment, for example, Figure 3 , Figure 4 , Figure 5A ,or Figure 5B The processing method of virtual dance is shown.

[0196] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0197] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0198] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0199] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing virtual dance, characterized in that: The method comprises: Displaying a virtual scene, wherein the virtual scene includes a virtual object; In response to a choreography trigger operation, playing target audio, and During the process of playing the target audio, obtaining the motion data of the target object, and controlling the virtual object to perform dance movements corresponding to the motion data; In response to completing the choreography operation, dance content is generated, wherein the dance content includes the dance movements and the target audio.

2. The method according to claim 1, characterized in that The virtual dance processing method is applied to a first electronic device, the virtual scene further includes a graphic code, and the target audio is an audio selected from a plurality of candidate audios sent to a server by the second electronic device by scanning the graphic code.

3. The method according to claim 2, characterized in that Before playing the target audio in response to the choreography triggering operation, the method further includes: In response to receiving the upload completion instruction sent by the second electronic device, the selected audio is downloaded from the server to serve as the target audio.

4. The method according to claim 3, characterized in that After downloading the selected audio from the server, the method further comprises: A virtual decoration associated with the selected audio is displayed at a set position of the virtual scene, wherein the set position includes one of the following: a position selected by the target object, and a position where the virtual image is located.

5. The method according to claim 1, characterized in that The display of the virtual scene comprises: Display scene selection controls; In response to a trigger operation on the scene selection control, displaying a plurality of scene identifiers corresponding to a plurality of candidate virtual scenes one by one; In response to a selection operation on the plurality of scene identifiers, displaying a virtual scene corresponding to the selected scene identifier; In response to the virtual object creation operation, the virtual object created by the target object is displayed in the virtual scene corresponding to the selected scene identifier.

6. The method according to claim 1, characterized in that The method further comprises: Perform at least one of the following: In response to the playback progress of the target audio reaching a set time point, displaying a special effect in the virtual scene; In response to the playback progress of the target audio reaching a key rhythm point, a special effect is displayed in the virtual scene.

7. The method according to claim 6, characterized in that Before playing the target audio in response to the choreography triggering operation, the method further includes: In response to an edit trigger operation, displaying at least one of a timeline of the target audio and a rhythm graph of the target audio in the virtual scene; In response to a playback progress setting operation on the time axis, a time point corresponding to the set playback progress is used as a set time point, and a special effect is added at the set time point; Prompt information is displayed at key rhythm points in the rhythm graph, and special effects are added at the key rhythm points in response to a trigger operation on the prompt information.

8. The method according to claim 1, characterized in that The step of obtaining the action data of the target object includes: Collecting image data of the target object when choreographing dance movements through an image acquisition device; Analyze the skeletal motion trajectory of the target object from the image data; Determining three-dimensional skeleton data of the target object when choreographing the dance action according to the skeleton motion trajectory as action data of the target object; The controlling the virtual object to perform a dance action corresponding to the action data includes: The virtual object is controlled to synchronously execute the dance movements choreographed by the target object according to the three-dimensional skeleton data.

9. The method according to claim 8, characterized in that The step of determining the three-dimensional skeleton data of the target object when choreographing the dance action according to the skeleton movement trajectory includes: Collecting target part motion data of the target object when choreographing the dance motion by a motion capture device; According to the skeleton motion trajectory and the target part motion data, the three-dimensional skeleton data of the target object when choreographing the dance movement is determined.

10. The method according to claim 1, characterized in that Before generating the dance content, the method further includes: Displaying a timeline of the target audio and a plurality of skeletal action graphs corresponding one-to-one to a plurality of time points included in the timeline; In response to an adjustment operation on a target skeletal action graph, the dance action performed by the virtual object at a time point corresponding to the target skeletal action graph is adjusted, wherein the target skeletal action graph is a skeletal action graph selected from the multiple skeletal action graphs.

11. The method according to claim 10, characterized in that Before the step of responding to the adjustment operation on the target skeletal action graph, the method further comprises: In response to a selection operation on the plurality of skeletal action graphs, the selected skeletal action graph is used as the target skeletal action graph, and the target skeletal action graph is enlarged.

12. The method according to claim 1, characterized in that Before generating the dance content, the method further includes: Acquire a plurality of skeletal action graphs corresponding one-to-one to a plurality of time points included in the time axis of the target audio; For each of the skeletal action graphs, calling a machine learning model to detect the skeletal actions included in the skeletal action graph to obtain a detection result, wherein the types of the detection result include standard and non-standard; The detection result is displayed in the virtual scene as the non-standard skeleton action diagram.

13. The method according to any one of claims 1 to 12, characterized in that: In response to completing the choreography operation, the method further includes: Acquire the action parameters of the dance action, wherein the action parameters include at least one of the following: action amplitude, frequency, and speed; Based on the action parameters, a score of the target object is determined, and the score is displayed in the virtual scene.

14. The method according to any one of claims 1 to 12, characterized in that: The dance content includes a dance video. After generating the dance content, the method further includes: generating a link to the dance video; In response to a sharing trigger operation, displaying a plurality of candidate social platforms; In response to a selection operation on the plurality of candidate social platforms, the link shared by the target object is displayed in the selected social platform.

15. The method according to any one of claims 1 to 12, characterized in that: The dance content includes a dance video. After generating the dance content, the method further includes: generating a link to the dance video; In response to a sharing trigger operation, displaying a plurality of candidate objects; In response to a selection operation on the plurality of candidate objects, the link is sent to a third electronic device associated with the selected object.

16. The method according to claim 15, characterized in that The virtual dance processing method is applied to a first electronic device, the first electronic device runs a client, and the link is used for the third electronic device to perform the following processing: Displaying the link in a human-computer interaction interface of the third electronic device; In response to a trigger operation on the link, playing the dance video; In response to the dance video being played, jumping to a download page, wherein the download page includes a download control; In response to a triggering operation on the download control, the client is downloaded from the server.

17. A virtual dance processing device, characterized in that: The device comprises: A display module, used for displaying a virtual scene, wherein the virtual scene includes a virtual object; A playing module, used for playing target audio in response to a choreography trigger operation; An acquisition module, used for acquiring the motion data of the target object during the process of playing the target audio; A control module, used for controlling the virtual object to perform dance movements corresponding to the movement data; A generating module is used for generating dance content in response to completing the choreography operation, wherein the dance content includes the dance movements and the target audio.

18. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the virtual dance processing method according to any one of claims 1 to 16 when executing the executable instructions stored in the memory.

19. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the virtual dance processing method according to any one of claims 1 to 16 is implemented.

20. A computer program product comprising a computer program or computer executable instructions, characterized in that When the computer program or computer executable instructions are executed by a processor, the virtual dance processing method according to any one of claims 1 to 16 is implemented.

Images