Dance query and deduction system and method based on virtual avatar

Abstract

The invention discloses a dance query and deduction system and method based on a virtual cloned body, and the method comprises the steps: continuously capturing a dance motion of a user through a motion capturing technology, carrying out the vectorization of the dance motion into first dance motion information, and generating a corresponding virtual cloned body; inputting the cue word and the first dance action information into a pre-trained multi-mode generative artificial intelligence model to output second dance action information with consistent features, generating a corresponding deduced virtual branch according to the second dance action information, and prompting a user to perform the same action as the deduced virtual branch to obtain a deduced virtual branch. And then analyzing the similarity of the second dance movement information and the third dance movement information, and displaying and deducing the virtual clones according to the high-low sequence of the similarity to serve as a reference basis for dance deduction, so that the technical effect of improving the dance editing efficiency by using the virtual clones is achieved.

Description

Technical Field

[0001] The present invention relates to a dance query and deduction system and method thereof, in particular to a dance query and deduction system and method thereof based on a virtual avatar. Background Art

[0002] In recent years, with the popularity and rapid development of virtual avatars, various applications of virtual avatars in games or social media have sprung up like mushrooms after rain. However, how to make the use of virtual avatars more diverse and extensive has always been one of the problems that various manufacturers are eager to solve.

[0003] Generally speaking, traditional virtual avatars are usually just simple image displays or controlled game characters. However, although this method can provide basic interactivity, it often lacks depth and has limitations, such as being only used for image display on social media or being controlled as a game character, so the usability of the virtual avatar is poor.

[0004] In view of this, some manufacturers have proposed combining artificial intelligence technology to improve the appearance and flexibility of the virtual avatar, so that it can become a movie character, or combine with music to become a virtual singer. However, this method still makes the virtual avatar performatively, and does not help to improve or enhance the efficiency of the user. For example, in choreography, this method can only make the image of the virtual avatar more specific and the movements more flexible, but it cannot improve the efficiency of choreography. Furthermore, although the virtual avatar is similar to the human body in appearance, there are still considerable differences in the skeleton structure and joint movement, so the movements that the virtual avatar can make may not be made by the human body, resulting in the virtual avatar's dance movements may be beyond reality or have a strong sense of disobedience. Therefore, even if the virtual avatar is applied to choreography, it still cannot effectively solve the problem of poor usability of the virtual avatar.

[0005] In summary, it can be seen that the existing technology has long had the problem of poor usability of virtual avatar applications in choreography, so it is necessary to propose improved technical means to solve this problem. Summary of the invention

[0006] The invention discloses a dance query and deduction system based on virtual avatar and a method thereof.

[0007] First, the present invention discloses a dance query and deduction system based on a virtual avatar, the system comprising: a motion capture device and a computer host. The motion capture device is used to continuously capture dance movements through multiple sensors and motion capture technology, wherein, at the beginning, vectorized first dance movement information is generated according to the dance movement, and after receiving a trigger signal, vectorized third dance movement information is generated according to the dance movement. Then, in the part of the computer host, it is connected to the motion capture device, and the computer host comprises: a non-transitory computer-readable storage medium and a hardware processor. The non-transitory computer-readable storage medium is used to store a plurality of computer-readable instructions and a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is a plurality of vectorized second dance movement information and at least one of the corresponding images and videos; the hardware processor is electrically connected to the non-transitory computer-readable storage medium to execute the plurality of computer-readable instructions, so that the hardware processor is configured to: initially, receive the first dance movement information from the motion capture device to generate and display a virtual avatar with dance movements through 3D modeling technology; receive a prompt word, and input the prompt word and the first dance movement information into the multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs the first dance movement information; The method comprises the steps of: generating second dance movement information that matches the characteristics of the first dance movement information, and outputting second dance movement information that can be connected before and after the first dance movement information; generating a plurality of deduced virtual avatars with different dance movements through 3D modeling technology for display together with the virtual avatars; prompting to select one of the deduced virtual avatars to perform the same movement, and when the deduced virtual avatar is selected, transmitting a trigger signal to the motion capture device so that it continues to capture the dance movement; and after determining that the movement is completed, receiving third dance movement information from the motion capture device, analyzing the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar, and then sorting and displaying the deduced virtual avatar according to the similarity as a reference for dance deduction.

[0008] In addition, the present invention also discloses a dance query and deduction method based on a virtual avatar, the steps of which include: a motion capture device continuously captures dance movements through multiple sensors and motion capture technology, initially, the motion capture device generates vectorized first dance movement information according to the dance movement, and after receiving a trigger signal, the motion capture device generates vectorized third dance movement information according to the dance movement; a computer host is connected to the motion capture device and stores a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is at least one of the vectorized second dance movement information and its corresponding images and videos; initially, the computer host receives the first dance movement information from the motion capture device to generate a virtual avatar with dance movements through 3D modeling technology and displays it; the computer host receives a prompt word, and inputs the prompt word together with the first dance movement information. The invention relates to a method for generating a plurality of deduced virtual avatars having different dance movements according to the output second dance movement information and inputting the second dance movement information into a multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs second dance movement information that is consistent with the characteristics of the first dance movement information, and outputs second dance movement information that can be connected before and after the first dance movement information; the computer host generates a plurality of deduced virtual avatars having different dance movements through 3D modeling technology for display together with the virtual avatars; the computer host prompts to select one of the deduced virtual avatars to perform the same action, and when the deduced virtual avatar is selected, a trigger signal is transmitted to the motion capture device so that it continuously captures the dance movement; after determining that the action is completed, the computer host receives third dance movement information from the motion capture device, analyzes the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar, and then sorts and displays the deduced virtual avatar according to the similarity as a reference for dance deduction.

[0009] The system and method disclosed in the present invention are as described above. The difference from the prior art is that the present invention continuously captures the user's dance movements through motion capture technology to vectorize them into first dance movement information, and generates a corresponding virtual avatar. Then, the prompt word and the first dance movement information are input into a pre-trained multimodal generative artificial intelligence model to output second dance movement information with matching features, and generate a corresponding deduced virtual avatar based on it. The user is then prompted to perform the same movements as the deduced virtual avatar, thereby forming third dance movement information. The similarity between the second and third dance movement information is then analyzed, and the deduced virtual avatar is displayed in order of similarity as a reference for dance deduction.

[0010] Through the above-mentioned technical means, the present invention can achieve the technical effect of improving choreography efficiency by utilizing virtual avatars. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1It is a system block diagram of the dance query and deduction system based on virtual avatar of the present invention.

[0012] FIG. 2A to FIG. 2C The present invention is a method flow chart of a dance query and deduction method based on a virtual avatar.

[0013] Figure 3 It is a schematic diagram of using the present invention to query and deduce dance movements.

[0014] Figure 4 A schematic diagram of applying the present invention to screen and deduce a virtual avatar and display a demonstration virtual avatar.

[0015] The following are the descriptions of the reference numerals:

[0016] 110: Motion capture device

[0017] 120: Computer host

[0018] 121: Non-transitory computer readable storage medium

[0019] 122:Hardware Processor

[0020] 310: Motion capture device

[0021] 311a~311n: Sensor

[0022] 320: User

[0023] 330: Virtual Avatar

[0024] 341,342: Deducing Virtual Avatars

[0025] 400: Setting window

[0026] 410: Set block

[0027] 421,422: Deducing Virtual Avatars

[0028] 430: first display area

[0029] 431: Demonstration of Virtual Avatar

[0030] 440: Second display area

[0031] Step 210: The motion capture device continuously captures dance movements through multiple sensors and motion capture technology. Initially, the motion capture device generates vectorized first dance movement information according to the dance movement, and after receiving a trigger signal, the motion capture device generates vectorized third dance movement information according to the dance movement.

[0032] Step 220: The computer host is connected to the motion capture device and stores the pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is at least one of the vectorized plurality of second dance motion information and the corresponding images and videos thereof.

[0033] Step 230: Initially, the computer host receives the first dance movement information from the motion capture device to generate a virtual avatar with the dance movement through 3D modeling technology and display it.

[0034] Step 240: The computer host receives the prompt word, and inputs the prompt word and the first dance movement information into the multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs the second dance movement information that matches the characteristics of the first dance movement information, and outputs the second dance movement information that can be connected before and after the first dance movement information.

[0035] Step 250: The computer host generates a plurality of deduced virtual avatars with different dance movements using 3D modeling technology based on the output second dance movement information to be displayed together with the virtual avatar.

[0036] Step 260: The computer host prompts to select one of the deduced virtual avatars to perform the same action. When the deduced virtual avatar is selected, the trigger signal is transmitted to the motion capture device to enable it to continuously capture the dance action.

[0037] Step 270: After determining that the action is completed, the computer host receives the third dance action information from the motion capture device, and analyzes the similarity between the third dance action information and the second dance action information corresponding to the selected virtual avatar, and then displays the virtual avatar in a sorted manner according to the similarity as a reference for dance deduction. DETAILED DESCRIPTION

[0038] The following will describe the implementation modes of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0039] First, please refer to Figure 1 , Figure 1The system block diagram of the dance query and deduction system based on virtual avatar of the present invention includes: a motion capture device 110 and a computer host 120. The motion capture device 110 is used to continuously capture dance movements through multiple sensors and motion capture technology, wherein, at the beginning, vectorized first dance movement information will be generated according to the dance movement, and after receiving the trigger signal, vectorized third dance movement information will be generated according to the dance movement. In actual implementation, the sensor can be worn on the user's joints, limbs, trunk, head, etc., so as to accurately capture the user's movements in three-dimensional space, and the generated dance movement information is transmitted to the computer host 120 in real time for storage and processing. In addition, the sensor can include multiple camera elements and multiple accelerometers, which are used to detect the user's posture and movement through the camera element, and detect the movement speed and acceleration through the accelerometer, and the detection results are vectorized as the first dance movement information and the third dance movement information.

[0040] Next, in the part of the computer host 120, which is connected to the motion capture device 110, the computer host 120 includes: a non-transitory computer-readable storage medium 121 and a hardware processor 122. Among them, the non-transitory computer-readable storage medium 121 is used to store a plurality of computer-readable instructions, and to store a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is a vectorized plurality of second dance movement information and at least one of its corresponding images and videos. In actual implementation, the multimodal generative artificial intelligence model is a machine learning model that can process information of different forms (i.e., multimodal) such as pictures, videos, and texts, and is used to understand and generate cross-modal content. In addition, the non-transitory computer-readable storage medium 121 may include: a hard disk, an optical disk, a flash memory, or the like. In addition, the computer readable instructions are executed by the computer host 120, and the computer readable instructions may be assembly language instructions, instruction set architecture instructions, machine instructions, machine-related instructions, microinstructions, firmware instructions, or source code or object code written in any combination of one or more programming languages, wherein the programming languages ​​include object-oriented programming languages, such as Common Lisp, Python, C++, Objective-C, Smalltalk, Delphi, Java, Swift, C#, Perl, Ruby and PHP, and conventional procedural programming languages, such as C language or similar programming languages. In actual implementation, the second dance movement information and the aforementioned first dance movement information and third dance movement information may all be stored in a structured file format (such as JSON, XML, etc.).

[0041] The hardware processor 122 is electrically connected to the non-transitory computer-readable storage medium 121 to execute the plurality of computer-readable instructions, so that the hardware processor 122 is configured to: initially, receive the first dance movement information from the motion capture device 110 to generate and display a virtual avatar having dance movements through 3D modeling technology; receive a prompt word, and input the prompt word and the first dance movement information into the multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs the second dance movement information that is consistent with the characteristics of the first dance movement information, and outputs the second dance movement information that can be connected before and after the first dance movement information. In fact, different dance movements can be judged according to music rhythm, movement consistency, time synchronization, speed consistency, etc. Whether it can be connected, for example, the speed difference and continuity can be judged according to the value of the accelerometer, and if the difference exceeds the characteristic value, it is considered that it cannot be connected; the output second dance movement information is respectively generated through 3D modeling technology to generate multiple deduced virtual avatars with different dance movements to be displayed together with the virtual avatar; prompting to select one of the deduced virtual avatars to perform the same action, when the deduced virtual avatar is selected, sending a trigger signal to the motion capture device 110 to enable it to continuously capture the dance movement; and after determining that the action is completed, receiving the third dance movement information from the motion capture device 110, and analyzing the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar, and then sorting and displaying the deduced virtual avatar according to the similarity as a reference for dance deduction. In actual implementation, when the similarity cannot meet the preset threshold value, the computer host 120 may deem the dance action difficult to be realized, so it deletes the selected deduced virtual avatar and its corresponding second dance action information, and prompts the user to reselect one of the deduced virtual avatars to perform the same action. The computer host 120 will re-judge whether the similarity between the third dance action information and the second dance action information corresponding to the selected deduced virtual avatar meets the threshold value until all deduced virtual avatars have been selected by the user. In addition, the computer host 120 can make the multimodal generative artificial intelligence model output the second dance action information that can be connected before and after the first dance action information through at least one of multimodal fusion, generative adversarial network, feature extraction and matching, and time series analysis. In addition, the computer host 120 can also connect the second dance action information of the deduced virtual avatar with the first dance action information respectively, and generate a corresponding demonstration virtual avatar for display through 3D modeling technology, and display at least one of the image and video corresponding to the second dance action information. The specific connection method can be achieved through a multimodal generative artificial intelligence model to generate coherent subsequent movements based on the first dance movement information and the second dance movement information, and even dynamically adjust the dance movements according to the rhythm of the music.

[0042] It should be particularly noted that in actual implementation, the present invention can be partially or completely implemented based on hardware. For example, the hardware processor 122 in the system can be implemented by an integrated circuit chip, a system on chip (System on Chip, SoC), a complex programmable logic device (Complex Programmable Logic Device, CPLD), and a field programmable gate array (Field Programmable Gate Array, FPGA). The non-transitory computer-readable storage medium 121 described in the present invention carries computer-readable instructions (or computer program instructions) for enabling the hardware processor 122 to implement various aspects of the present invention. The non-transitory computer-readable storage medium 121 can be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the non-transitory computer-readable storage medium 121 include: a hard disk, a random access memory, a read-only memory, a flash memory, an optical disk, a floppy disk, and any suitable combination of the above. The non-transient computer-readable storage medium 121 used herein is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical signal through a fiber optic cable), or an electrical signal transmitted through a wire. In addition, the computer-readable instructions described herein can be downloaded from the non-transient computer-readable storage medium 121 to each computing / processing device, or downloaded to an external computer device or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, hubs, and / or gateways. The network card or network interface in each computing / processing device receives a computer-readable instruction from the network and forwards the computer-readable instruction for storage in the non-transient computer-readable storage medium 121 in each computing / processing device.

[0043] See also FIG. 2A to FIG. 2C , FIG. 2A to FIG. 2CThe present invention is a method flow chart of a dance query and deduction method based on a virtual avatar, wherein the steps include: a motion capture device 110 continuously captures dance movements through multiple sensors and motion capture technology, and initially, the motion capture device 110 generates vectorized first dance movement information according to the dance movement, and after receiving a trigger signal, the motion capture device 110 generates vectorized third dance movement information according to the dance movement (step 210); a computer host 120 is connected to the motion capture device 110, and stores a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is at least one of the vectorized second dance movement information and its corresponding images and videos (step 220); initially, the computer host 120 receives the first dance movement information from the motion capture device 110 to generate a virtual avatar with dance movements through 3D modeling technology and display it (step 230); the computer host 120 receives a prompt word, and inputs the prompt word together with the first dance movement information. The computer host 120 generates a plurality of deduced virtual avatars with different dance movements using 3D modeling technology for the output second dance movement information to be displayed together with the virtual avatar (step 250); the computer host 120 prompts the user to select one of the deduced virtual avatars to perform the same action, and when the deduced virtual avatar is selected, transmits a trigger signal to the motion capture device 110 to continuously capture the dance movement (step 260); and after determining that the action is completed, the computer host 120 receives the third dance movement information from the motion capture device 110, analyzes the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar, and then displays the deduced virtual avatar in a sorted manner according to the similarity as a reference for dance deduction (step 270). Through the above steps, the user's dance movements can be continuously captured through motion capture technology to be vectorized into first dance movement information, and a corresponding virtual avatar is generated. Then, the prompt word and the first dance movement information are input into a pre-trained multimodal generative artificial intelligence model to output second dance movement information with matching features, and a corresponding deduced virtual avatar is generated accordingly. The user is then prompted to perform the same movements as the deduced virtual avatar to form third dance movement information. Then, the similarity between the second and third dance movement information is analyzed, and the deduced virtual avatar is displayed in order of similarity as a reference for dance deduction.

[0044] The following match Figure 3 and Figure 4 The following description is given by way of example: Figure 3 Indicated, Figure 3 Schematic diagram of the application of the present invention to query and deduce dance movements. Assuming that the user 320 lacks inspiration when creating a dance, he can wear sensors (311a-311n) and use the motion capture device 310 and motion capture technology to capture a dance movement of the user 320. At this time, the motion capture device 310 will vectorize this dance movement into first dance movement information and transmit it to the computer host 120 for storage and processing. Then, the computer host 120 generates the following based on the received first dance movement information through 3D modeling technology: Figure 3 A virtual avatar 330 with corresponding dance movements is shown. Afterwards, the user can input a prompt word on the computer host 120, and the prompt word can include a specified music rhythm, dance style, etc. The computer host 120 will input the prompt word and the first dance movement information input by the user 320 into the pre-trained multimodal generative artificial intelligence model, so that it outputs the second dance movement information that matches the characteristics of the first dance movement information, for example, consistent with the dance movement of the user 320, and outputs the second dance movement information that can be connected before and after the first dance movement information, for example: other dance movements that can be connected before and after the dance movement of the user 320. It should be particularly noted that since the training data of the multimodal generative artificial intelligence model is a plurality of vectorized second dance movement information and at least one of their corresponding images and videos, after the model training is completed, the second dance movement information that matches the characteristics can be identified based on the prompt word and the first dance movement information, and the second dance movement information that can be connected with the first dance movement information can be identified.

[0045] Next, the computer host 120 will use the output second dance action information to generate multiple virtual avatars (341, 342) with corresponding dance actions through 3D modeling technology, so as to Figure 3As shown, it is displayed together with the virtual avatar 330. At this time, the user 320 can browse the virtual avatar 330 and the deduced virtual avatar (341, 342) to know the presentation method of the dance movement. Then, the computer host 120 will prompt the user 320 to select one of these deduced virtual avatars (341, 342) to perform the same action, the purpose is to test whether this action can be performed by a person, and even to determine the difficulty of the action. For example, if the same action cannot be performed, it means that this action cannot be performed by a person, or the difficulty is very high. In actual implementation, when the user 320 selects one of the deduced virtual avatars (341, 342), the computer host 120 will send a trigger signal to the motion capture device 110 to enable it to continuously capture the dance movement of the user 320 to generate the corresponding third dance movement information, and so on until all the deduced virtual avatars (341, 342) have been selected. At this point, the corresponding third dance movement information is generated for each deduced virtual avatar (341, 342). Then, the computer host 120 analyzes the similarity between each third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar, and then displays the deduced virtual avatars (341, 342) in order according to the similarity as a reference for dance deduction. For example, if the similarity is low, it means that the movement difficulty is high, and vice versa, if the similarity is high, it means that the movement difficulty is low. Therefore, according to the selection of the user 320, the deduced virtual avatars with high movement difficulty can be displayed first, or the deduced virtual avatars with low movement difficulty can be displayed first.

[0046] like Figure 4 Indicated, Figure 4 Schematic diagram of applying the present invention to screen and display a virtual avatar. In actual implementation, the user can also set a threshold value of similarity in the setting block 410 through the setting window 400, so that the computer host 120 can delete the selected virtual avatar and its corresponding second dance action information when the similarity cannot meet the preset threshold value. For example, assuming that two virtual avatars (421, 422) are generated, among which the virtual avatar 421 performs actions that cannot be performed by humans, then no matter how the user follows the instructions, the similarity between the third dance action information and the second dance action information will definitely be lower than the threshold value. At this time, the user can delete the selected virtual avatar and its corresponding second dance action information. Figure 4The cross is shown on the deduced virtual avatar 421, which means that the deduced virtual avatar 421 and its corresponding second dance action information are deleted. Then, the computer host 120 can prompt the user to reselect the deduced virtual avatar 422 to perform the same action, and re-judge whether the similarity between the third dance action information and the second dance action information corresponding to the selected deduced virtual avatar 422 meets the threshold value, until all deduced virtual avatars (421, 422) have been selected. In this way, the deleted deduced virtual avatar 421 and its corresponding second dance action information are the dance actions that the user, or even humans, cannot achieve, and the retained ones are feasible and helpful references for the user to create dance actions.

[0047] In addition, in actual implementation, the computer host 120 can also connect the second dance movement information of the deduced virtual avatar with the first dance movement information, and generate a demonstration virtual avatar 431 through 3D modeling technology to display the connection result in the first display area 430, and display at least one of the image and video corresponding to the second dance movement information in the second display area 440. In this way, the user can more intuitively know the presentation effect of the dance movements before and after the connection. It should be supplemented that the virtual avatar, the deduced virtual avatar and the demonstration virtual avatar described in the present invention can all be presented through virtual reality (VR), augmented reality (AR), mixed reality (MR) or similar methods, through a head-mounted device, a naked-eye 3D display or the like.

[0048] To sum up, it can be seen that the difference between the present invention and the prior art lies in that the user's dance movements are continuously captured through motion capture technology to be vectorized into first dance movement information, and a corresponding virtual avatar is generated. Then, the prompt word and the first dance movement information are input into a pre-trained multimodal generative artificial intelligence model to output second dance movement information with matching features, and a corresponding deduced virtual avatar is generated accordingly. The user is then prompted to perform the same movements as the deduced virtual avatar, thereby forming third dance movement information. Then, the similarity between the second and third dance movement information is analyzed, and the deduced virtual avatar is displayed in order of similarity as a reference for dance deduction. This technical means can solve the problems existing in the prior art, thereby achieving the technical effect of using virtual avatars to improve choreography efficiency.

[0049] Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any ordinary technician in the field can make some modifications and changes without departing from the spirit and scope of the present invention. Therefore, the scope of patent protection of the present invention shall be based on the scope defined by the claims attached to this specification.

Claims

1. A dance query and deduction system based on virtual avatar, characterized in that: The dance query and deduction system comprises: A motion capture device for continuously capturing dance movements through a plurality of sensors and motion capture technology, wherein, initially, vectorized first dance movement information is generated according to the dance movement, and after receiving a trigger signal, vectorized third dance movement information is generated according to the dance movement; and A computer host, used to connect to the motion capture device, the computer host comprising: A non-transitory computer-readable storage medium for storing a plurality of computer-readable instructions and a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is at least one of a plurality of vectorized second dance movement information and corresponding images and videos thereof; and A hardware processor is electrically connected to the non-transitory computer-readable storage medium and is used to execute the plurality of computer-readable instructions, so that the hardware processor is configured to: Initially, receiving the first dance movement information from the motion capture device to generate a virtual avatar having the dance movement through 3D modeling technology and displaying the virtual avatar; receiving a prompt word, and inputting the prompt word and the first dance movement information into the multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs the second dance movement information that matches the features of the first dance movement information, and outputs the second dance movement information that can be connected before and after the first dance movement information; Using the output second dance movement information through 3D modeling technology to generate a plurality of deduced virtual avatars with different dance movements to display together with the virtual avatar; prompting to select one of the deduced virtual avatars to perform the same action, and when the deduced virtual avatar is selected, transmitting the trigger signal to the motion capture device to enable it to continuously capture the dance action; and After the action is determined to be completed, the third dance action information is received from the motion capture device, and the similarity between the third dance action information and the second dance action information corresponding to the selected deduced virtual avatar is analyzed, and then the deduced virtual avatar is displayed in order according to the similarity as a reference for dance deduction.

2. The dance query and deduction system based on virtual avatar according to claim 1, characterized in that: When the similarity fails to meet the threshold value, the hardware processor deletes the selected deduced virtual avatar and its corresponding second dance movement information, prompts to reselect one of the deduced virtual avatars to perform the same action, and re-judges whether the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar meets the threshold value, until all the deduced virtual avatars have been selected.

3. The dance query and deduction system based on virtual avatar according to claim 1, characterized in that: The computer host uses at least one of multimodal fusion, generative adversarial network, feature extraction and matching, and time series analysis to enable the multimodal generative artificial intelligence model to output the second dance movement information that can be connected before and after the first dance movement information.

4. The dance query and deduction system based on virtual avatar according to claim 1, characterized in that: The sensor includes a plurality of camera elements and a plurality of accelerometers, and is used to detect postures and movements through the camera elements, and to detect movement speed and acceleration through the accelerometers, and to quantize the detection results as the first dance movement information and the third dance movement information.

5. The dance query and deduction system based on virtual avatar according to claim 1, characterized in that: The computer host connects the second dance movement information of the deduced virtual avatar with the first dance movement information respectively, and generates at least one corresponding demonstration virtual avatar for display through 3D modeling technology, and displays at least one of the image and video corresponding to the second dance movement information.

6. A dance query and deduction method based on virtual avatar, characterized in that: The steps include: The motion capture device continuously captures dance movements through multiple sensors and motion capture technology. Initially, the motion capture device generates vectorized first dance movement information according to the dance movement, and after receiving a trigger signal, the motion capture device generates vectorized third dance movement information according to the dance movement. The computer host is connected to the motion capture device and stores a pre-trained multimodal generative artificial intelligence model, wherein the training data of the multimodal generative artificial intelligence model is at least one of a plurality of vectorized second dance motion information and corresponding images and videos thereof; Initially, the computer host receives the first dance movement information from the motion capture device to generate a virtual avatar with the dance movement through 3D modeling technology and displays it; The computer host receives a prompt word, and inputs the prompt word and the first dance movement information into the multimodal generative artificial intelligence model, so that the multimodal generative artificial intelligence model outputs the second dance movement information that matches the features of the first dance movement information, and outputs the second dance movement information that can be connected before and after the first dance movement information; The computer host generates a plurality of deduced virtual avatars with different dance movements using 3D modeling technology for displaying the output second dance movement information together with the virtual avatar; The computer host prompts to select one of the deduced virtual clones to perform the same action, and when the deduced virtual clone is selected, transmits the trigger signal to the motion capture device to enable it to continuously capture the dance action; and After determining that the action is completed, the computer host receives the third dance action information from the motion capture device, analyzes the similarity between the third dance action information and the second dance action information corresponding to the selected deduced virtual avatar, and then displays the deduced virtual avatar in order of similarity as a reference for dance deduction.

7. The dance query and deduction method based on virtual avatar according to claim 6, characterized in that: When the similarity fails to meet the threshold value, the computer host deletes the selected deduced virtual avatar and its corresponding second dance movement information, prompts the user to reselect one of the deduced virtual avatars to perform the same action, and re-judges whether the similarity between the third dance movement information and the second dance movement information corresponding to the selected deduced virtual avatar meets the threshold value, until all the deduced virtual avatars have been selected.

8. The dance query and deduction method based on virtual avatar according to claim 6, characterized in that: The computer host uses at least one of multimodal fusion, generative adversarial network, feature extraction and matching, and time series analysis to enable the multimodal generative artificial intelligence model to output the second dance movement information that can be connected before and after the first dance movement information.

9. The dance query and deduction method based on virtual avatar according to claim 6, characterized in that: The sensor includes a plurality of camera elements and a plurality of accelerometers, and is used to detect postures and movements through the camera elements, and to detect movement speed and acceleration through the accelerometers, and to quantize the detection results as the first dance movement information and the third dance movement information.

10. The dance query and deduction method based on virtual avatar according to claim 6, characterized in that: The computer host connects the second dance movement information of the deduced virtual avatar with the first dance movement information respectively, and generates a demonstration virtual avatar for display through 3D modeling technology, and displays at least one of the image and video corresponding to the second dance movement information.

Images