Live action replication method, system, device and medium based on exoskeleton device

Abstract

The embodiment of the application discloses a live action reproduction method, system, device and medium based on an exoskeleton device. The technical scheme provided by the embodiment of the application receives the angular velocity and angular acceleration of each specified human body key point of an action capture target and the first position transformation information of a specified root node; then generates the second position transformation information of a driving target based on the first position transformation information; thereafter, drives the exoskeleton device to drive the driving target to transform the core position based on the second position transformation information, and drives the exoskeleton device to drive the driving target to rotate the corresponding joint node based on the angular velocity and angular acceleration, so as to reproduce the action of the action capture target. By using the above technical means, the action of the action capture target can be accurately reproduced through the exoskeleton device, so as to enhance the interaction effect between the action capture target and the driving target, improve the authenticity and flexibility of the interaction, and further improve the interaction experience of the user.

Description

Technical Field

[0001] This application relates to the field of Internet live streaming technology, and in particular to a method, system, device and medium for reproducing live streaming actions based on an exoskeleton device. Background Technology

[0002] Currently, with the development of internet technology, the application of live streaming is becoming increasingly widespread. By building a multifunctional live streaming platform that integrates audio, video, desktop sharing, document sharing, and interactive elements, comprehensive communication and interaction via voice, video, and data can be conducted directly online. For example, by capturing teachers' lecture videos in real time and displaying the video stream on students' devices, lectures can be conducted in a live online format, expanding the teaching scenarios and eliminating the limitations of physical location.

[0003] However, in traditional online live streaming scenarios, the host and the audience can only interact through images, voice, or text. The interaction methods are relatively simple, rigid, and lack flexibility, resulting in relatively poor interaction effects. Summary of the Invention

[0004] This application provides a method, system, device, and medium for reproducing live-streaming actions based on an exoskeleton device, which can enhance the interaction between the broadcaster and the viewer and solve the technical problem of the lack of flexibility in the interaction between the broadcaster and the viewer.

[0005] In a first aspect, embodiments of this application provide a method for reproducing live-streamed movements based on an exoskeleton device, applied to the exoskeleton device, comprising:

[0006] Receive motion capture information from the live interactive terminal. The motion capture information includes the angular velocity and angular acceleration of each specified human key point of the motion capture target, as well as the first position transformation information of the specified root node.

[0007] Acquire the first body parameters of the motion capture target and the second body parameters of the current driving target, and generate the second position transformation information of the driving target based on the first body parameters, the second body parameters and the first position transformation information;

[0008] The exoskeleton device is driven by the second position change information to drive the target to change its core position, and the exoskeleton device is driven by the angular velocity and angular acceleration to drive the target to rotate the corresponding joint points, so as to reproduce the motion capture target's motion.

[0009] In a second aspect, embodiments of this application provide a live streaming interactive system, including:

[0010] Motion capture equipment is used to collect motion capture information of motion capture targets;

[0011] The live streaming interaction terminal is used to acquire motion capture information and forward it to the client.

[0012] The client is used to interact with the live streaming interaction terminal, receive motion capture information and forward it to the exoskeleton device;

[0013] An exoskeleton device for performing the live motion reproduction method based on an exoskeleton device as described in the first aspect.

[0014] In a third aspect, embodiments of this application provide an exoskeleton device, comprising:

[0015] Memory and one or more processors;

[0016] The memory is configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the live motion reproduction method based on the exoskeleton device as described in the first aspect.

[0018] In a fourth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform the live motion reproduction method based on an exoskeleton device as described in the first aspect.

[0019] In a fifth aspect, embodiments of this application provide a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the live action reproduction method based on an exoskeleton device as described in the first aspect.

[0020] This application embodiment receives motion capture information from a live interactive terminal. The motion capture information includes the angular velocity and angular acceleration of each specified key point on the motion capture target, as well as the first position transformation information of a specified root node. Then, it acquires the first body parameters of the motion capture target and the second body parameters of the currently driven target. Based on the first body parameters, the second body parameters, and the first position transformation information, it generates the second position transformation information of the driven target. Subsequently, based on the second position transformation information, it drives an exoskeleton device to change the core position of the driven target, and based on the angular velocity and angular acceleration, it drives the exoskeleton device to rotate the driven target at corresponding joint points, thereby reproducing the motion of the motion capture target. Using the above technical means, motion capture information of the motion capture target can be collected, displacement adaptation can be performed between the motion capture target and the driven target, and the motion of the motion capture target can be accurately reproduced through an exoskeleton device. This enhances the interaction effect between the motion capture target and the driven target, improves the realism and flexibility of the interaction, and ultimately enhances the user's interactive experience.

[0021] Furthermore, the embodiments of this application adapt positional transformation information by matching the body parameters of the motion capture target and the driving target, and drive the motion by angular velocity and angular acceleration. This can achieve precise motion driving, avoid large errors in motion reproduction due to differences in body shape, and improve the accuracy of motion reproduction. Attached Figure Description

[0022] Figure 1 This is a flowchart of a live motion reproduction method based on an exoskeleton device provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure of the live interactive system in the embodiments of this application;

[0024] Figure 3 This is a flowchart of the action comparison process in an embodiment of this application;

[0025] Figure 4 This is a flowchart of the action threshold setting process in the embodiments of this application;

[0026] Figure 5 This is a schematic diagram of the structure of a live motion reproduction system based on an exoskeleton device provided in this application embodiment;

[0027] Figure 6 This is a structural schematic diagram of an exoskeleton device provided in this application embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] This application provides a live motion reproduction method based on an exoskeleton device, which aims to collect motion capture information of the motion capture target, perform displacement adaptation between the motion capture target and the driving target, and accurately reproduce the motion of the motion capture target through the exoskeleton device, so as to enhance the interaction between the motion capture target and the driving target.

[0030] In traditional live streaming interactive scenarios, the interaction between the broadcaster and the audience is usually based on basic voice or text. To enhance the interactive experience, emoticons, images, or virtual avatars can also be used. However, these interaction methods are generally virtual, and for some interactive scenarios that require a realistic experience, they cannot provide the corresponding interactive effects. Their interaction methods are relatively simple, rigid, and lack flexibility. Therefore, this application provides a live streaming action reproduction method based on an exoskeleton device to solve the technical problems of insufficient flexibility and realism in the interaction between the broadcaster and the audience.

[0031] Example:

[0032] Figure 1 A flowchart of a live motion reproduction method based on an exoskeleton device provided in this application embodiment is given. The live motion reproduction method based on an exoskeleton device provided in this embodiment can be executed by the exoskeleton device.

[0033] The following description uses an exoskeleton device as the primary component for executing a live-stream motion reproduction method based on the exoskeleton device. (Refer to...) Figure 1 The method for reproducing live-streamed movements based on exoskeleton devices specifically includes:

[0034] S110. Receive motion capture information from the live interactive terminal. The motion capture information includes the angular velocity and angular acceleration of each specified human key point of the motion capture target, as well as the first position transformation information of the specified root node.

[0035] S120: Obtain the first body parameters of the motion capture target and the second body parameters of the current driving target, and generate the second position transformation information of the driving target based on the first body parameters, the second body parameters and the first position transformation information;

[0036] S130. Based on the second position change information, the exoskeleton device is driven to change the core position of the target, and based on the angular velocity and angular acceleration, the exoskeleton device is driven to rotate the target at the corresponding joint points to reproduce the motion capture target's motion.

[0037] This application's embodiments, when using exoskeleton devices for live streaming interaction, employ motion capture combined with the exoskeleton device to reproduce actions, thereby replicating the user's actions on one end of the live stream onto the user on the other end, achieving a realistic and flexible live streaming interaction effect. This enhances the user's live streaming interaction experience and expands live streaming interaction scenarios to meet the live streaming interaction needs in specific situations.

[0038] Specifically, refer to Figure 2This application provides a schematic diagram of the structure of a live interactive system according to an embodiment of the present application. The system includes a motion capture device for collecting motion capture information of a motion capture target; a live interactive terminal for acquiring motion capture information and forwarding it to a client; a client for interacting with the live interactive terminal, receiving motion capture information, and forwarding it to an exoskeleton device; and an exoskeleton device for executing the aforementioned live motion reproduction method based on the exoskeleton device. In a live streaming scenario, the streamer and the audience interact using the live interactive terminal 11 and the client 12, respectively. The live interactive terminal 11 connects to both the client 12 and the motion capture device 13, collecting motion capture information of the user through the motion capture device 13 and defining this user as the motion capture target. Based on the collected motion capture information, it is sent from the live interactive terminal 11 to the client 12, which then forwards it to the exoskeleton device 14. Another user wears the exoskeleton device 14, defining this user as the driving target. Thus, the motion reproduction of the motion capture target can be achieved based on the live motion reproduction method based on the exoskeleton device according to this embodiment of the present application.

[0039] For example, motion capture equipment on the broadcaster's end can collect the broadcaster's motion capture information, which is then sent to the viewer's end. Viewers, wearing exoskeletons, receive this motion capture information and then use the exoskeleton to replicate the broadcaster's movements, thus enabling live interaction between the broadcaster and the viewer. Similarly, motion capture equipment on the viewer's end can collect the viewer's motion capture information and send it to the broadcaster's exoskeleton, allowing the broadcaster to replicate the viewer's movements. With the help of exoskeletons, the target can realistically perceive the motion capture target's posture, thereby enhancing the flexibility and realism of live interaction.

[0040] In certain specific live streaming interactive scenarios, the aforementioned live streaming interactive framework can be used to expand the forms of live streaming and interaction, thereby improving the live streaming effect. For example, exoskeleton devices can be used for online live fitness and sports instruction. By acquiring the instructor's motion capture information, the exoskeleton device can more efficiently and realistically assist learners in replicating the instructor's movements, thereby improving training effectiveness and optimizing the learning interaction experience.

[0041] Specifically, when using motion capture equipment to collect motion capture information, different precision motion capture equipment can be selected according to the precision requirements of limb control in the live interactive scenario. These include, but are not limited to, optical motion capture (infrared, cameras based on computer vision algorithms), inertial motion capture (such as sensors worn on the motion capture target), and hybrid optical and inertial motion capture.

[0042] When motion capture is enabled, the motion capture target demonstrates the target action. The motion capture device then collects the corresponding motion capture information. This embodiment employs inertial motion capture, where the motion capture target wears a motion capture suit equipped with relevant sensors to collect motion capture information. Depending on the required accuracy, different levels of motion capture suits can be selected. The motion capture suit includes, but is not limited to, pressure sensors and gyroscopes installed on joints such as the head, neck, chest, shoulders, elbows, wrists, finger joints, waist, hips, thighs, calves, ankles, soles, and toe joints, as well as the soles of the feet. These joint points are designated key human body points. The motion capture suit uses pressure sensors and gyroscopes installed on corresponding key human body parts (bodysuit) to locate joint positions via pressure sensors and collect angular velocities at corresponding joint positions via gyroscopes. Angular acceleration is obtained from the changes in angular velocity, thus yielding the angular velocity and angular acceleration of each designated key human body point of the motion capture target.

[0043] Furthermore, this embodiment of the application also obtains first position transformation information by detecting the position change of a specified root node. The specified root node is generally located at the navel position of the motion capture target, which represents the core of the motion capture target. By detecting the position change of this core position, the position transformation data of the motion capture target is used for subsequent adaptation to drive the target position transformation.

[0044] It should be noted that the body parameters (such as height and limb length) of the motion capture target and the driving target are likely to be inconsistent. Therefore, when processing motion capture data, the absolute values ​​of the spatial coordinate system are not used as motion capture information. Instead, joint angular velocities and angular accelerations are used as the targets for exoskeleton motion reconstruction to achieve accurate motion reconstruction. Furthermore, this embodiment also adapts the position transformation information of the driving target according to the body parameters of the motion capture target and the driving target. It is understood that because the body parameters of the motion capture target and the driving target are different, when they perform the same action, their core position transformations are different, and the positions of their joint point transformations are also different, while their joint rotation angular velocities and angular accelerations are the same. Based on this, this embodiment achieves accurate motion reproduction by adapting the position transformation information of the driving target and matching the angular velocities and angular accelerations of each joint point.

[0045] In adapting the position change information of the driving target, the ratio between the first body parameter of the motion capture target and the second body parameter of the current driving target is determined. To ensure the adaptation of the position change information, this embodiment defines the ratio of the two body parameters as equal to the ratio of their position change information. For example, the taller the target, the greater the magnitude of its position change. Therefore, by obtaining the heights of the motion capture target and the driving target as body parameters, determining their ratio, and dividing the first position change information by this ratio, the second position change information of the driving target can be obtained. In practical applications, to improve adaptation accuracy, the body parameters can also be normalized values ​​of parameters such as height, weight, and limb length. The corresponding position change information is calculated based on the ratio of these parameter values ​​to comprehensively consider the differences in the parameters of various parts of the two bodies, achieving accurate position change adaptation.

[0046] Furthermore, based on the aforementioned determined second position transformation information and the angular velocities and angular accelerations of various designated key points on the motion capture target, the exoskeleton device can adaptively control the target to change its core position according to the second position transformation information. Similarly, the navel position is typically chosen as the core position of the target. Then, according to the collected angular velocities and angular accelerations of each key point, the exoskeleton device drives the corresponding joints of the target to rotate according to the aforementioned angular velocities and angular accelerations. Thus, the motion reproduction of the motion capture target is completed.

[0047] In practical applications, the aforementioned motion capture information and body parameters can be acquired in real-time and reproduced through the exoskeleton device, or they can be pre-stored and extracted for motion reproduction as needed. For example, in a live-streaming teaching scenario, after students wear the exoskeleton device, they can instantly follow the host's synchronized movements. The exoskeleton will then use the motion capture information received from the host to drive the students' limbs in real-time to reproduce the input movements. Another example is that the exoskeleton device can read locally stored teaching data to obtain motion capture information for a segment of movement at different times, and then input the motion capture information into the exoskeleton device sequentially according to the timeline for motion reproduction.

[0048] For example, based on exoskeleton devices, in assisted live sports teaching scenarios, the time learners spend mastering skills can be shortened and training efficiency improved by reproducing the teaching movements. It is understandable that learning and mastering a sport often requires a significant amount of time to master the basic movements. Even with a coach's guidance, this process is limited by factors such as the coach's language expression, skill level, the learner's understanding, limb coordination, and irrational muscle memory, thus affecting training results and efficiency, making it difficult to quickly understand and master the correct limb movement sequence and techniques taught by the instructor. However, by using exoskeleton devices, taking online table tennis teaching as an example, precise spatial and temporal data of the instructor's limbs can be collected in real time and transmitted to the viewer's exoskeleton device for reproduction. After wearing the exoskeleton, the user moves their limbs to learn various table tennis techniques.

[0049] For example, when learning the power generation technique in table tennis, non-professionals often mistakenly believe that hitting the ball is simply a matter of the extremities, neglecting the crucial lower limb power generation method, based solely on visual observation of athletes' demonstrations. What appears to be a movement relying on the extremities actually involves the entire body's muscle groups—a chain reaction of power generation. For instance, the basic preparatory stance in table tennis involves legs slightly wider than shoulder-width apart, left foot slightly forward of right foot, knees slightly bent, upper body relaxed, chest slightly concave, abdomen tucked in, shoulders and upper arms relaxed and naturally perpendicular to the ground, forearms at a 90° angle parallel to the ground, and weight on the balls of the feet. Before hitting the ball, the power transmission process is as follows: the balls of the feet push off the ground, rotating forward and upward, followed by a forward thrust of the hips, with the upper arm leading the forearm forward, and at the moment of contact, the forearm retracts, and the wrist and fingers tighten the racket to increase speed. Here, the power generation sequence at each stage is progressive, with the speed of the previous limb being transferred to the next, gradually accumulating until it reaches the extremities, achieving a speed far exceeding that of simply using extremity power. Because table tennis rallies are fast-paced, requiring quick reflexes, professional athletes' body movements are very small, appearing to consist only of arm swings. This leads to amateur enthusiasts easily misinterpreting the movements. Even with a coach's guidance, it's difficult to grasp the force transmission mechanism. However, the exoskeleton device of this application, employing the aforementioned live-streamed motion reproduction method based on the exoskeleton device, enables hands-on motion instruction. Precise motion capture information from the instructor is collected and transmitted via network to the exoskeleton devices of each student. After wearing the exoskeleton device, students can, with the assistance of the exoskeleton device, reproduce the instructor's movements in real time using the aforementioned live-streamed interactive method. This assists users in training their limbs, achieving precise and efficient motion learning.

[0050] Optionally, based on the above live streaming interaction framework, different live streaming interaction effects can be achieved based on different operating modes. The operating modes are as follows:

[0051] 1. Idle Mode: This is the default mode. In this mode, the exoskeleton device does not provide driving force, and the wearer moves on their own.

[0052] II. Recording Mode: In this mode, the exoskeleton device does not provide driving force, and the wearer moves on their own. The exoskeleton device will record the wearer's motion data and generate a file after the recording is completed.

[0053] 3. Follow Mode: After entering this mode, once the wearer is ready, they can receive the anchor's motion data. The exoskeleton device will guide the wearer to replicate the anchor's movements. The wearer can enter idle mode at any time.

[0054] IV. Learning Mode: The wearer can select a previously saved motion template. The exoskeleton device will then use the selected motion template to move the body. In this mode, the user can select the motion speed multiplier. Users can choose slow, normal, or faster speeds to experience the learning motion.

[0055] V. Comparison Mode: In this mode, the exoskeleton device does not provide driving force. After the user selects a local motion template, they actively move, and the system records the motion data. Then, by comparing the data with the template motion, the system informs the user of the differences between the motion and the template.

[0056] Based on the different modes mentioned above, action interactions can be achieved under different scenario requirements, thereby improving the user interaction effect.

[0057] Optionally, after driving the exoskeleton device to rotate the target at the corresponding joint points based on angular velocity and angular acceleration, the embodiments of this application further include:

[0058] S1401. Collect target motion information of the driving target based on the exoskeleton device;

[0059] S1402. Compare the target action information with the motion capture information to generate action comparison results, and output the action comparison results to the client and / or live interactive terminal corresponding to the target for display.

[0060] Referring to the comparison mode described above, after the exoskeleton device drives the target to reproduce the motion capture target's movements, the target's motion information is collected and compared with the motion capture information to determine the differences. These differences are then presented to the user. The user can then adjust the exoskeleton device's fit based on these differences to achieve better motion reproduction.

[0061] In one embodiment, after the exoskeleton device collects target motion information of the driven target based on the exoskeleton device, it further includes: driving a pre-built first virtual image to display the driven target's motion on the client and / or live streaming interactive terminal based on the target motion information.

[0062] By locating each joint of the first virtual avatar, the target action information of each joint of the driving target is mapped to each joint of the first virtual avatar. This allows the actions of each joint of the first virtual avatar to be driven, and the virtual avatar can then reproduce and display the driving target's actions on the client and / or live streaming interaction end. This facilitates both parties in the interaction to view the effect of the driving target's action reproduction, further enhancing the user interaction experience.

[0063] In addition, after receiving motion capture information from the live streaming interactive terminal, the process also includes: driving a pre-built second virtual avatar based on the motion capture information to display the motion capture target's actions on the client.

[0064] Similarly, by locating the joints of the second virtual avatar, the motion capture information of each key point of the motion capture target is mapped to the joints of the second virtual avatar. This allows the movement of each joint of the second virtual avatar to be driven, and the motion capture target's movements can be reproduced and displayed on the client side using the virtual avatar. In this way, driving the target can not only assist teaching through exoskeleton devices, but also allow for a deeper understanding of the movement details through the reproduction of the motion capture target's movements on the client side. In scenarios such as sports teaching, this helps to deepen memory and improve the effectiveness of movement learning.

[0065] On the other hand, after the exoskeleton device drives the target to rotate the corresponding joint points based on angular velocity and angular acceleration, it also includes: storing motion capture information and first body parameters in the local memory of the exoskeleton device for motion capture target motion reproduction.

[0066] Referring to the above learning model, motion capture information and initial body parameters are stored in the local memory of the exoskeleton device as motion templates. During motion learning, users can select a locally stored motion template to guide their body movements, thus reproducing the motion, improving learning efficiency, and deepening memory of the learned motion.

[0067] In one embodiment, the exoskeleton device further drives the target to reproduce movements based on a set movement rate or a set multiple of the movement rate of the motion capture target. In sports teaching scenarios, adjusting the movement reproduction rate can meet the needs of different learning progresses and requirements. For example, for beginners, movements can be reproduced at a slower rate to ensure the user masters the complete movement sequence. For more experienced learners, the movement rate can be adaptively increased to improve learning efficiency.

[0068] Optionally, in one instance, the exoskeleton device also has pre-set motion thresholds to ensure the wearer is not injured by excessive movements, thus improving exercise safety. For example... Figure 4 As shown, the action threshold setting process includes:

[0069] S1001. Based on the exoskeleton device, perform physical function tests on the driven target to obtain physical function information, which includes at least one of muscle strength, maximum speed and body flexibility.

[0070] S1002. Set the motion threshold of the driving target based on the body function information, and drive the exoskeleton device based on the motion threshold.

[0071] Understandably, taking sports instruction as an example, the broadcaster, as the demonstrator, may possess physical abilities exceeding those of the average person. Therefore, when an exoskeleton device is first worn, the wearer undergoes physical function testing to obtain information such as muscle strength, maximum speed, and flexibility. Furthermore, the wearer's maximum physical capabilities are used as a movement threshold, which is stored in the exoskeleton device. When a movement exceeds this threshold, the device will only reproduce the movement within the range allowed by the threshold, preventing injury and thus improving sports safety.

[0072] Furthermore, the exoskeleton device of this application embodiment can be used not only to assist in motion training but also to control robots to perform dangerous tasks (the exoskeleton device itself can be considered a robot if it is not worn). After installing a video capture and signal system on the robot, the scene is reconstructed, and the operator can wear a motion capture suit to control the robot in real time to perform dangerous tasks such as firefighting and high-altitude operations. On the other hand, the exoskeleton device can also be used for medical assistance, such as helping stroke or paralyzed patients move or conducting rehabilitation training. This application embodiment does not impose fixed limitations on the specific use cases of the exoskeleton device, and will not elaborate further here.

[0073] The above describes a process where motion capture information is received from the live streaming interaction terminal. This information includes the angular velocity and angular acceleration of each specified key point on the motion capture target, as well as the first position transformation information of the specified root node. The process then acquires the first body parameters of the motion capture target and the second body parameters of the currently driven target. Based on these parameters and the first position transformation information, the second position transformation information of the driven target is generated. Subsequently, based on the second position transformation information, the exoskeleton device is driven to move the driven target to change its core position. Furthermore, based on the angular velocity and angular acceleration, the exoskeleton device is driven to rotate the driven target at corresponding joint points, thus reproducing the motion of the motion capture target. Using this technique, motion capture information of the motion capture target can be collected, displacement adaptation between the motion capture target and the driven target can be performed, and the motion of the motion capture target can be accurately reproduced through the exoskeleton device. This enhances the interactive effect between the motion capture target and the driven target, improves the realism and flexibility of the interaction, and ultimately enhances the user's interactive experience.

[0074] Furthermore, the embodiments of this application adapt positional transformation information by matching the body parameters of the motion capture target and the driving target, and drive the motion by angular velocity and angular acceleration. This can achieve precise motion driving, avoid large errors in motion reproduction due to differences in body shape, and improve the accuracy of motion reproduction.

[0075] Based on the above embodiments, Figure 5 A schematic diagram of a live motion reproduction system based on an exoskeleton device provided in this application. (Reference) Figure 5 The live motion reproduction system based on exoskeleton device provided in this embodiment specifically includes: a receiving module 21, a generating module 22, and a driving module 23.

[0076] The receiving module 21 is configured to receive motion capture information from the live interactive terminal. The motion capture information includes the angular velocity and angular acceleration of each specified human key point of the motion capture target, as well as the first position transformation information of the specified root node.

[0077] The generation module 22 is configured to acquire the first body parameters of the motion capture target and the second body parameters of the current driving target, and generate the second position transformation information of the driving target based on the first body parameters, the second body parameters and the first position transformation information;

[0078] The drive module 23 is configured to drive the exoskeleton device to change the core position of the drive target based on the second position change information, and to drive the exoskeleton device to rotate the drive target at the corresponding joint points based on angular velocity and angular acceleration, so as to reproduce the motion capture target's motion.

[0079] Specifically, after driving the exoskeleton device to rotate the target at the corresponding joint points based on angular velocity and angular acceleration, the method further includes: collecting target motion information of the target based on the exoskeleton device; comparing the target motion information with motion capture information to generate motion comparison results; and outputting the motion comparison results to the client and / or live interactive terminal corresponding to the target for display.

[0080] Specifically, after collecting target motion information of the driven target based on the exoskeleton device, it also includes:

[0081] Based on the target's action information, a pre-built first virtual avatar is displayed on the client and / or live streaming interactive terminal to drive the target's actions.

[0082] Specifically, after receiving motion capture information from the live streaming interactive terminal, the process also includes:

[0083] A pre-built second virtual avatar, driven by motion capture information, displays the motion capture target's actions on the client side.

[0084] Specifically, after the exoskeleton device, driven by angular velocity and angular acceleration, rotates the target at the corresponding joint points, it also includes:

[0085] Motion capture information and initial body parameters are stored in the local memory of the exoskeleton device for the reproduction of the motion capture target's movements.

[0086] Specifically, it also includes: performing physical function tests on the driven target based on the exoskeleton device to obtain physical function information, which includes at least one of muscle strength, maximum speed and body flexibility; setting the motion threshold of the driven target based on the physical function information, and driving the exoskeleton device based on the motion threshold.

[0087] Specifically, it also includes:

[0088] Based on a set motion rate or a set multiple of the motion capture target's motion rate, the exoskeleton device is driven to reproduce the motion of the target.

[0089] The above describes a process where motion capture information is received from the live streaming interaction terminal. This information includes the angular velocity and angular acceleration of each specified key point on the motion capture target, as well as the first position transformation information of the specified root node. The process then acquires the first body parameters of the motion capture target and the second body parameters of the currently driven target. Based on these parameters and the first position transformation information, the second position transformation information of the driven target is generated. Subsequently, based on the second position transformation information, the exoskeleton device is driven to move the driven target to change its core position. Furthermore, based on the angular velocity and angular acceleration, the exoskeleton device is driven to rotate the driven target at corresponding joint points, thus reproducing the motion of the motion capture target. Using this technique, motion capture information of the motion capture target can be collected, displacement adaptation between the motion capture target and the driven target can be performed, and the motion of the motion capture target can be accurately reproduced through the exoskeleton device. This enhances the interactive effect between the motion capture target and the driven target, improves the realism and flexibility of the interaction, and ultimately enhances the user's interactive experience.

[0090] Furthermore, the embodiments of this application adapt positional transformation information by matching the body parameters of the motion capture target and the driving target, and drive the motion by angular velocity and angular acceleration. This can achieve precise motion driving, avoid large errors in motion reproduction due to differences in body shape, and improve the accuracy of motion reproduction.

[0091] The live motion reproduction system based on exoskeleton devices provided in this application embodiment can be configured to execute the live motion reproduction method based on exoskeleton devices provided in the above embodiment, and has corresponding functions and beneficial effects.

[0092] Based on the above practical examples, this application also provides an exoskeleton device, referring to... Figure 6The exoskeleton device includes a processor 31, a memory 32, and a communication module 33. The memory 32, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the live motion reproduction method based on the exoskeleton device described in any embodiment of this application (e.g., the receiving module, generating module, and driving module in the live motion reproduction system based on the exoskeleton device). The communication module 33 is configured to perform data transmission. The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, thereby implementing the aforementioned live motion reproduction method based on the exoskeleton device. The exoskeleton device provided above can be configured to execute the live motion reproduction method based on the exoskeleton device provided in the above embodiments, possessing corresponding functions and beneficial effects.

[0093] Based on the above embodiments, this application also provides a computer-readable storage medium storing computer-executable instructions. These computer-executable instructions, when executed by a computer processor, are configured to perform a live-stream motion reproduction method based on an exoskeleton device. The storage medium can be any type of memory device or storage device. Of course, the computer-readable storage medium provided in this application is not limited to the live-stream motion reproduction method based on an exoskeleton device as described above; it can also execute related operations in any of the live-stream motion reproduction methods based on an exoskeleton device provided in any embodiment of this application.

[0094] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause the processor to execute all or part of the steps of the live motion reproduction method based on the exoskeleton device described in the various embodiments of this application.

Claims

1. A method for reproducing live-streamed movements based on an exoskeleton device, applied to an exoskeleton device, characterized in that, include: Receive motion capture information from the live interactive terminal, the motion capture information including the angular velocity and angular acceleration of each specified human key point of the motion capture target, and the first position transformation information of the specified root node; The first body parameters of the motion capture target and the second body parameters of the current driving target are obtained. The second position transformation information of the driving target is generated based on the first body parameters, the second body parameters and the first position transformation information. The ratio of the first body parameters to the second body parameters is equal to the ratio of the first position transformation information to the second position transformation information. The first body parameters and the second body parameters are parameter values ​​obtained by normalizing height, weight and limb length. Based on the second position change information, the exoskeleton device is driven to move the target to change its core position. Based on the angular velocity and the angular acceleration, the exoskeleton device is driven to rotate the target to the corresponding joint points to reproduce the motion capture target's movement. This includes performing a body function test on the target based on the exoskeleton device to obtain body function information, which includes at least one of muscle strength, maximum speed, and body flexibility. Based on the body function information, a motion threshold for the target is set. Based on the motion threshold, the exoskeleton device is driven to reproduce the target's movement according to the force transmission process when the target exerts force.

2. The live-stream motion reproduction method based on an exoskeleton device according to claim 1, characterized in that, After driving the exoskeleton device to rotate the target at the corresponding joint point based on the angular velocity and the angular acceleration, the method further includes: The exoskeleton device collects target motion information of the driven target. The target motion information is compared with the motion capture information to generate a motion comparison result, and the motion comparison result is output to the client corresponding to the driving target and / or the live interactive terminal for display.

3. The live-stream motion reproduction method based on an exoskeleton device according to claim 2, characterized in that, After acquiring the target motion information of the driven target based on the exoskeleton device, the process further includes: Based on the target action information, a pre-constructed first virtual avatar is driven to display the target's actions on the client and / or the live streaming interactive terminal.

4. The live-stream motion reproduction method based on an exoskeleton device according to claim 2, characterized in that, After receiving motion capture information from the live streaming interactive terminal, it also includes: Based on the motion capture information, a pre-constructed second virtual avatar is driven to display the motion capture target's actions on the client.

5. The live-stream motion reproduction method based on an exoskeleton device according to claim 1, characterized in that, After driving the exoskeleton device to rotate the target at the corresponding joint point based on the angular velocity and the angular acceleration, the method further includes: The motion capture information and the first body parameters are stored in the local memory of the exoskeleton device for the reproduction of the motion capture target's movements.

6. The live-stream motion reproduction method based on an exoskeleton device according to claim 1, characterized in that, Also includes: Based on a set motion rate or a set multiple of the motion capture target's motion rate, the exoskeleton device is driven to move the target to reproduce the motion.

7. A live streaming interactive system, characterized in that, include: Motion capture equipment is used to collect motion capture information of motion capture targets; The live streaming interaction terminal is used to acquire the motion capture information and forward it to the client. The client is used to interact with the live streaming interaction terminal, receive the motion capture information and forward it to the exoskeleton device; An exoskeleton device for performing the live motion reproduction method based on an exoskeleton device as described in any one of claims 1-6.

8. An exoskeleton device, characterized in that, include: Memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the live motion reproduction method based on an exoskeleton device as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a computer processor, are configured to perform the live motion reproduction method based on an exoskeleton device as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a computer or processor, cause the computer or processor to perform the live motion reproduction method based on an exoskeleton device as described in any one of claims 1-6.

Images