Motion management method of virtual motion object and electronic equipment

Abstract

The embodiment of the invention provides a motion management method of a virtual motion object and electronic equipment, and the method comprises the steps: obtaining the real motion information of a real motion object corresponding to the virtual motion object, and determining the first motion information of the virtual motion object according to the real motion information; obtaining second motion information of the virtual motion object from a preset standard motion information base; determining a motion information fusion coefficient; performing motion information fusion processing on the first motion information and the second motion information according to the motion information fusion coefficient, and determining target motion information corresponding to the virtual motion object; and controlling the motion of the virtual motion object according to the target motion information. Thus, through motion information fusion processing, the motion of the virtual motion object can be more fitted with the motion of the real motion object, and the motion is more standard and smoother, so that the virtual motion object can present the real motion of the real motion object more vividly and smoother, and a better motion effect is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a motion management method for a virtual moving object and an electronic device. Background Art

[0002] Currently, in the field of computer vision, it is possible to map the motion information of real-world moving objects, such as people and animals, onto computer-generated virtual moving objects, achieving motion mapping (also known as motion synchronization) between real and virtual moving objects. This motion mapping technology opens up new possibilities in fields such as animation, game development, virtual reality, and digital entertainment. With the continuous advancement of computer vision technology, users are increasingly demanding the motion effects of virtual moving objects achieved through this motion mapping technology. Therefore, how to make the motion of virtual moving objects more closely match that of real objects, and how to make the motion of virtual moving objects more vivid and smooth, thereby achieving better motion effects for virtual moving objects, are currently being explored in the field. Summary of the Invention

[0003] An embodiment of the present application provides a motion management method and electronic device for a virtual motion object, which performs motion information fusion processing based on first motion information of the virtual motion object determined according to real motion information of the real motion object, second motion information of the virtual motion object obtained from a preset standard motion information library, and a determined motion information fusion coefficient of the first motion information and the second motion information, and obtains target motion information of the virtual motion object to control the motion of the virtual motion object, so that the motion of the virtual motion object can be more closely aligned with the motion of the real motion object, and more standard and smoother, thereby making the virtual motion object's presentation of the real motion of the real motion object more vivid and smoother, and achieving better motion effects.

[0004] To solve the above technical problems, in the first aspect, an embodiment of the present application provides a motion management method for a virtual motion object, the method comprising: obtaining real motion information of a real motion object corresponding to the virtual motion object, and determining first motion information of the virtual motion object based on the real motion information; and obtaining second motion information of the virtual motion object from a preset standard motion information library; and determining a motion information fusion coefficient; performing motion information fusion processing on the first motion information and the second motion information based on the motion information fusion coefficient to determine target motion information corresponding to the virtual motion object; and controlling the motion of the virtual motion object based on the target motion information.

[0005] Using the above technical solution, after obtaining the real motion information of a real moving object, the first motion information of the corresponding virtual moving object is determined, and the second motion information of the virtual moving object is obtained from a standard motion information library. A motion information fusion coefficient is further determined, and the first motion information and the second motion information are fused according to the motion information fusion coefficient to obtain target motion information, thereby controlling the motion of the virtual moving object according to the target motion information. In this way, by fusion processing the first motion information of the virtual moving object and the second motion information corresponding to the virtual moving object obtained from the standard motion information library based on the motion information fusion coefficient to obtain target motion information, the movement presented by the virtual moving object can be more closely aligned with the real moving object, and more standard, smooth, vivid, and smooth, thereby achieving a better motion effect for the virtual moving object.

[0006] In a possible implementation of the first aspect above, after determining the first motion information of the corresponding virtual motion object, the method further includes: controlling the motion of the virtual motion object according to the first motion information.

[0007] Using the above technical solution, the movement of a virtual moving object can be controlled based on the first motion information. Secondary motion information of the virtual moving object can then be obtained from a standard motion information library. A motion information fusion coefficient can then be determined. The first and second motion information can then be fused using the fusion coefficient to obtain target motion information. This control of the virtual moving object's movement can then be achieved based on the target motion information. This allows for a smooth transition from the real motion information of the real object to the standard motion information. This relatively smooth transition allows the virtual moving object's movements to become increasingly standardized, ultimately ensuring that the resulting fused target motion information is closer to the standard.

[0008] In a possible implementation of the first aspect, after obtaining the second motion information corresponding to the virtual motion object from a preset standard motion information library, the method includes: controlling the motion of the virtual motion object according to the second motion information.

[0009] Using this technical solution, the motion of the virtual object can be controlled based on the second motion information. A motion information fusion coefficient is then determined. The first and second motion information are then fused based on the fusion coefficient to obtain target motion information. The virtual object's motion is then controlled based on the target motion information. This allows for a smooth transition from the standard motion information to the real motion information of the real object. This relatively smooth transition makes the virtual object's movements increasingly realistic, and the resulting fused target motion information is even closer to reality.

[0010] In a possible implementation of the first aspect above, the virtual motion object and the real motion object respectively include corresponding multiple target motion parts, and the motion information fusion coefficient is determined, including: determining the motion complexity and motion data stability of each target motion part; and determining the motion information fusion coefficient of each target motion part based on the motion complexity and motion data stability of each target motion part.

[0011] By adopting the above technical solution, each target motion part can obtain a different motion information fusion coefficient according to the corresponding motion complexity and motion data stability. In this way, the first motion information and the second motion information of different target motion parts can be fused to different degrees according to different motion information fusion coefficients, making the fusion processing more flexible, and thus making the final motion movement of the virtual motion object smoother and less abrupt.

[0012] In a possible implementation of the first aspect, each target motion part includes at least one motion node, the first motion information includes the first rotation information and the first position information of each motion node included in each target motion part, and the second motion information includes the second rotation information and the second position information of each motion node included in each target motion part. Then, according to the motion information fusion coefficient, the first motion information and the second motion information are subjected to motion information fusion processing to determine the target motion information corresponding to the virtual motion object, including: based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information and the second rotation information corresponding to each motion node included in each target motion part, and the motion information corresponding to each target motion part The method comprises the following steps: performing rotation information fusion processing on each motion node included in each target motion part according to the motion information fusion coefficient, and obtaining rotation fusion information corresponding to each motion node; and performing position information fusion processing on each motion node included in each target motion part according to the first position information and the second position information corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, and obtaining position fusion information corresponding to each motion node; obtaining target fusion information corresponding to each motion node according to the rotation fusion information corresponding to each motion node and the position fusion information corresponding to each motion node, so as to obtain target motion information of the virtual motion object.

[0013] Using the above technical solution, based on the quaternion spherical linear interpolation fusion algorithm, rotation information fusion processing is performed on each motion node included in each target motion part according to the first rotation information, the second rotation information, and the motion information fusion coefficient corresponding to each target motion part, to obtain rotation fusion information. Based on the position linear interpolation fusion algorithm, based on the first position information and the second position information, and the motion information fusion coefficient corresponding to each target motion part, position information fusion processing is performed on each motion node included in each target motion part to obtain position fusion information corresponding to each motion node. Based on the rotation fusion information and position fusion information corresponding to each motion node, the target motion information of the virtual motion object is obtained. In this way, the rotation information fusion processing makes the rotation of each motion node of the virtual motion object more stable and reasonable based on the real rotation of the real motion object and the standard rotation of the standard motion information, and the position information fusion processing makes the position of each motion node of the virtual motion object more reasonable.

[0014] In a possible implementation of the first aspect above, based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information and the second rotation information corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, rotation information fusion processing is performed on each motion node included in each target motion part to obtain rotation fusion information corresponding to each motion node, including: determining the angle between each first rotation information and each second rotation information; based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information, the second rotation information, the angle corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, rotation information fusion processing is performed on each motion node included in each target motion part to obtain rotation fusion information corresponding to each motion node.

[0015] In a possible implementation of the first aspect, the rotation fusion information corresponding to the motion node is obtained in the following manner:

[0016]

[0017]

[0018] in, is the rotation fusion information corresponding to the motion node, is the quaternion of the first rotation information of the motion node, is the quaternion of the second rotation information of the motion node, is the motion information fusion coefficient, for and The angle between them.

[0019] By adopting the above technical solution, based on the quaternion spherical linear interpolation fusion algorithm, rotation information fusion processing is performed on each motion node included in each target motion part according to the first rotation information, the second rotation information, the angle between the first rotation information and the second rotation information, and the motion information fusion coefficient corresponding to each target motion part, so that the rotation of the virtual motion object can be more stable and reasonable based on the real rotation of the real motion object and the standard rotation of the standard motion information.

[0020] In a possible implementation of the first aspect, the position fusion information corresponding to the moving node is obtained in the following manner:

[0021]

[0022] in, is the position fusion information corresponding to the motion node, is the second position information of the moving node, is the first position information of the moving node, is the motion information fusion coefficient.

[0023] By adopting the above technical solution, based on the position linear interpolation fusion algorithm, according to the first position information and the second position information, as well as the motion information fusion coefficient corresponding to each target motion part, the position information fusion processing of each motion node included in each target motion part is performed to obtain the position fusion information corresponding to each motion node, so that the position of each motion node of the virtual motion object is more reasonable.

[0024] In a possible implementation of the first aspect above, the motion complexity and motion data stability of each target motion part are determined, including: determining the number of motion nodes included in each target motion part and the motion amplitude information of each motion node, and determining the motion complexity of each target motion part based on the number and motion amplitude information; determining the confidence information of each motion node included in each target motion part, and determining the motion data stability of each target motion part based on the confidence information.

[0025] Using this technical solution, the motion complexity of each target moving part is determined based on the number of target moving parts and the motion amplitude information of each motion node, and the stability of the motion data of each target moving part is determined based on the confidence information of each motion node. This results in different motion complexity and motion data stability for different target moving parts, leading to different motion information fusion coefficients. This achieves varying degrees of fusion for different target moving parts. For different real moving objects, the fused target motion information differs, and the virtual moving object exhibits different motion actions, making the motion management of virtual moving objects more intelligent and flexible.

[0026] In a possible implementation of the first aspect above, the method also includes: determining a target frequency; obtaining real motion information of a real motion object corresponding to the virtual motion object, including: obtaining real motion information of the real motion object corresponding to the virtual motion object according to the target frequency; determining a motion information fusion coefficient, including: determining the motion information fusion coefficient according to the target frequency.

[0027] By adopting the above technical solution, the real motion information of the real motion object corresponding to the virtual motion object is obtained according to different target frequencies and the motion information fusion coefficient is determined, which can flexibly control the frequency of the motion information fusion of the virtual motion object.

[0028] In a possible implementation of the first aspect, determining the target frequency includes: determining a motion fineness of a real moving object; and determining the target frequency according to the motion fineness.

[0029] In a possible implementation of the first aspect above, the motion fineness is obtained through calibration or based on motion complexity.

[0030] By adopting the above technical solution, the motion fineness is obtained according to the motion complexity, and then the target frequency is obtained, so that the target frequency can be dynamically controlled according to the motion complexity.

[0031] In a possible implementation of the first aspect above, real motion information of a real motion object corresponding to a virtual motion object is obtained, and first motion information of the virtual motion object is determined based on the real motion information, including: obtaining real motion information of each target motion part included in the real motion object corresponding to the virtual motion object, the real motion information including real position information of each motion node included in each target motion part; obtaining first position information of each motion node included in each target motion part based on the real position information of the motion nodes; and determining relative position information between the motion nodes based on the real position information of the motion nodes; and determining first rotation information of each motion node included in each target motion part based on the relative position information.

[0032] In a possible implementation of the first aspect above, based on the real position information of the motion node, the first position information of each motion node included in each target motion part is obtained, including: based on the real position information of the motion node, the real position information is mapped to the virtual motion object to obtain the first position information of the virtual motion object.

[0033] In a possible implementation of the first aspect above, the first rotation information of each motion node included in each target motion part is determined based on the relative position information, including: determining the rotation axis and rotation angle of each motion node included in the target motion part based on the relative position information; and obtaining the first rotation information based on the rotation axis and rotation angle.

[0034] In a possible implementation of the first aspect, the first rotation information of the moving node includes a rotation axis and a rotation angle of the moving node. The method further includes obtaining the first rotation information in the following manner:

[0035]

[0036] in, is the first rotation information, , , is the value of the rotation axis corresponding to the x, y, and z coordinate axes, is the rotation angle.

[0037] In a possible implementation of the first aspect, the rotation axis is obtained as follows:

[0038]

[0039] in, is the rotation axis, , A is the first motion node, B is the second motion node, is the relative position vector of the first motion node relative to the second motion node, is the relative position vector of the second motion node relative to the first motion node.

[0040] In a possible implementation of the first aspect, the rotation angle is obtained as follows:

[0041]

[0042] in, is the cosine value of the rotation angle of the first motion node and the second motion node, is the modulus of the relative position vector of the first motion node relative to the second motion node, is the modulus of the relative position vector of the second motion node relative to the first motion node.

[0043] By adopting the above technical solution, the first position information and first rotation information of the virtual moving object are obtained according to the real position information of the real moving object, which facilitates the motion information fusion processing of the first position information and the first rotation information of the virtual moving object, making the movement of the virtual moving object more vivid and smooth.

[0044] In a possible implementation of the first aspect above, obtaining real motion information of a real motion object corresponding to a virtual motion object, and determining first motion information of the virtual motion object based on the real motion information, includes: obtaining real motion information of the real motion object corresponding to the virtual motion object; performing smoothing filtering on the real motion information, and determining the first motion information corresponding to the virtual motion object based on the real motion information after smoothing filtering.

[0045] By adopting the above technical solution and a smoothing filtering method, the real motion information is smoothed and filtered. The filtered data can reduce measurement errors, thereby obtaining more stable real motion information.

[0046] On the second aspect, the implementation method of the present application also discloses a virtual motion object motion management system, including a motion capture module, a motion fusion module and a motion drive module, wherein the motion capture module is used to obtain real motion information of a real motion object corresponding to the virtual motion object; the motion fusion module is used to determine the first motion information of the virtual motion object based on the real motion information, and obtain the second motion information of the virtual motion object from a preset standard motion information library, determine the motion information fusion coefficient, and perform motion information fusion processing on the first motion information and the second motion information according to the motion information fusion coefficient to determine the target motion information of the virtual motion object; the motion drive module is used to control the motion of the virtual motion object according to the target motion information.

[0047] In a possible implementation of the second aspect above, the action fusion module includes a standard motion information library management module and a motion information interpolation fusion algorithm management module, wherein the standard motion information library management module is used to store each second motion information in the form of an animation file; the motion information interpolation fusion algorithm management module is used to manage the quaternion spherical linear interpolation fusion algorithm and the position linear interpolation fusion algorithm.

[0048] In a third aspect, the implementation method of the present application also discloses an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device implements the motion management method of a virtual motion object provided by any one of the implementation methods of the first aspect above.

[0049] In a fourth aspect, the implementation of the present application further discloses a computer-readable storage medium, which stores a computer program. The computer program can be executed by an electronic device to implement the motion management method of a virtual motion object provided by any implementation of the first aspect above.

[0050] In a fifth aspect, the implementation of the present application further discloses a computer program product, including a computer program, which, when executed by an electronic device, implements the motion management method of a virtual motion object provided by any one of the implementations of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings used in the description of the implementation methods.

[0052] Figure 1 A schematic flow chart of a motion management method for a virtual motion object provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a process for determining first motion information provided by an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of a process for determining motion information fusion coefficients provided by an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a process for determining target motion information provided by an embodiment of the present invention;

[0056] Figure 5 A schematic structural diagram of a motion management system for a virtual motion object provided by an embodiment of the present invention;

[0057] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] Consider a scenario where the virtual moving object is a virtual digital human, and the real moving object is a dancer, performing a synchronized dance. Currently, achieving synchronized dance between the virtual digital human and the dancer typically involves tracking the dancer's posture using posture estimation methods to capture their motion information. This information is then used to drive the skeletal motion of the virtual digital human within a 3D engine. However, existing techniques often simply map the captured dancer's real-world motion information onto the virtual digital human, resulting in stiff movements and a significant deviation between the virtual digital human's movements and the dancer's, thus failing to achieve the desired performance.

[0059] Therefore, how to smooth the motion of virtual digital humans to make their movements more vivid and smooth, and to achieve better motion effects for virtual moving objects, is a problem that is currently being explored in the field.

[0060] Based on this, the implementation method of the present application provides a motion management method for a virtual motion object, which determines the first motion information of the corresponding virtual motion object based on the real motion information of the real motion object, obtains the second motion information corresponding to the first motion information of the virtual motion object from the standard motion information library, determines the motion information fusion coefficient of the first motion information and the second motion information, and performs motion information fusion processing on the first motion information of the virtual motion object and the corresponding second motion information in the standard motion information library according to the motion information fusion coefficient to obtain the target motion information of the virtual motion object to control the motion of the virtual motion object, so that the motion of the virtual motion object can be more in line with the motion of the real motion object, and more standard and smoother, so that the virtual motion object can present the real motion of the real motion object more vividly and smoothly, thereby achieving better motion effect.

[0061] like Figure 1 As shown, the motion management method of the virtual motion object provided by the implementation of the present application specifically includes the following steps:

[0062] S100 , obtaining real motion information of a real motion object corresponding to a virtual motion object, and determining first motion information of the virtual motion object according to the real motion information.

[0063] S200: Acquire second motion information of a virtual motion object from a preset standard motion information library.

[0064] S300: Determine a motion information fusion coefficient.

[0065] S400 , performing motion information fusion processing on the first motion information and the second motion information according to the motion information fusion coefficient to determine target motion information corresponding to the virtual motion object.

[0066] S500: Control the movement of the virtual moving object according to the target movement information.

[0067] In the implementation of the present application, after obtaining the real motion information of the real moving object, the first motion information of the corresponding virtual moving object is determined, and the second motion information of the virtual moving object is obtained from the standard motion information library. The motion information fusion coefficient is further determined, and the first motion information and the second motion information are subjected to motion information fusion processing according to the motion information fusion coefficient to obtain target motion information, and the motion of the virtual moving object is controlled according to the target motion information. In this way, the first motion information of the virtual moving object and the second motion information corresponding to the virtual moving object obtained from the standard motion information library are subjected to motion information fusion processing using the motion information fusion coefficient to obtain target motion information, so that the movement presented by the virtual moving object can be more consistent with the real moving object, and more standard, smooth, vivid and smooth, thereby achieving a better motion effect for the virtual moving object.

[0068] Next, taking the aforementioned dance movement synchronization scenario in which a virtual digital human (as an example of a virtual motion object) is controlled to perform corresponding movements according to the dance movements of a dancer (as an example of a real motion object) as an example, the motion management method of a virtual motion object provided by the implementation method of the present application is described in detail.

[0069] First, step S100 is executed.

[0070] In step S100, the real moving object and the virtual moving object include a plurality of corresponding target moving parts, each of which includes at least one motion node. The target moving parts of the real moving object and the target moving parts of the virtual moving object correspond one-to-one, and the motion nodes included in each target moving part of the real moving object and the motion nodes included in each target moving part of the virtual moving object correspond one-to-one.

[0071] like Figure 2 As shown, obtaining real motion information of a real motion object corresponding to a virtual motion object and determining first motion information of the virtual motion object according to the real motion information includes the following steps.

[0072] S110 , obtaining real motion information of each target motion part included in the real motion object corresponding to the virtual motion object.

[0073] The real motion information includes the real position information of each motion node corresponding to each target motion part.

[0074] Exemplarily, the motion information of each skeleton point (as an example of a motion node) of each motion part (as an example of a target motion part) of the dancer corresponding to the virtual digital human is obtained. The motion information of the skeleton point includes position information of the skeleton point.

[0075] Specifically, the pose recognition (MediaPipe Pose Landmark Detection) module deployed in the image collector can be used to capture and track the skeletal points of each dancer's moving parts (as examples of motion nodes) in real time, thereby obtaining motion information of the dancer's skeletal points (as examples of real motion information). Alternatively, the pose recognition module in the motion capture device can be used to capture and track the skeletal points of each dancer's moving parts in real time, thereby obtaining motion information of the dancer's skeletal points.

[0076] The image collector and motion capture device include cameras and other devices. The motion information of the dancer's skeleton points in each frame of the image is obtained by continuously acquiring the dancer's dance movements.

[0077] MediaPipe Pose Landmark Detection is a high-fidelity human pose tracking method based on BlazePose. It can output the motion information of each skeleton point of the human pose based on real-time performance on most modern mobile devices and desktop environments.

[0078] In the implementation of this application, MediaPipe Pose Landmark Detection can be used for motion capture to obtain human body posture joint information, and the skeletal animation of the virtual digital human can be driven according to the joint information in a 3D engine (such as Blender, Unity). The screen space key points are often reverse-projected into the world space, and then the bones are bound and driven.

[0079] In the implementation of the present application, the total number of skeleton points can be 33, and of course other numbers of skeleton points can also be defined.

[0080] Furthermore, in the implementation of the present application, a target frequency is also determined, and based on the target frequency, real motion information of a real motion object corresponding to the virtual motion object is obtained.

[0081] Exemplarily, determining the target frequency includes determining the motion fineness (also referred to as motion accuracy) of the real moving object, and determining the target frequency according to the motion fineness.

[0082] Specifically, the target frequency is obtained as follows:

[0083]

[0084] in, represents the target frequency, Indicates the fineness of the motion. The higher the motion fineness, the higher the target frequency, and the higher the frequency of obtaining the real motion information of the real moving object.

[0085] Furthermore, the motion fineness is obtained through calibration, that is, the motion fineness may be a pre-set fixed value.

[0086] Of course, the motion precision can also be obtained according to the motion complexity of each target motion part of the real motion object.

[0087] In the implementation of the present application, the real motion information includes information such as the number of motion nodes included in each target motion part and the motion amplitude of each motion node.

[0088] Determining the motion complexity of each target motion part includes: determining the number of motion nodes included in each target motion part and motion amplitude information of each motion node, and determining the motion complexity of each target motion part according to the number and motion amplitude information.

[0089] Exemplarily, the more bone points (as an example of the number of motion nodes) included in each motion part (as an example of the target motion part), the greater the motion amplitude of each bone point (as the motion amplitude information of the motion node), and the higher the motion complexity of the corresponding motion part (as an example of the target motion part).

[0090] Among them, the higher the movement complexity of each target movement part, the more complex the movement, and the higher the movement precision.

[0091] For example, the movement complexity of each target movement part can be divided into three levels: low complexity, medium complexity, and high complexity. If the movement complexity level of more than half of the movement parts is high, it is determined that the dancer's movement sophistication is relatively high.

[0092] Alternatively, the movement complexity of each moving part is assigned a value, with low complexity being assigned a value of -1, medium complexity being assigned a value of 0, and high complexity being assigned a value of 1. The movement complexity of each moving part is summed and averaged to obtain an average value. If the average value is greater than 0, it is determined that the dancer has a high degree of movement refinement.

[0093] Furthermore, after obtaining the real motion information of the real motion object corresponding to the virtual motion object, smoothing filtering is performed on the real motion information, and then the first motion information corresponding to the virtual motion object is determined based on the real motion information after the smoothing filtering.

[0094] Specifically, the motion information of skeleton points captured using MediaPipe can be noisy. Especially during fast movements or camera shake, the skeleton point positions (which serve as an example of the true position of each motion node) may be momentarily erroneous. Therefore, to smooth the skeleton point trajectories, a smoothing filter is used to process the position information of each skeleton point. This filtered data reduces measurement errors, resulting in more stable skeleton point position information.

[0095] In the implementation of the present application, for each frame of image, smoothing filtering can be implemented based on the position information of each skeleton point of the current frame image, the position information of each skeleton point of the previous frame image, and the smoothing coefficient based on the following formula:

[0096]

[0097] in, is the position information of each skeleton point after smoothing (as an example of the processed real motion information), is the smoothing coefficient, , is the position information of each skeleton point in the current frame image, It is the position information of each skeleton point in the previous frame image.

[0098] It should be noted that for the first frame image, is 0.

[0099] It should also be noted that each frame of image may include at least one skeleton point. The smoothing filtering processing of the position information of each skeleton point of each frame of image can be specifically performed by batch smoothing filtering processing on the position information of all skeleton points, or by individual smoothing filtering processing on the position information of each skeleton point of each frame of image until the position information of all skeleton points has completed the smoothing filtering processing.

[0100] In the implementation of the present application, smoothing filtering processing can be achieved based on low-pass filtering or Kalman filtering.

[0101] Furthermore, after obtaining the position information of the dancer's skeleton points corresponding to the virtual digital human and performing smoothing filtering on the position information of the skeleton points, the motion information corresponding to the virtual digital human is determined based on the processed position information of the skeleton points (as an example of first motion information).

[0102] The first motion information includes first rotation information and first position information of each bone point (as an example of a motion node of the target motion part included in the virtual motion object) included in each motion part of the virtual digital human (as an example of a target motion part included in the virtual motion object).

[0103] S120 , obtaining first position information of each moving node included in each target moving part according to the real position information of the moving node.

[0104] In the implementation method of the present application, the first position information of each motion node included in each target motion part is obtained based on the real position information of the motion node, including: mapping the real position information to the virtual motion object based on the real position information of the motion node to obtain the first position information of the virtual motion object.

[0105] Exemplarily, the position information of the dancer's skeleton points (as an example of the real position information of the motion nodes) is used as the position information of each skeleton point included in each motion part of the virtual digital human (as an example of the first position information of each motion node).

[0106] S130: Determine relative position information between the moving nodes based on the real position information of the moving nodes.

[0107] For example, by calculating the position coordinates of any two bone points and Interpolation is performed (as an example of position information) to obtain the relative position information between each pair of skeleton points. The relative position vector represents the relative position and orientation of two skeleton points in 3D space. By changing the relative position vector of a skeleton point, the relative motion between the skeleton points can be determined.

[0108] Exemplarily, there are a first motion node and a second motion node, the first motion node is called motion node A, and the second motion node is called motion node B.

[0109] That is, assuming two skeleton points A (as an example of motion node A) and B (as an example of motion node B), the position coordinates of A are , the position coordinates of B are , then the relative position information between the skeleton points can be obtained as follows:

[0110]

[0111] in, is the relative position information of bone point A and bone point B, is the position information of the skeleton point B, , is the position information of the skeleton point A, .

[0112] S140: Determine first rotation information of each motion node included in each target motion part according to the relative position information.

[0113] The first rotation information of the moving node includes the rotation axis and rotation angle of the moving node.

[0114] In the implementation of the present application, the first rotation information of each motion node included in each target motion part is determined according to the relative position information, including: determining the rotation axis and rotation angle of each motion node included in the target motion part according to the relative position information, and obtaining the first rotation information according to the rotation axis and rotation angle.

[0115] For example, the rotation axis is first calculated. The rotation axis is the cross product of the relative position vectors between the two bone points. The cross product generates a vector perpendicular to the two vectors, indicating the direction of the rotation axis.

[0116] The rotation axis is obtained as follows:

[0117]

[0118] in, is the rotation axis, , is the relative position vector of bone point A relative to bone point B, It is the relative position vector of bone point B relative to bone point A.

[0119] Furthermore, the rotation angle of the two skeleton points is determined based on the relative position information , rotation angle It can be calculated by the dot product between the relative position vectors of two bone points.

[0120] Specifically, the rotation angle is obtained as follows:

[0121]

[0122] in, is the cosine value of the rotation angle between bone point A and bone point B, is the modulus of the relative position vector of bone point A relative to bone point B, It is the modulus of the relative position vector of bone point B relative to bone point A.

[0123] Furthermore, the rotation axis and rotation angle are combined to use quaternion form to represent the rotation of the bone point.

[0124] Specifically, the first rotation information of the relative motion between skeleton point A and skeleton point B is obtained as follows:

[0125]

[0126] in, is the first rotation information, , , is the value of the rotation axis corresponding to the x, y, and z coordinate axes, is the rotation angle.

[0127] In the implementation of the present application, the use of quaternions to represent rotation can effectively avoid the gimbal lock problem that may occur when using Euler angles.

[0128] In this way, by directly assigning a value to the rotation property of the joint transform in Unity and applying the first rotation information to the corresponding skeletal points of the virtual digital human, the skeletal points of the virtual digital human can be driven to rotate. However, if the dancer's movements are stiff, the corresponding virtual digital human's movements will also be stiff, which is not conducive to the presentation of the movements. Therefore, in the implementation of this application, the motion information of the virtual digital human is also smoothed to make the movement of the virtual digital human more vivid and smooth.

[0129] In an implementation of the present application, after determining the first motion information of the corresponding virtual motion object, the method includes: controlling the motion of the virtual motion object according to the first motion information.

[0130] Exemplarily, after obtaining the real position information of the real moving object and performing smoothing filtering, the first position information corresponding to the virtual moving object is obtained, and the relative position information is calculated based on the real position information, thereby obtaining the rotation axis and rotation angle corresponding to the virtual moving object. The first position information, rotation axis, and rotation angle are applied to each motion node of the virtual moving object to drive the virtual digital human to move, and steps S200 to S500 are further executed to perform motion information fusion processing and smooth the motion of the virtual digital human.

[0131] Next, execute step S200.

[0132] In step S200 , a standard motion information library is set, and standard motion information matching the first motion information is retrieved from the preset standard motion information library as second motion information corresponding to the virtual motion object.

[0133] The second motion information includes second rotation information and second position information of each motion node included in each target motion part.

[0134] Among them, the standard motion information library may contain motion information of a virtual motion object under different motion actions, and retrieving standard motion information matching the first motion information from the preset standard motion information library may be to determine the similarity value between the first motion information and each standard motion information, and determine the standard motion information with the highest similarity value as the second motion information.

[0135] Of course, the standard motion information library can also include different standard motion segments of virtual motion objects and motion information for these standard motion segments. Each standard motion segment includes multiple standard motion actions, each of which includes the time progression of the virtual motion object in the standard motion segment when performing the corresponding standard motion action, the position and rotation information of the motion nodes of each moving part of the virtual motion object performing the corresponding standard motion action, and the scaling information of the virtual motion object.

[0136] Retrieving the standard motion information that matches the first motion information from the preset standard motion information library can be based on the time progress of the real motion object in the corresponding target motion segment when performing the target motion action, determining the standard motion action under the same time progress in the standard motion segment corresponding to the target motion segment, and obtaining the motion information corresponding to the standard motion action as the second motion information.

[0137] For example, using dance as an example, a standard action library (as an example of a standard motion information library) is set up. The standard action library stores a series of preset standard dance moves, which can be retrieved as needed. These standard dance moves are stored in the AnimationClip format in Unity, allowing them to be directly played, controlled, and interpolated in the Unity engine.

[0138] The standard action library is a collection of multiple AnimationClips. Each AnimationClip represents a dance action segment. These dance action segments can be any standard dance steps or dance moves.

[0139] These dance action clips are stored in action files (AnimationClip files) in the .anim format. They consist of a series of keyframes, each of which contains the following information:

[0140] Time: The time position (i.e., time progression) of the keyframe in the standard dance clip (usually in seconds).

[0141] Position information: the world coordinates or local coordinates of each skeletal point of the virtual motion object at that time point (as an example of the second position information of the motion nodes included in the target motion part).

[0142] Rotation information: the rotation information of the virtual motion object at that point in time, usually represented by quaternion (as an example of the second rotation information of the motion node included in the target motion part).

[0143] Scaling information: the scaling ratio of the virtual motion object at that point in time.

[0144] For example, when a dancer dances in front of a dance machine, he or she imitates the music and dance clips given by the dance machine. Therefore, according to the time progress of the dance clip where the dancer's corresponding dance movement is located, the standard dance movement (as an example of a standard movement movement) that matches the dance movement at the same time progress in the corresponding standard dance clip can be retrieved from the dance movement library as the standard movement movement corresponding to the virtual digital human, and the position and rotation information of each bone point including the movement parts involved in the standard dance movement can be obtained.

[0145] In this way, the corresponding standard movement information can be easily retrieved from the standard movement library according to the time. For example, when a dancer dances to music, the standard movement information can be obtained in real time according to the current time progress.

[0146] Regarding step S300 , in one implementation, determining the motion information fusion coefficient includes: determining according to real motion information of the real motion object.

[0147] Exemplarily, a comparison table of motion information fusion coefficients and real motion information is set, and the corresponding motion information fusion coefficients are searched according to the real motion information.

[0148] In another implementation method, because sports movements such as dance movements are in a continuous changing process, they are highly complex and diverse in form. Therefore, the requirements for fusion transition effects of different movement parts will vary greatly. If only fixed weight values are used, the motion information fusion coefficients for different motion objects are all the same value, which lacks flexibility and cannot meet the complex and changeable requirements of dance movements. Therefore, how to balance the weights of real-time captured real motion information and standard dance movements to prevent abrupt jumps and unnatural movement transitions has become a difficult problem that the industry urgently needs to overcome.

[0149] In another implementation of the present application, after obtaining the second motion information corresponding to the virtual motion object from a preset standard motion information library, the method includes: controlling the motion of the virtual motion object according to the second motion information.

[0150] Exemplarily, after obtaining the second motion information corresponding to the virtual motion object, the first motion information corresponding to the virtual motion object is determined based on the real motion information of the real motion object, and the motion information fusion coefficient is further determined. The first motion information and the second motion information are subjected to motion information fusion processing according to the motion information fusion coefficient to obtain target motion information, so as to control the virtual motion object to transition from standard motion to real motion according to the target motion information.

[0151] Based on this, in another implementation of this application, Figure 3 As shown, determining the motion information fusion coefficient includes the following steps:

[0152] S310: Determine the motion complexity and motion data stability of each target motion part.

[0153] Exemplarily, the number of motion nodes included in each target motion part and the motion amplitude information of each motion node are determined based on the method shown above, and the motion complexity of each target motion part is determined according to the number and the motion amplitude information.

[0154] Furthermore, in the implementation of this application, the real motion information also includes confidence information for each motion node included in each target motion part. The confidence information for each motion node primarily reflects the change in the motion node between the previous frame image and the current frame image. The greater the change, the lower the confidence, and the smaller the change, the higher the confidence.

[0155] Determining the stability of the motion data of each target motion part includes: determining confidence information of each motion node included in each target motion part, and determining the stability of the motion data of each target motion part according to the confidence information.

[0156] For example, the confidence level of each skeleton point is obtained based on MediaPipe Pose Landmark Detection, and the motion data stability of the corresponding target motion part is obtained based on the confidence level of each skeleton point. The higher the confidence level, the higher the motion data stability.

[0157] S320 , determining a motion information fusion coefficient of each target motion part according to the motion complexity and motion data stability of each target motion part.

[0158] In one implementation of the present application, the motion information fusion coefficient is determined by:

[0159]

[0160] in, is the motion information fusion coefficient, is a function that is dynamically calculated based on the motion complexity and motion data stability corresponding to the first motion information. is the motion complexity corresponding to the first motion information, is the stability of the motion data corresponding to the first motion information, Able to adaptively adjust the motion information fusion coefficient, .

[0161] In the implementation of the present application, motion complexity and motion data stability are obtained by recording real motion information over continuous time.

[0162] Furthermore, in another implementation of the present application, the motion information fusion coefficient includes a first motion information fusion coefficient and a second motion information fusion coefficient, wherein the first motion information fusion coefficient is , the second motion information fusion coefficient is .

[0163] In one implementation, the first motion information fusion coefficient is obtained according to the above method, and the second motion information fusion coefficient is obtained according to It can be obtained by looking up a preset comparison table of the first motion information fusion coefficient and the second motion information fusion coefficient.

[0164] In another implementation, the first motion information fusion coefficient is obtained according to the aforementioned method, and the second motion information fusion coefficient is obtained by looking up a comparison table between the second motion information and the second motion information fusion coefficient.

[0165] In another implementation, the first motion information fusion coefficient is obtained according to the aforementioned method, and the second motion information fusion coefficient is obtained according to the following method:

[0166]

[0167] in, is the second motion information fusion coefficient, It is a function that is dynamically calculated based on the motion complexity and motion data stability corresponding to the second motion information. is the motion complexity corresponding to the second motion information, is the stability of the motion data corresponding to the second motion information, The second motion information fusion coefficient can be adaptively adjusted.

[0168] If a static fusion of standard motion and real motion is performed based on a fixed fusion coefficient, it will not be able to meet the different transition effects requirements of different dance segments in complex dance movements. Therefore, in the implementation method of this application, a dynamic weight (i.e., motion fusion coefficient) control mechanism is introduced. Under this mechanism, the motion information fusion coefficient is dynamically adjusted according to the actual requirements of the dancer's dance movements. For example, the motion information fusion coefficient is calculated based on the dancer's movement complexity (i.e., movement complexity) and the stability of the real-time captured motion data. In this way, in relatively smooth movement transitions, a higher motion information fusion coefficient is selected to increase the weight of the real-time motion information, while in dance movements that require rapid conversion, a lower motion information fusion coefficient is selected so that the real-time captured motion information is given a lower weight, thereby ensuring the naturalness and smoothness of the displayed movements.

[0169] Furthermore, in the implementation of the present application, a motion information fusion coefficient is calculated for each target motion part, so that different motion information fusion coefficients can be obtained according to the actual motion information of each target motion part. For example, the key movements of some dances are in the upper body (as an example of a target motion part). The corresponding motion information fusion coefficients are dynamically determined according to the motion complexity and motion data stability of the real motion information corresponding to the upper and lower bodies. Based on the respective motion information fusion coefficients, the fused motion information of the upper body is closer to the first motion information captured in real time, and the fused motion information of the lower body completely uses the second motion information. Or, for example, if some dance hand movements pose a great challenge to the dancer, the corresponding motion information fusion coefficients are dynamically determined according to the motion complexity and motion data stability of the real motion information of the upper body, hands, and lower body. Based on the respective corresponding motion information fusion coefficients, the fused motion information of the upper body is closer to the first motion information, and the fused motion information of the hands completely uses the second motion information. This can ensure the consistency and beauty of the final motion effect.

[0170] Furthermore, in the implementation of the present application, the motion information fusion coefficient is also determined according to the target frequency.

[0171] Exemplarily, the motion information fusion coefficient is calculated based on the target frequency.

[0172] Therefore, for scenarios requiring high precision in motion information fusion (such as during transitions between fast dance moves or when movements are rapidly changing and complex), the frequency of real-time data capture needs to be increased. Therefore, based on the target frequency, more frequent acquisition of the dancer's motion information and calculation of the motion information fusion coefficients are necessary to dynamically adjust the weighting between the first and second motion information, making motion information fusion more flexible. Specifically, if the dancer's movements are consistently large and complex over a continuous period of time, increasing the calculation frequency (i.e., the target frequency) can ensure that key movement details are not missed.

[0173] Next, step S400 is executed.

[0174] like Figure 4 As shown, in step S400, performing motion information fusion processing on the first motion information and the second motion information according to the motion information fusion coefficient to determine the target motion information of the virtual motion object includes the following steps:

[0175] S410, based on the quaternion spherical linear interpolation fusion (Slerp) algorithm, rotation information fusion processing is performed on each motion node included in each target motion part according to the first rotation information and the second rotation information corresponding to each motion node included in each target motion part, as well as the motion information fusion coefficient corresponding to each target motion part, to obtain rotation fusion information corresponding to each motion node.

[0176] In the implementation of the present application, based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information and the second rotation information corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, rotation information fusion processing is performed on each motion node included in each target motion part to obtain rotation fusion information corresponding to each motion node, including: determining the angle between each first rotation information and each second rotation information; based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information, the second rotation information, the angle corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, rotation information fusion processing is performed on each motion node included in each target motion part to obtain rotation fusion information corresponding to each motion node.

[0177] Exemplarily, the rotation fusion information corresponding to the motion node is obtained in the following manner:

[0178]

[0179]

[0180] in, is the rotation fusion information corresponding to the motion node, is the quaternion of the first rotation information of the motion node, is the quaternion of the second rotation information of the motion node, is the motion information fusion coefficient, for and The angle between them.

[0181] In virtual digital human animation, especially in application scenarios such as dancing that require complex limb joint rotation, the interpolation of rotational movements is particularly critical. In the implementation method of this application, the quaternion spherical linear interpolation fusion algorithm is used to perform motion information fusion processing on each target motion part based on the motion information fusion coefficient of each target motion part, which can ensure that the rotational movement of each motion joint transitions evenly on the four-dimensional unit sphere. It can also effectively avoid the universal joint lock phenomenon and ensure the smoothness and continuity of complex rotational movements. For example, for the rotation of the motion joints of the target motion parts such as the arms and legs of the virtual digital human, the Slerp interpolation algorithm can ensure that the rotation of each motion joint between different motions is smooth and unobtrusive, and can avoid unnatural rotation trajectories.

[0182] For example, based on the quaternion spherical linear interpolation fusion algorithm, the quaternion rotation information of each skeletal point of each moving part of the dancer and the quaternion rotation information of each skeletal point of each moving part of the corresponding standard dance movement are fused according to the motion information fusion coefficient corresponding to each moving part, thereby obtaining the rotational fusion information of each skeletal point. In this way, based on the motion information fusion coefficient corresponding to the target moving part, the rotational information fusion processing is performed on all skeletal points included in the target moving part to obtain the rotational fusion information corresponding to each skeletal point, thereby achieving different degrees of rotational fusion for each target moving part.

[0183] Furthermore, if a motion node is a target motion part, different degrees of rotational fusion can be achieved for each motion node, thus making the rotational fusion more detailed.

[0184] S420, based on the linear interpolation (LERP) fusion algorithm, position information fusion processing is performed on each motion node included in each target motion part according to the first position information and the second position information corresponding to each motion node included in each target motion part, as well as the motion information fusion coefficient corresponding to each target motion part, to obtain position fusion information corresponding to each motion node.

[0185] The position fusion information corresponding to the motion node is obtained in the following way:

[0186]

[0187] in, is the position fusion information corresponding to the motion node, is the first position information of the moving node, is the second position information of the moving node, is the motion information fusion coefficient.

[0188] Of course, in addition to rotation, the position change of the virtual digital human is also a key factor affecting the naturalness of movement. Through the linear interpolation method with dynamic weight adjustment, the position interpolation of the virtual digital human is made more precise. According to the current movement state of the virtual digital human, the appropriate interpolation ratio is adjusted based on the fusion of motion information to ensure that the movement and rotation of the position are closely coordinated to avoid "floating" or "abrupt" effects. In addition, for certain important movement nodes of the virtual digital human (such as the feet and shoulders), additional weights are added based on the dynamically determined motion information fusion coefficients of each target movement part to ensure the consistency of the movement of these key parts with the whole body movement.

[0189] For example, based on the motion information fusion coefficient corresponding to each moving part, a position information fusion process is performed on the position information of each skeletal point of each moving part of the dancer and the position information of the skeletal points of each moving part of the corresponding standard dance movements, thereby obtaining the position fusion information of each skeletal point. In this way, based on the motion information fusion coefficient corresponding to the target moving part, the position information fusion process is performed on all skeletal points included in the target moving part to obtain the position fusion information corresponding to each skeletal point, thereby achieving different degrees of position fusion for each target moving part.

[0190] It should be noted that the second motion information has a time attribute, and the second motion information corresponding to the first motion information at a specified time in the dance segment can be obtained. That is, for the position information included in the motion information at a corresponding time, the first position information can be obtained based on real-time capture data, and the second position information can be obtained from the standard motion information library.

[0191] Furthermore, if a motion node is a target motion part, different degrees of rotational fusion can be achieved for each motion node, thus making the rotational fusion more detailed.

[0192] The quaternion spherical linear interpolation algorithm and the position linear interpolation algorithm are mainstream interpolation methods. They have demonstrated certain advantages and achieved success in many application scenarios. Simple linear interpolation will have speed inconsistencies and rotational deformities, which are difficult to meet the needs of high-quality dance performance. This application introduces an interpolation fusion algorithm that combines the weight-controlled quaternion spherical linear interpolation algorithm and the position linear interpolation algorithm, so that virtual digital humans can achieve seamless transitions between real-time dance movements and standard dance movements.

[0193] S430 , obtaining target fusion information corresponding to each motion node according to the rotation fusion information corresponding to each motion node and the position fusion information corresponding to each motion node, so as to obtain target motion information of the virtual motion object.

[0194] Exemplarily, based on the rotation fusion information and position fusion information corresponding to each skeletal point, the fusion information of the motion node [rotation fusion information, position fusion information] (as an example of target fusion information) is obtained, and then the fusion information of each target motion part [motion node 1 [rotation fusion information 1, position fusion information 1], ..., motion node N [rotation fusion information N, position fusion information N]] is obtained. In this way, the fusion information of the virtual digital human is obtained {target motion part 1 [motion node 1 [rotation fusion information 1, position fusion information 1], ..., motion node N [rotation fusion information N, position fusion information N]], ..., target motion part N [motion node 1 [rotation fusion information 1, position fusion information 1], ..., motion node N [rotation fusion information N, position fusion information N]]}.

[0195] In this application, by introducing multi-scale fusion of different target motion parts and a dynamic adjustment mechanism for each motion node included in each target motion part, the interpolation fusion weights at different scales are dynamically adjusted according to the difficulty and complexity of the motion, as well as other requirements of the transition phase. The motion information fusion coefficient is calculated based on different calculation frequencies to control the calculation accuracy of the motion information fusion coefficient. This makes the fusion of motion information of the virtual digital human more flexible, and by fusing the motion information of different motion nodes, the naturalness and accuracy of the virtual digital human's movements are improved.

[0196] Furthermore, step S500 is executed to drive the virtual digital human to move according to the target motion information.

[0197] For example, applying the target motion information including the rotation fusion information and the position fusion information to the skeleton points of the virtual digital human can drive the skeleton points of the virtual digital human to rotate.

[0198] This can be achieved by directly assigning a value to the rotation property of the transform of the bone point in Unity.

[0199] The motion management method for virtual motion objects provided by the implementation of this application is a method for implementing dance movement detection and virtual digital human movement fusion. It uses the MediaPipe Pose Landmark Detection module to achieve real-time capture and tracking of 33 joints (i.e., skeletal points) of the dancing body, and maps the captured joints to the skeletal model of the virtual digital human, thereby driving the virtual digital human to dance synchronously. Furthermore, the corresponding dance movements are retrieved from a pre-built standard movement library, and based on a hybrid fusion algorithm combining quaternion spherical linear interpolation (Slerp) and position linear interpolation (Lerp), the current movement of the virtual digital human and the standard movement are dynamically weighted and fused. This allows the virtual digital human to retain the realism of the movement while complying with standard specifications, improving the natural fluency and accuracy of the movement. It has significant technical advantages and beneficial effects in the fields of dance teaching, virtual interpretation, game development, etc.

[0200] Specifically, a quaternion spherical linear interpolation fusion algorithm fuses real-time rotation data with rotation data from a standard motion library, ensuring a smooth and continuous transition in skeletal animation rotations, avoiding the rapid rotations or unnatural sudden movements that can occur with traditional linear interpolation. Furthermore, a position-based linear interpolation fusion algorithm ensures the natural and fluid movement of the virtual human.

[0201] In addition, the flexible motion information fusion coefficient control mechanism can determine which effect should be preferred based on different dance styles (such as modern dance, ancient dance, folk dance, etc.) and different application scenarios (dance teaching scenarios, virtual performance scenarios, etc.) through the user's visual interactive interface or preset mode selection, so as to freely adjust the motion information fusion coefficient and realize the dynamic transition from fully following real-time motion information to fully playing standard motion information.

[0202] By building multiple high-quality standard movement libraries and incorporating interpolation and fusion algorithms, the system seamlessly integrates virtual human movements with standard dance moves in various scenarios. This not only automatically selects appropriate dance steps from the standard movement library, but also supports the generation of new personalized movements based on user needs, meeting the diverse and creative demands of dance performance. This not only overcomes the technical shortcomings of existing virtual human dance drivers, but also demonstrates strong advantages in user experience, system efficiency, adaptability, and scalability, providing innovative solutions for the development of related industries.

[0203] Furthermore, the above steps S100, S200, and S300 may be executed in the order of S100, S200, and S300, or may be executed simultaneously, or S200 may be executed first, and then S100, S300, etc., which may be selected as needed.

[0204] Furthermore, if Figure 5 As shown, the implementation of the present application also provides a virtual motion object motion management system, including a motion capture module, a motion fusion module and a motion drive module.

[0205] The motion capture module is used to obtain the real motion information of the real motion object corresponding to the virtual motion object.

[0206] For example, the joint coordinates (as an example of real position information), confidence information, number of joints, and motion amplitude information of each frame of the dancer's RGB video are obtained based on MediaPipe Pose Landmark Detection.

[0207] The motion fusion module is used to determine the first motion information of the virtual motion object based on the real motion information, and obtain the second motion information of the virtual motion object from a preset standard motion information library, determine the motion information fusion coefficient, and perform motion information fusion processing on the first motion information and the second motion information according to the motion information fusion coefficient to determine the target motion information of the virtual motion object.

[0208] Among them, the motion fusion module includes a standard motion information library management module and a motion information interpolation fusion algorithm management module (also called a motion interpolation fusion algorithm management module). The standard motion information library management module is used to construct multiple sets of standard motion information databases (such as standard dance motion databases) and store standard motion information (that is, second motion information) in the form of animation files, so as to facilitate the subsequent rapid retrieval and application of standard motion information.

[0209] The motion information interpolation fusion algorithm management module is used to manage the quaternion spherical linear interpolation fusion algorithm and the position linear interpolation fusion algorithm. On the basis of driving the virtual motion object, it combines the standard motion information in the standard motion information library and uses the interpolation fusion algorithm to make the motion of each motion node of the virtual motion object smoother.

[0210] Exemplarily, corresponding preset standard dance movements are retrieved from the standard movement library, and the current motion information (i.e., the first motion information) of the virtual digital human is smoothly fused with the standard motion information (i.e., the second motion information) based on the quaternion spherical linear interpolation fusion algorithm and the position linear interpolation fusion algorithm.

[0211] The motion driving module is used to drive the virtual motion object to move according to the first motion information, and to control the motion of the virtual motion object according to the target motion information.

[0212] For example, after obtaining the real position information of a real moving object and performing a smoothing filter process, the first position information corresponding to the virtual digital human is obtained. Relative position information is then calculated based on the real position information, and the rotation axis and rotation angle (first rotation information) corresponding to the virtual moving object are then obtained. The rotation axis, rotation angle, and first position information are then applied to each motion node of the virtual moving object to drive the virtual digital human to move. The virtual digital human is further driven to move based on the target motion information determined by the motion fusion module. In this way, the virtual digital human's motion is converted from real motion to relatively standard motion.

[0213] The motion management system for virtual motion objects provided by the implementation of this application supports modular design, facilitating the expansion of new functionality as needed. For example, in addition to dance movements, the system can be expanded to include multiple human posture recognition, expression capture, gesture control, and other aspects, further enriching the expressive capabilities of virtual humans. Furthermore, the modular design of the standard motion information library and interpolation fusion algorithm facilitates the addition of new motion information and algorithm optimization, providing strong scalability.

[0214] The motion management method and system for virtual motion objects provided by the implementation of the present application can be widely used in multiple fields, including but not limited to virtual reality (VR), augmented reality (AR), online education, virtual idols, game interaction, etc. In these fields, the high immersion and interactivity of the user experience will be greatly improved, especially in the application of entertainment and education, which can create more attractive and interactive virtual characters.

[0215] See Figure 6 , Figure 6 The figure shows a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 6 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0216] Processor 122 executes computer-executable instructions stored in memory, causing processor 122 to implement the technical solution of the motion management method for a virtual moving object in the above-described embodiment. Processor 122 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0217] The memory 123 is connected to the processor 122 via a system bus and communicates with the processor 122. The memory 123 is used to store computer program instructions.

[0218] By way of example and not limitation, memory 123 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more thereof. Where appropriate, memory 123 may include removable or non-removable (or fixed) media. Where appropriate, memory 123 may be internal or external to the integrated gateway device. In certain embodiments, memory 123 is a non-volatile solid-state memory. In certain embodiments, memory 123 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more thereof. The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0219] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. System buses can be divided into address buses, data buses, and control buses. For ease of illustration, the diagram uses only a single thick line, but this does not imply a single bus or type of bus. Transceivers enable communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and non-volatile memory.

[0220] Furthermore, the electronic device may be, for example, a computer, a mobile phone, a server or other electronic device.

[0221] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the motion management method of the virtual motion object in the above embodiment.

[0222] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the technical solution of the motion management method of a virtual motion object of the above embodiment.

[0223] In some possible implementations, various aspects of the method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a processor of an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of the method according to various exemplary implementations of the present application described above in this specification. For example, the electronic device can execute the motion management method of the virtual motion object recorded in the embodiments of the present application.

[0224] The program product may employ any combination of one or more readable media. The readable medium may be a readable data medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CDROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0225] The implementation method of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the motion management method of the virtual motion object in the above embodiment.

[0226] It should be noted that, in addition to the implementation of the present application described in the above-mentioned specific embodiments, those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Although the description of the present application is introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may be extended from the present application. In order to provide an in-depth understanding of the present application, the above description contains many specific details, and the present application can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0227] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0228] It should be noted that the terms "first", "second", etc. are only used for distinction and description, and cannot be understood as indicating or implying relative importance.

[0229] It should be noted that in the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0230] Although the present application has been illustrated and described with reference to certain preferred implementations of the present application, those skilled in the art should understand that the above description is provided as a further detailed explanation of the present application in conjunction with specific implementations, and that the specific implementation of the present application should not be limited to these descriptions. Those skilled in the art may make various changes in form and detail, including simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. A motion management method for a virtual motion object, characterized in that: The virtual moving object and the real moving object corresponding to the virtual moving object respectively include a plurality of corresponding target moving parts, each of the target moving parts includes at least one moving node, and the method includes: Acquiring real motion information of each target motion part included in the real motion object corresponding to the virtual motion object, and determining first motion information corresponding to each target motion part included in the virtual motion object according to the real motion information; and Acquire second motion information corresponding to each target motion part included in the virtual motion object from a preset standard motion information library; and Determining a motion information fusion coefficient of each target motion part, wherein the motion information fusion coefficient of each target motion part is obtained according to the motion complexity and motion data stability of each target motion part, the motion complexity of each target motion part is obtained according to the number of motion nodes included in each target motion part and the motion amplitude information of each motion node, and the motion data stability of each target motion part is obtained according to the confidence information of each motion node included in each target motion part; performing motion information fusion processing on the first motion information and the second motion information corresponding to each target motion part according to the motion information fusion coefficient of each target motion part, to determine target motion information corresponding to each target motion part included in the virtual motion object; The movement of the virtual moving object is controlled according to the target movement information corresponding to each target movement part.

2. The motion management method of a virtual motion object according to claim 1, characterized in that: The first motion information includes first rotation information and first position information of each motion node included in each target motion part, and the second motion information includes second rotation information and second position information of each motion node included in each target motion part. Then, according to the motion information fusion coefficient of each target motion part, motion information fusion processing is performed on the first motion information and the second motion information corresponding to each target motion part to determine the target motion information corresponding to each target motion part included in the virtual motion object, including: Based on the quaternion spherical linear interpolation fusion algorithm, according to the first rotation information and the second rotation information corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, rotation information fusion processing is performed on each motion node included in each target motion part to obtain rotation fusion information corresponding to each motion node; and Based on the position linear interpolation fusion algorithm, according to the first position information and the second position information corresponding to each motion node included in each target motion part, and the motion information fusion coefficient corresponding to each target motion part, position information fusion processing is performed on each motion node included in each target motion part to obtain position fusion information corresponding to each motion node; According to the rotation fusion information corresponding to each motion node and the position fusion information corresponding to each motion node, the target fusion information corresponding to each motion node is obtained to obtain the target motion information corresponding to each target motion part included in the virtual motion object.

3. The motion management method of a virtual motion object according to claim 2, characterized in that: The rotation fusion information corresponding to the motion node is obtained in the following manner: in, is the rotation fusion information corresponding to the motion node, is the quaternion of the first rotation information of the motion node, is the quaternion of the second rotation information of the moving node, is the motion information fusion coefficient, for and The angle between The position fusion information corresponding to the moving node is obtained in the following manner: in, is the position fusion information corresponding to the motion node, is the second position information of the moving node, is the first position information of the moving node, is the motion information fusion coefficient.

4. The motion management method of a virtual motion object according to claim 3, characterized in that: The method further comprises: Determine the target frequency; Then, obtaining the real motion information of each target motion part included in the real motion object corresponding to the virtual motion object includes: Acquiring, according to the target frequency, real motion information of each target motion part included in the real motion object corresponding to the virtual motion object; Determining the motion information fusion coefficient of each target motion part includes: The motion information fusion coefficient of each target motion part is determined according to the target frequency.

5. The motion management method of a virtual motion object according to claim 4, characterized in that: Acquiring real motion information of each target motion part included in the real motion object corresponding to the virtual motion object, and determining first motion information of each target motion part included in the virtual motion object according to the real motion information, including: Acquire the real motion information of each target motion part included in the real motion object corresponding to the virtual motion object, wherein the real motion information includes real position information of each motion node included in the target motion part; Obtaining the first position information of each of the moving nodes included in each of the target moving parts according to the real position information of the moving nodes; and Determining relative position information between the moving nodes based on the real position information of the moving nodes; The first rotation information of each of the motion nodes included in each of the target motion parts is determined according to the relative position information.

6. The motion management method of a virtual motion object according to claim 5, characterized in that: The first rotation information of the moving node includes a rotation axis and a rotation angle of the moving node. The method further includes obtaining the first rotation information by: in, is the first rotation information, , , is the value of the rotation axis corresponding to the x, y, and z coordinate axes, is the rotation angle; The rotation axis is obtained as follows: in, is the rotation axis, , A is the first motion node, B is the second motion node, is the relative position vector of the first motion node relative to the second motion node, is the relative position vector of the second motion node relative to the first motion node; The rotation angle is obtained as follows: in, is the cosine value of the rotation angle of the first motion node and the second motion node, is the modulus of the relative position vector of the first motion node relative to the second motion node, is the modulus of the relative position vector of the second motion node relative to the first motion node.

7. The motion management method for a virtual motion object according to any one of claims 1 to 6, characterized in that: Acquiring real motion information of each target motion part included in the real motion object corresponding to the virtual motion object, and determining first motion information of each target motion part included in the virtual motion object according to the real motion information, including: Acquiring the real motion information of each target motion part included in the real motion object corresponding to the virtual motion object; The real motion information is subjected to smoothing filtering, and the first motion information corresponding to each target motion part included in the virtual motion object is determined according to the real motion information subjected to the smoothing filtering.

8. An electronic device, characterized in that: include: a processor, and a memory communicatively coupled to the processor; Memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device implements the motion management method of the virtual motion object according to any one of claims 1 to 7.

Images