Joint motion data adjustment method, device and electronic equipment

Abstract

The present application provides a joint motion data adjustment method, device and electronic device, which perform dynamic correction on the joint motion data based on spring-damper system modeling, thereby achieving smooth denoising of the joint motion data while well simulating real human body movement, so that the adjusted joint motion data has a higher degree of naturalness when driving the virtual model to move.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and electronic equipment for adjusting joint motion data. Background Art

[0002] Some image recognition technologies can identify humans or animals in two-dimensional video images, thereby obtaining three-dimensional joint motion data. For example, a motion recognition model can identify key limbs or joints of a human target in a two-dimensional video image, obtaining three-dimensional motion pose data for these key limbs or joints. This 3D motion pose data can then be used to drive a corresponding virtual human model to move. However, due to quality defects in the original video or errors in the machine learning model itself, the output of the three-dimensional joint motion data may contain data noise, such as sudden changes in the data. Therefore, the output data of the motion recognition model typically requires denoising and smoothing. Common smoothing methods typically perform a weighted average of the results from multiple frames to obtain the final output. However, these smoothing algorithms often only process position and direction, without considering velocity and angular velocity, resulting in a lack of motion information in the output. Furthermore, they fail to integrate velocity and angular velocity with position and direction data, resulting in a less natural output data. Summary of the Invention

[0003] In order to overcome the above-mentioned deficiencies in the prior art, the present application aims to provide a method for adjusting joint motion data, comprising:

[0004] Acquire first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame from the joint motion data sequence of the target joint, wherein the first data frame is a data frame that is located before the second data frame in the time dimension;

[0005] Calculating second deviation data to be adjusted for the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data;

[0006] According to the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, a preset spring damping motion model is input to calculate and obtain second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame;

[0007] The second joint motion data is adjusted according to the second adjusted deviation data.

[0008] In one possible implementation, the step of inputting a preset spring damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, and calculating and obtaining second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame includes:

[0009] Based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, a preset spring damping motion model is input to calculate and obtain second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame under critical spring damping conditions.

[0010] In a possible implementation, the second deviation data to be adjusted includes a second position deviation to be adjusted, a second displacement speed deviation to be adjusted, a second angle deviation to be adjusted, and a second rotation speed deviation to be adjusted;

[0011] The second adjusted deviation data includes a second adjusted position deviation, a second adjusted displacement speed deviation, a second adjusted angle deviation, and a second adjusted rotation speed deviation;

[0012] The second position deviation to be adjusted represents the position deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, and the second displacement speed deviation to be adjusted represents the displacement speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame; the second angle deviation to be adjusted represents the rotation angle deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, and the second rotation speed deviation to be adjusted represents the angular rotation speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame;

[0013] The step of inputting a preset spring damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, and calculating and obtaining second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame under a critical spring damping condition comprises:

[0014] The second adjusted position deviation is calculated according to the following formula The second adjusted displacement velocity deviation The second adjusted angle deviation and the second adjusted rotational speed deviation

[0015]

[0016]

[0017]

[0018]

[0019] fast(z)=(1+z+0.48×z 2 +0.235×z 3 ) -1

[0020] in, is the second position deviation to be adjusted, is the second displacement velocity deviation to be adjusted, is the second angle deviation to be adjusted, is the second rotation speed deviation to be adjusted, d is the damping parameter, Δt is the time difference between the first data frame and the second data frame; fast(z) is the polynomial fitting function of the spring damping motion model.

[0021] In a possible implementation, the first joint motion data includes first joint position data, first joint angle data, first joint movement speed data, and first joint rotation speed data;

[0022] The second joint motion data includes second joint position data, second joint rotation angle data, second joint movement speed data and second joint rotation speed data;

[0023] The first adjusted deviation data includes a first adjusted position deviation, a first adjusted displacement speed deviation, a first adjusted angle deviation, and a first adjusted rotation speed deviation;

[0024] The step of calculating the second deviation data to be adjusted for the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data includes:

[0025] The second position deviation to be adjusted is calculated according to the following formula The second displacement speed deviation to be adjusted The second angle deviation to be adjusted and the second rotation speed deviation to be adjusted

[0026]

[0027]

[0028]

[0029]

[0030] in, is the first joint position data; is the first joint angle data; The moving speed data of the first joint; The first joint rotation speed data;

[0031] is the second joint position data; is the second joint angle data; The second joint movement speed data; is the second joint rotation speed data;

[0032] is the first adjusted position deviation; is the first adjusted displacement velocity deviation; The first adjusted angle deviation; is the first adjusted rotation speed deviation.

[0033] In a possible implementation, the method further includes:

[0034] Acquire a two-dimensional target video image;

[0035] Key joints are identified on the two-dimensional target video image to obtain three-dimensional second joint motion data of at least one of the target joints.

[0036] In a possible implementation, the method further includes:

[0037] The motion model node corresponding to the target joint is driven to move according to the adjusted second joint motion data.

[0038] In a possible implementation, the step of obtaining first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame includes:

[0039] respectively obtaining first joint motion data of a plurality of different target joints, first adjusted deviation data of the first data frame, and second joint motion data of the second data frame;

[0040] The step of driving the motion model node corresponding to the target joint to move according to the adjusted second joint motion data comprises:

[0041] According to the adjusted second joint motion data of a plurality of different target joints, different motion model nodes corresponding to the respective target joints in the same motion model are driven to move respectively.

[0042] Another object of the present application is to provide a joint motion data adjustment device, comprising:

[0043] a data acquisition module, configured to acquire, from a joint motion data sequence of a target joint, first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame, wherein the first data frame is a data frame that precedes the second data frame in a time dimension;

[0044] a deviation calculation module, configured to calculate second deviation data to be adjusted for the second data frame based on the first joint motion data, the first adjusted deviation data, and the second joint motion data;

[0045] a deviation processing module, configured to calculate, based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and a damping parameter corresponding to the target joint, a preset spring damping motion model, and obtain second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame in the spring damping system;

[0046] A data adjustment module is used to adjust the second joint motion data according to the second adjusted deviation data.

[0047] Another object of the present application is to provide an electronic device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the joint motion data adjustment method provided in the present application is implemented.

[0048] Another object of the present application is to provide a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed by one or more processors, they implement the joint motion data adjustment method provided in the present application.

[0049] Compared with the prior art, this application has the following beneficial effects:

[0050] The joint motion data adjustment method, device and electronic device provided in the embodiments of the present application perform dynamic correction on the joint motion data based on spring-damping system modeling, thereby achieving smoothing and denoising of the joint motion data while well simulating real human body movements, so that the adjusted joint motion data has a higher degree of naturalness when driving the virtual model to move. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A schematic flow chart of the steps of the joint motion data adjustment method provided in an embodiment of the present application.

[0053] Figure 2 A schematic diagram of a live broadcast system provided in an embodiment of the present application.

[0054] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present application.

[0055] Figure 4 Schematic diagram of the functional modules of the joint motion data adjustment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without making any creative efforts shall fall within the scope of protection of the present application.

[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not require further definition or explanation in subsequent drawings.

[0059] In the description of this application, it should be noted that the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0060] See Figure 1 , Figure 1 This is a schematic flow chart of the steps of a joint motion data adjustment method provided in this embodiment. The following describes each step of the method in detail.

[0061] Step S110, obtaining the first joint motion data of the first data frame of the target joint, the first adjusted deviation data of the first data frame and the second joint motion data of the second data frame from the joint motion data sequence of the target joint, wherein the first data frame is a data frame that is located before the second data frame in the time dimension.

[0062] In one possible implementation, the joint motion data of the target joint may be three-dimensional joint motion data of the target joint output by a motion recognition model, such as the three-dimensional coordinate position, rotation angle, displacement velocity, rotation velocity, etc. of the target joint. The motion recognition model may output a joint motion data sequence consisting of multiple data frames. In this embodiment, each data frame may be processed sequentially, wherein, for a second data frame to be adjusted, the second data frame and the first data frame preceding the second data frame need to be used together for calculation processing.

[0063] Step S120 , calculating second deviation data to be adjusted of the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data.

[0064] In this embodiment, in order to correct the sudden change or noise that may occur when the joint motion data of the target joint changes from the first data frame to the second data frame, it is necessary to adjust the deviation data between the first data frame and the second data frame. Therefore, it is necessary to first calculate the second deviation data to be adjusted of the second data frame based on the first joint motion data, the first adjusted deviation data and the second joint motion data.

[0065] Step S130, based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, a preset spring damping motion model is input to calculate and obtain second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame.

[0066] In this embodiment, the movement of the target joint is considered to be movement in a spring-damping system, and the second adjusted deviation data when the target joint changes from the first data frame to the second data frame is calculated through a preset spring-damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint.

[0067] Furthermore, considering that under normal circumstances, repeated oscillations will not occur during the movement of human joints, in step S130, a preset spring damping motion model can be input based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameters corresponding to the target joint, to calculate and obtain the second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame under critical damping (that is, there is no oscillation in the system, and it can reach the position indicated by the second data frame from the position indicated by the first data frame at the fastest speed).

[0068] Step S140: adjusting the second joint motion data according to the second adjusted deviation data.

[0069] Since the joint motion data is dynamically corrected based on the spring-damper system modeling, it can achieve smooth denoising of the joint motion data while well simulating real human motion, making the adjusted second joint motion data more natural when driving the virtual model to move.

[0070] Specifically, in this embodiment, the motion state of the target joint in the spring damping system with critical damping can be modeled first. Assume that the target position and target velocity of the target joint in one motion are x g ,v g , the initial position and initial velocity are x 0 ,v 0 , then weighted to get the intermediate result x t As shown in formula (1):

[0071] x t =(1-t·d)·x 0 +t·d· g =x 0 +t·d·x g -x 0 )

[0072] v 0 =d·(x g -x 0 )

[0073] x t =x 0+t·v 0

[0074] Among them, t represents the interval time between two data, d is the damping parameter corresponding to the target key, t·d∈[0,1] represents the weighted weight, and d·(x g -x 0 ) is considered as the speed at which data changes.

[0075] Because the current velocity is only related to the initial position x 0 and the target position x g The data difference is related to the previous speed, which does not take into account the previous speed, resulting in discontinuity in speed. Therefore, the speed compensation term is increased as follows (2):

[0076] x t =x 0 +t·v 0

[0077] v t =v 0 +t·s·(x g -x 0 )+t·d·(v g -v 0 )

[0078] Where s represents the stiffness parameter, and the compensation term t·s·(x g -x 0 ) changes with time to ensure the smoothness of the velocity, and the guiding term t·d·(v g -v 0 ) to make the speed close to the target speed v g , get the intermediate result v of speed t , thus completing the display modeling of the motion state, where the main parameters are the stiffness parameter s and the damping parameter d.

[0079] Then, the above motion state is implicitly modeled, and the modeling formula of the spring damping system is obtained as follows (3):

[0080] x t =j·e -yt ·cos(w·t+p)+c

[0081] v t =-y·j·e -yt ·cos(w·t+p)-w·j·e -yt ·sin(w·t+p)

[0082] Among them, j represents the amplitude, y controls the decay time, t represents the system time, w represents the vibration frequency, p represents the vibration phase, c represents the offset of the entire function, and x t ,vt Represents the position and velocity at time t. The velocity can be obtained by differentiating the position expression.

[0083] According to the target position x g 、Target speed v g , initial position x 0 and initial velocity v 0 , comparing the explicit and implicit modeling formulas, we can get the solution of the above parameters c, y, w as shown in the following equation (4):

[0084]

[0085]

[0086]

[0087] When the system is critically damped, the condition w = 0 is satisfied, that is, there is no oscillation in the system. Based on the critical damping condition and calculated according to the initial state of the system, Equation (3) is finally simplified to obtain the motion formula of the spring damper system under critical damping as follows (5):

[0088] x t =j0·e -yt +t·j1·e -yt +c

[0089] v t =-y·j0·e -yt -y·t·j1·e -yt +j1·e -yt

[0090] Where, parameter c = x g +(d·v g ) / (d 2 / 4), parameter y=d / 2, parameter j0=x 0 -c, parameter j1=v 0 +j0·y, that is, formula (5) can be obtained by parameter x 0 ,v 0 ,d completely represents, which shows that in the critical damping state, the response of the spring damping system is only determined by the damping parameter d. Similarly, the critical damping spring system of direction and angular velocity can also obtain similar results as Equation (5).

[0091] Typically, the joint motion data output by the motion recognition model includes the position and rotation angle of the root joint, and the local rotation angles and local positions of other key joints. Taking the i-th target joint as an example, the first joint position data and the first joint rotation angle data at the t-1th frame (such as the first data frame in this embodiment) are denoted as p i t-1 and qi t-1 , the second joint position data and the second joint rotation angle data at the t-th frame (such as the second data frame in this embodiment) are p t i and q t i .

[0092] From this, the second joint movement speed data of the target joint at the tth frame can be calculated And the second joint rotation speed data Where Δt is the time difference between the previous and next frames, It is expressed in axis angle. Similarly, the first joint movement speed data at frame t-1 can be calculated and the first joint rotation speed data In this embodiment, similar calculations can be performed for each data frame, which will not be described in detail here.

[0093] In step S120, the second joint motion data corresponding to the second data frame to be adjusted may be The first joint motion data corresponding to the first data frame before the second data frame Calculate the second deviation data to be adjusted of the second data frame compared to the first data frame, the second deviation data to be adjusted including the second position deviation to be adjusted, the second displacement speed deviation to be adjusted, the second angle deviation to be adjusted, and the second rotation speed deviation to be adjusted. The second position deviation to be adjusted represents the position deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, the second displacement speed deviation to be adjusted represents the displacement speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame; the second angle deviation to be adjusted represents the rotation angle deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, and the second rotation speed deviation to be adjusted represents the angular rotation speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame.

[0094] Because the system's motion deviation is transmitted over time, the previous motion deviation must also be considered when calculating the deviation. Therefore, in this embodiment, the first adjusted deviation data from the previous frame, which has been adjusted by the spring damping system, is used. The purpose of applying the spring damping system is to gradually reduce the motion deviation to zero deviation, that is, to transition from the motion state of the previous frame to the motion state of the current frame, where the motion deviation of the system in the initial frame is 0, as shown in the following formula (6):

[0095]

[0096]

[0097]

[0098]

[0099] in, is the second position deviation to be adjusted, is the second displacement velocity deviation to be adjusted, is the second angle deviation to be adjusted, is the second rotation speed deviation to be adjusted, It is also expressed in axis angle;

[0100] is the first adjusted position deviation; is the first adjusted displacement velocity deviation; The first adjusted angle deviation; is the first adjusted rotation speed deviation;

[0101] is the first joint position data; is the first joint angle data; The moving speed data of the first joint; The first joint rotation speed data;

[0102] is the second joint position data; is the second joint angle data; The second joint movement speed data; is the second joint rotation speed data.

[0103] Assume that the initial state of the spring-damper system corresponding to the second data frame is position deviation and velocity deviation The target position deviation and target speed deviation are 0, that is, x g =0,v g = 0, time parameter t = Δt, then the parameters c, y, j0, 1 in equation (5) can be updated to c = 0, y = d / 2,

[0104] In this embodiment, a polynomial fitting exponential function is used to improve the calculation performance, that is, e -z =fast(z)=(1+z+0.48×z 2 +0.235×z 3 ) -1 , then formula (5) can be expressed as formula (7),

[0105]

[0106]

[0107]

[0108]

[0109] Thus, in step S130, the second position deviation to be adjusted can be The second displacement speed deviation to be adjusted The second angle deviation to be adjusted The second rotation speed deviation to be adjusted The damping parameter d and the time difference Δt between the first data frame and the second data frame are used to calculate the second adjusted position deviation. The second adjusted displacement velocity deviation The second adjusted angle deviation and the second adjusted rotational speed deviation

[0110] Among them, the second position deviation to be adjusted The second displacement speed deviation to be adjusted The second angle deviation to be adjusted and the second rotation speed deviation to be adjusted In step S120, the first joint motion data (including the first joint position data) may be used to The first joint angle data The first joint movement speed data The first joint rotation speed data ), second joint motion data (including the second joint position data The second joint angle data The second joint movement speed data The second joint rotation speed data ) and the first adjusted deviation data (the first adjusted position deviation The first adjusted displacement velocity deviation The first adjusted angle deviation The first adjusted rotational speed deviation ) calculated.

[0111] It should be noted that the second adjusted deviation data (including the second adjusted position deviation) calculated in step S130 The second adjusted displacement velocity deviation The second adjusted angle deviation and the second adjusted rotational speed deviation ) can be used when executing step S120 for the next frame (such as the t+1th frame) of the second data frame.

[0112] After obtaining the second adjusted deviation data in step S130, the second joint motion data may be adjusted according to the second adjusted deviation data in step S140. For example, the adjusted second joint position data Adjusted second joint movement speed data The second joint angle data after adjustment Adjusted second joint rotation speed data

[0113] In one possible implementation, in step S110, first joint motion data of a plurality of different target joints, first adjusted deviation data of the first data frame, and second joint motion data of the second data frame can be obtained. Then, the processing actions of steps S120 and S130 are performed for each target joint. Finally, in step S140, different motion model nodes corresponding to each target joint in the same motion model can be driven to move based on the adjusted second joint motion data of the plurality of different target joints.

[0114] In one possible implementation, in step S110, a two-dimensional target video image may be acquired, and then key joint recognition may be performed on the two-dimensional target video image to obtain three-dimensional second joint motion data of at least one target joint. For example, a joint motion data sequence of at least one target joint may be calculated based on the position activity data of the key joint in the two-dimensional target video image using a motion recognition model.

[0115] In a possible implementation, the joint motion data adjustment method provided in this embodiment can be applied to the virtual image motion processing in the live broadcast system. For example, please refer to Figure 2The joint motion data adjustment method provided in this embodiment can be run on the server 201 of the live broadcast system. The server 201 can obtain a two-dimensional live video image including a full-body or half-body image of the anchor user from the anchor user terminal 202, then identify the two-dimensional live video image using a motion recognition model to obtain the joint motion data of each target joint, and adjust the joint motion data using the joint motion data adjustment method provided in this embodiment. Then, based on the adjusted joint motion data, the virtual avatar corresponding to the anchor is driven to move, and the live video image including the virtual avatar is sent to the anchor user terminal 202 and / or the audience user terminal 203.

[0116] Based on the same invention concept, please refer to Figure 3 Based on the same inventive concept, this embodiment also provides an electronic device that can have a certain image processing capability. For example, the electronic device can be a personal computer or Figure 2 Server 201 is shown.

[0117] Please refer to Figure 3 , Figure 3 is a block diagram of the electronic device 100. The electronic device 100 includes a joint motion data adjustment device 110, a machine-readable storage medium 120, and a processor 130.

[0118] The machine-readable storage medium 120, the processor 130 and the communication unit 140 are electrically connected to each other directly or indirectly to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other via one or more communication buses or signal lines. The joint motion data adjustment device 110 includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable module stored in the machine-readable storage medium 120, such as the software function module and computer program included in the joint motion data adjustment device 110.

[0119] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0120] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0121] Please refer to Figure 4 This embodiment further provides a joint motion data adjustment device 110, which includes at least one functional module that can be stored in a machine-readable storage medium 120 in software form. Functionally, the joint motion data adjustment device 110 can include a data acquisition module 111, a deviation calculation module 112, a deviation processing module 113, and a data adjustment module 114.

[0122] The data acquisition module 111 is used to obtain the first joint motion data of the first data frame of the target joint, the first adjusted deviation data of the first data frame and the second joint motion data of the second data frame from the joint motion data sequence of the target joint, wherein the first data frame is a data frame that is located before the second data frame in the time dimension.

[0123] In this embodiment, the data acquisition module 111 can be used to perform Figure 1As shown in step S110 , for a detailed description of the data acquisition module 111 , please refer to the description of step S110 .

[0124] The deviation calculation module 112 is configured to calculate second deviation data to be adjusted of the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data.

[0125] In this embodiment, the deviation calculation module 112 can be used to perform Figure 1 As shown in step S120 , for a detailed description of the deviation calculation module 112 , please refer to the description of step S120 .

[0126] The deviation processing module 113 is used to input a preset spring damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, and calculate the second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame.

[0127] In this embodiment, the deviation processing module 113 can be used to perform Figure 1 As shown in step S130 , for a detailed description of the deviation processing module 113 , please refer to the description of step S130 .

[0128] The data adjustment module 114 is configured to adjust the second joint motion data according to the second adjusted deviation data.

[0129] In this embodiment, the data adjustment module 114 can be used to perform Figure 1 As shown in step S140 , for a detailed description of the data adjustment module 114 , please refer to the description of step S140 .

[0130] To sum up, the joint motion data adjustment method, device and electronic device provided in the embodiments of the present application perform dynamic correction on the joint motion data based on spring damping system modeling, thereby achieving smoothing and denoising of the joint motion data while well simulating real human body movements, so that the adjusted joint motion data has a higher degree of naturalness when driving the virtual model to move.

[0131] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0132] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0133] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0134] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0135] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for adjusting joint motion data, characterized in that: include: Acquire first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame from the joint motion data sequence of the target joint, wherein the first data frame is a data frame that is located before the second data frame in the time dimension; Calculating second deviation data to be adjusted for the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data; According to the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, a preset spring damping motion model is input to calculate and obtain second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame; The second joint motion data is adjusted according to the second adjusted deviation data.

2. The method according to claim 1, characterized in that The step of inputting a preset spring damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, and calculating and obtaining second adjusted deviation data when the joint motion data of the target joint in the spring damping system changes from the first data frame to the second data frame includes: Based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, a preset spring damping motion model is input to calculate and obtain second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame under critical spring damping conditions.

3. The method according to claim 2, characterized in that The second deviation data to be adjusted includes a second position deviation to be adjusted, a second displacement speed deviation to be adjusted, a second angle deviation to be adjusted, and a second rotation speed deviation to be adjusted; The second adjusted deviation data includes a second adjusted position deviation, a second adjusted displacement speed deviation, a second adjusted angle deviation, and a second adjusted rotation speed deviation; The second position deviation to be adjusted represents the position deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, and the second displacement speed deviation to be adjusted represents the displacement speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame; the second angle deviation to be adjusted represents the rotation angle deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame, and the second rotation speed deviation to be adjusted represents the angular rotation speed deviation data of the target joint between the position indicated by the first data frame and the position indicated by the second data frame; The step of inputting a preset spring damping motion model based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and the damping parameter corresponding to the target joint, and calculating and obtaining second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame under a critical spring damping condition comprises: The second adjusted position deviation is calculated according to the following formula The second adjusted displacement velocity deviation The second adjusted angle deviation and the second adjusted rotational speed deviation fast(z)=(1+z+0.48×z 2 +0.235×z 3 ) -1 in, is the second position deviation to be adjusted, is the second displacement velocity deviation to be adjusted, is the second angle deviation to be adjusted, is the second rotation speed deviation to be adjusted, d is the damping parameter, Δt is the time difference between the first data frame and the second data frame; fast(z) is the polynomial fitting function of the spring damping motion model.

4. The method according to claim 3, characterized in that The first joint motion data includes first joint position data, first joint rotation angle data, first joint movement speed data and first joint rotation speed data; The second joint motion data includes second joint position data, second joint rotation angle data, second joint movement speed data and second joint rotation speed data; The first adjusted deviation data includes a first adjusted position deviation, a first adjusted displacement speed deviation, a first adjusted angle deviation, and a first adjusted rotation speed deviation; The step of calculating the second deviation data to be adjusted for the second data frame according to the first joint motion data, the first adjusted deviation data, and the second joint motion data includes: The second position deviation to be adjusted is calculated according to the following formula The second displacement speed deviation to be adjusted The second angle deviation to be adjusted and the second rotation speed deviation to be adjusted in, is the first joint position data; is the first joint angle data; The moving speed data of the first joint; The first joint rotation speed data; is the second joint position data; is the second joint angle data; The second joint movement speed data; is the second joint rotation speed data; is the first adjusted position deviation; is the first adjusted displacement velocity deviation; The first adjusted angle deviation; is the first adjusted rotation speed deviation.

5. The method according to claim 1, wherein The method further comprises: Acquire a two-dimensional target video image; Key joints are identified on the two-dimensional target video image to obtain three-dimensional second joint motion data of at least one of the target joints.

6. The method according to claim 1, characterized in that The method further comprises: The motion model node corresponding to the target joint is driven to move according to the adjusted second joint motion data.

7. The method according to claim 6, characterized in that The step of obtaining first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame comprises: respectively obtaining first joint motion data of a plurality of different target joints, first adjusted deviation data of the first data frame, and second joint motion data of the second data frame; The step of driving the motion model node corresponding to the target joint to move according to the adjusted second joint motion data comprises: According to the adjusted second joint motion data of a plurality of different target joints, different motion model nodes corresponding to the respective target joints in the same motion model are driven to move respectively.

8. A joint motion data adjustment device, characterized in that: include: a data acquisition module, configured to acquire, from a joint motion data sequence of a target joint, first joint motion data of a first data frame of the target joint, first adjusted deviation data of the first data frame, and second joint motion data of a second data frame, wherein the first data frame is a data frame that precedes the second data frame in a time dimension; a deviation calculation module, configured to calculate second deviation data to be adjusted for the second data frame based on the first joint motion data, the first adjusted deviation data, and the second joint motion data; a deviation processing module, configured to calculate, based on the time difference between the first data frame and the second data frame, the second deviation data to be adjusted, and a damping parameter corresponding to the target joint, a preset spring damping motion model, and obtain second adjusted deviation data when the joint motion data of the target joint changes from the first data frame to the second data frame in the spring damping system; A data adjustment module is used to adjust the second joint motion data according to the second adjusted deviation data.

9. An electronic device, characterized in that: The method comprises a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images