A repair method, device and storage medium

Abstract

The application relates to a repairing method, device and storage medium. The method comprises the following steps: acquiring first data; the first data comprises motion data of a three-dimensional (3D) human body model; the first data is corrected according to the 3D human body model and a first constraint condition, to obtain first corrected data; the first constraint condition is used for constraining semantic consistency and / or motion consistency in the correction process; the semantic consistency refers to the fact that the action semantics of human body components in the 3D human body model remain consistent; the motion consistency comprises at least one of the following: consistency of a motion amplitude, consistency of a motion direction, consistency of a motion speed or consistency of a motion acceleration. The embodiment of the application can correct the motion data of the 3D human body model, and can maintain the semantic consistency, motion trajectory consistency and time sequence continuity of the original action, so that the repaired 3D human body model is free of mode penetration and has natural, coherent, smooth and stable action.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a repair method, apparatus and storage medium. Background Technology

[0002] The metaverse is a hot topic in current academic and industrial research, and digital human technology is one of its key features. With the advancement of technology, digital humans are increasingly entering application scenarios, including digital idols, virtual social interaction, smart diagnosis and treatment, hosts, and even digital astronauts. The interaction and cognition of digital humans in scenarios require the construction of a digital human with natural movement, rich expressions, and intelligent brain. Among them, appropriate action expression is one of the core elements of whether a digital human is vivid and intelligent. There is a key problem in the movement of digital humans, namely the clipping problem. Clipping refers to the unreasonable interweaving of model skin panels, which greatly affects the visual effect of digital humans and greatly affects the promotion and implementation of digital human products. At present, the methods to fix the clipping problem mainly include: (1) directly adjusting the clipping area manually to restore it to normal. This method requires professional animators and is time-consuming and labor-intensive, with high repair costs; (2) using clipping optimization methods to optimize the clipping and obtain non-clipping results. However, the existing clipping optimization methods have poor optimization effects and it is difficult to obtain smooth non-clipping results. Summary of the Invention

[0003] In view of this, a repair method, apparatus, storage medium and computer program product are proposed.

[0004] In a first aspect, embodiments of this application provide a repair method, the method comprising: acquiring first data; the first data including motion data of a three-dimensional 3D human body model; modifying the first data according to the 3D human body model and a first constraint condition to obtain first modified data; the first constraint condition being used to constrain semantic consistency and / or motion consistency during the modification process; the semantic consistency referring to the consistency of the semantics of the actions constituted by human body parts in the 3D human body model; the motion consistency including at least one of: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration. Based on the above technical solution, the motion data of the 3D human body model is modified according to the 3D human body model and the first constraint condition to obtain the first modified data; the motion data of the 3D human body model can be modified; wherein, the first constraint condition can be used to constrain semantic consistency and / or motion consistency during the modification process, thereby maintaining the semantic consistency, motion trajectory consistency, and temporal continuity of the original actions during the modification process, obtaining semantically preserved and naturally coherent motion data, and making the actions of the modified 3D human body model natural, coherent, smooth, and stable.

[0005] According to the first aspect, in a first possible implementation of the first aspect, the first constraint is used to constrain semantic consistency during the correction process; the step of correcting the first data according to the 3D human body model and the first constraint to obtain first corrected data includes: dividing the 3D human body model into regions to obtain multiple regions; performing collision detection on the multiple regions to determine a first time period and regions where collisions occur within the multiple regions; the first time period is the time period in which collisions occur; within the first time period, correcting the first data according to the skeletal points in the regions where collisions occur and the first constraint to obtain the first corrected data.

[0006] Based on the above technical solution, by constraining the bone points in the collision area of ​​the 3D human body model during the correction process, the geometric semantic information of the original action can be preserved, and the geometric semantic consistency of the action before and after the repair can be maintained.

[0007] According to the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the step of performing collision detection on the plurality of regions to determine the regions where collisions occur in the first time period and among the plurality of regions includes: obtaining bounding boxes corresponding to the plurality of regions; performing collision detection on the bounding boxes corresponding to the plurality of regions to determine the regions where collisions occur in the first time period and among the plurality of regions.

[0008] Based on the above technical solution, by using bounding box design and bounding box collision detection to determine the time period and area of ​​collision, it is possible to quickly determine whether a geometric semantic action has occurred, and to quickly identify action segments and body regions with geometric semantics, thereby shortening the repair time and improving the repair efficiency.

[0009] According to the first or second possible implementation of the first aspect, in the third possible implementation of the first aspect, the first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

[0010] Based on the above technical solution, by constraining the semantic direction between skeletal points within the geometric semantic region during the correction process, the geometric semantic information of the original action can be preserved, and the geometric semantic consistency of the action before and after the repair can be maintained.

[0011] According to the first aspect or various possible implementations of the first aspect described above, in the fourth possible implementation of the first aspect, the first constraint condition is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion amplitude and the consistency of motion direction; the step of correcting the first data according to the 3D human body model and the first constraint condition to obtain the first corrected data includes: obtaining the motion amplitude and motion direction of the skeletal points of the 3D human body model; wherein the motion amplitude and motion direction of the skeletal points are obtained according to the first data; and correcting the first data according to the motion amplitude and motion direction of the skeletal points and the first constraint condition to obtain the first corrected data.

[0012] Based on the above technical solution, by constraining the range and direction of motion of the skeletal points of the 3D human body model during the correction process, the consistency of the motion trajectory before and after the repair can be maintained, making the movements after repair coherent and natural.

[0013] According to the first aspect or various possible implementations of the first aspect described above, in the fifth possible implementation of the first aspect, the first constraint condition is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion velocity and the consistency of motion acceleration; the step of correcting the first data according to the 3D human body model and the first constraint condition to obtain the first corrected data includes: determining a first weight and a second weight according to the motion velocity and motion acceleration of the skeletal points of the 3D human body model; wherein, the motion velocity and motion acceleration of the skeletal points are obtained according to the first data; the first weight is determined according to the motion velocity of the skeletal points in different time periods; the second weight is determined according to the motion acceleration of the skeletal points in different time periods; and the first data is corrected according to the motion velocity and motion acceleration of the skeletal points, the first weight, the second weight, and the first constraint condition to obtain the first corrected data.

[0014] Based on the above technical solution, by constraining the consistency of the motion speed and acceleration of the skeletal points of the 3D human body model during the correction process, the stability and continuity of the motion sequence before and after the repair can be maintained, making the repaired movements smooth and stable.

[0015] According to the first aspect or the various possible implementations of the first aspect described above, in the sixth possible implementation of the first aspect, the first constraint is obtained based on the L2 loss function.

[0016] As an example, the first constraint can be obtained based on the L2 loss function, which is used to constrain the consistency of motion amplitude and motion direction during the correction process. The corrected motion amplitude and motion direction can be obtained when the L2 loss function reaches its minimum value, which can maintain a higher consistency between the motion amplitude and motion direction before and after correction, thereby making the corrected action more coherent and natural.

[0017] As another example, the first constraint can be obtained based on the L2 loss function, which is used to constrain the consistency of motion speed and motion acceleration during the correction process. The corrected motion speed and motion acceleration can be obtained when the L2 loss function reaches its minimum value, which can keep the motion speed and motion acceleration before and after correction more consistent, thereby making the corrected motion smoother and more stable.

[0018] According to the first aspect or the various possible implementations of the first aspect described above, in the seventh possible implementation of the first aspect, the method further includes: correcting the first correction data according to the 3D human body model and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the intersecting skin patches in the 3D human body model during the correction process.

[0019] Based on the above technical solution, it is possible to repair the parts of a 3D human body model that have clipped through, and obtain motion data without clipping through.

[0020] As an example, the first data may include the motion data of the 3D human model during the time period in which clipping occurs. Based on the above technical solution, the first data is corrected according to the 3D human model and the first constraint to obtain the first corrected data; the first corrected data is corrected according to the 3D human model and the second constraint to obtain the second corrected data. The embodiments of this application can repair the motion data of the 3D human model that clips during movement. While repairing the clipping action, the semantic consistency, motion trajectory consistency and temporal continuity of the original action can be maintained, so that the repaired 3D human model has no clipping and the action is natural and coherent.

[0021] According to the seventh possible implementation of the first aspect, in the eighth possible implementation of the first aspect, the step of correcting the first correction data according to the 3D human body model and the second constraint to obtain the second correction data includes: obtaining the intersecting skin patches according to the first correction data; correcting the first correction data according to the intersecting skin patches and the second constraint to obtain the second correction data; wherein the second constraint is used to constrain the distance between the intersecting skin patches and / or the orientation of the intersecting skin patches during the correction process.

[0022] Based on the above technical solution, by constraining the distance and direction of the intersecting skin patches during the correction process, the parts of the 3D human body model that have clipping can be appropriately penalized, thereby repairing the clipping parts and obtaining motion data without clipping.

[0023] Secondly, embodiments of this application provide a repair device, the device comprising: an acquisition module for acquiring first data; the first data including motion data of a three-dimensional 3D human body model; a first correction module for correcting the first data according to the 3D human body model and a first constraint condition to obtain first corrected data; the first constraint condition being used to constrain semantic consistency and / or motion consistency during the correction process; the semantic consistency referring to the consistency of the action semantics of human body parts in the 3D human body model; the motion consistency including at least one of: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration.

[0024] Based on the above technical solution, the motion data of the 3D human body model is corrected according to the 3D human body model and the first constraint condition to obtain the first corrected data; the motion data of the 3D human body model can be corrected; wherein, the first constraint condition can be used to constrain semantic consistency and / or motion consistency during the correction process, thereby maintaining the semantic consistency, motion trajectory consistency and temporal continuity of the original action during the correction process, and obtaining semantically preserved and naturally coherent motion data, so that the action of the corrected 3D human body model is natural, coherent and smooth and stable. According to the second aspect, in the first possible implementation of the second aspect, the first constraint condition is used to constrain the collision area in the 3D human body model during the correction process; the first correction module is further used to: divide the 3D human body model into regions to obtain multiple regions; perform collision detection on the multiple regions to determine a first time period and the regions where collisions occur in the multiple regions; the first time period is the time period in which collisions occur; within the first time period, the first data is corrected according to the bone points in the collision area and the first constraint condition to obtain the first corrected data.

[0025] Based on the above technical solution, by constraining the bone points in the collision area of ​​the 3D human body model during the correction process, the geometric semantic information of the original action can be preserved, and the geometric semantic consistency of the action before and after the repair can be maintained.

[0026] According to the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the first correction module is further configured to: obtain bounding boxes corresponding to the plurality of regions; perform collision detection on the bounding boxes corresponding to the plurality of regions to determine the regions where collisions occur in the first time period and among the plurality of regions.

[0027] Based on the above technical solution, by using bounding box design and bounding box collision detection to determine the time period and area of ​​collision, it is possible to quickly determine whether a geometric semantic action has occurred, and to quickly identify action segments and body regions with geometric semantics, thereby shortening the repair time and improving the repair efficiency.

[0028] According to the first or second possible implementation of the second aspect, in the third possible implementation of the second aspect, the first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

[0029] Based on the above technical solution, by constraining the semantic direction between skeletal points within the geometric semantic region during the correction process, the geometric semantic information of the original action can be preserved, and the geometric semantic consistency of the action before and after the repair can be maintained.

[0030] According to the second aspect or various possible implementations of the second aspect described above, in the fourth possible implementation of the second aspect, the first constraint condition is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion amplitude and the consistency of motion direction; the first correction module is further used to: obtain the motion amplitude and motion direction of the skeletal points of the 3D human body model; wherein the motion amplitude and motion direction of the skeletal points are obtained based on the first data; and correct the first data based on the motion amplitude and motion direction of the skeletal points and the first constraint condition to obtain first corrected data.

[0031] Based on the above technical solution, by constraining the range and direction of motion of the skeletal points of the 3D human body model during the correction process, the consistency of the motion trajectory before and after the repair can be maintained, making the movements after repair coherent and natural.

[0032] According to the second aspect or various possible implementations of the second aspect described above, in the fifth possible implementation of the second aspect, the first constraint condition is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion velocity and the consistency of motion acceleration; the first correction module is further used to: determine a first weight and a second weight based on the motion velocity and motion acceleration of the skeletal points of the 3D human body model; wherein the motion velocity and motion acceleration of the skeletal points are obtained based on the first data; the first weight is determined based on the motion velocity of the skeletal points in different time periods; the second weight is determined based on the motion acceleration of the skeletal points in different time periods; and the first data is corrected based on the motion velocity and motion acceleration of the skeletal points, the first weight, the second weight, and the first constraint condition to obtain first corrected data.

[0033] Based on the above technical solution, by constraining the consistency of the motion speed and acceleration of the skeletal points of the 3D human body model during the correction process, the stability and continuity of the motion sequence before and after the repair can be maintained, making the repaired movements smooth and stable.

[0034] According to the second aspect or the various possible implementations of the second aspect described above, in the sixth possible implementation of the second aspect, the first constraint is obtained based on the L2 loss function.

[0035] As an example, the first constraint can be obtained based on the L2 loss function, which is used to constrain the consistency of motion amplitude and motion direction during the correction process. The corrected motion amplitude and motion direction can be obtained when the L2 loss function reaches its minimum value, which can maintain a higher consistency between the motion amplitude and motion direction before and after correction, thereby making the corrected action more coherent and natural.

[0036] As another example, the first constraint can be obtained based on the L2 loss function, which is used to constrain the consistency of motion speed and motion acceleration during the correction process. The corrected motion speed and motion acceleration can be obtained when the L2 loss function reaches its minimum value, which can keep the motion speed and motion acceleration before and after correction more consistent, thereby making the corrected motion smoother and more stable.

[0037] According to the second aspect or various possible implementations of the second aspect described above, in the seventh possible implementation of the second aspect, the device further includes: a second correction module, used to correct the first correction data according to the 3D human body model and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the intersecting skin patches in the 3D human body model during the correction process.

[0038] Based on the above technical solution, it is possible to repair the parts of a 3D human body model that have clipped through, and obtain motion data without clipping through.

[0039] As an example, the first data may include the motion data of the 3D human model during the time period in which clipping occurs. Based on the above technical solution, the first data is corrected according to the 3D human model and the first constraint to obtain the first corrected data; the first corrected data is corrected according to the 3D human model and the second constraint to obtain the second corrected data. The embodiments of this application can repair the motion data of the 3D human model that clips during movement. While repairing the clipping action, the semantic consistency, motion trajectory consistency and temporal continuity of the original action can be maintained, so that the repaired 3D human model has no clipping and the action is natural and coherent.

[0040] According to the seventh possible implementation of the second aspect, in the eighth possible implementation of the second aspect, the second correction module is further configured to: obtain the intersecting skin patches according to the first correction data; and correct the first correction data according to the intersecting skin patches and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the distance between the intersecting skin patches and / or the orientation of the intersecting skin patches during the correction process.

[0041] Based on the above technical solution, by constraining the distance and direction of the intersecting skin patches during the correction process, the parts of the 3D human body model that have clipping can be appropriately penalized, thereby repairing the clipping parts and obtaining motion data without clipping.

[0042] Thirdly, embodiments of this application provide a repair apparatus, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the first aspect or one or more repair methods of the first aspect when executing the instructions.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the first aspect or one or more of the repair methods of the first aspect.

[0044] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform one or more of the repair methods described in the first aspect.

[0045] For the technical effects of the third to fifth aspects mentioned above, please refer to the first or second aspect mentioned above. Attached Figure Description

[0046] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0047] Figure 1 A flowchart illustrating a manual repair method according to an embodiment of this application is shown.

[0048] Figure 2 A schematic diagram of an artificial repair method according to an embodiment of this application is shown.

[0049] Figure 3 A flowchart illustrating a molding optimization method according to an embodiment of this application is shown.

[0050] Figure 4 A flowchart illustrating another method for optimizing molding according to an embodiment of this application is shown.

[0051] Figure 5 A flowchart illustrating another method for optimizing molding according to an embodiment of this application is shown.

[0052] Figure 6 A schematic diagram of the architecture of a digital human system according to an embodiment of this application is shown.

[0053] Figure 7 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0054] Figure 8 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0055] Figure 9 A flowchart illustrating geometric semantic consistency constraints according to an embodiment of this application is shown.

[0056] Figure 10 A schematic diagram illustrating geometric semantic consistency constraints according to an embodiment of this application is shown.

[0057] Figure 11 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0058] Figure 12 A schematic diagram showing the decomposition of motion trajectory according to an embodiment of this application is shown.

[0059] Figure 13 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0060] Figure 14 A schematic diagram illustrating adaptive weight determination according to an embodiment of this application is shown.

[0061] Figure 15A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0062] Figure 16 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0063] Figure 17 A schematic diagram of a molding constraint process according to an embodiment of this application is shown.

[0064] Figure 18 A schematic diagram of a repair method according to an embodiment of this application is shown.

[0065] Figure 19 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0066] Figure 20 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0067] Figure 21 A flowchart illustrating a repair method according to an embodiment of this application is shown.

[0068] Figure 22 A block diagram of a repair apparatus according to an embodiment of this application is shown.

[0069] Figure 23 A schematic diagram of a repair device according to an embodiment of this application is shown. Detailed Implementation

[0070] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0071] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0072] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0073] In digital human technology, appropriate physical expression is one of the core elements for a digital human to be vivid and intelligent. For example, in conversational digital humans, body language can enhance the rhythm of a speech, making it more vivid and persuasive. Body language plays an important role in communication; firstly, it more accurately expresses intentions and conveys emotions, complementing the information conveyed by speech; secondly, it helps users focus more on the content being communicated with the digital human; thirdly, it enhances the persuasiveness, credibility, and realism of the digital human; and finally, it reflects the speaker's intentions and personality. A lack of body language or stiff body language in communication can lead to the uncanny valley effect. Furthermore, if digital humans move freely in a metaverse, including running, walking, and various other movements, they become more like humans, truly moving. In virtual scenarios such as karaoke bars, concerts, cyberpunk cars, and futuristic cities, diverse themed worlds can be created to empower the value of digital humans, such as live streaming, performance shows, speeches, and press conferences—scenario-based marketing. These scenarios all require digital humans to perform various movements according to different content. Among the related technologies, virtual assistant services such as Artificial Intelligence (AI) customer service, preset talent performances and actions, and intelligent AI atmosphere assistance have been developed to help users act freely in the virtual digital world and realize diverse values ​​through diverse application scenarios.

[0074] The clipping issue that occurs during the movement of digital humans significantly hinders the advancement and deployment of digital human products. The main methods for fixing this problem in related technologies include the following two:

[0075] (1) Manual repair method: This method involves directly adjusting the mold-through area manually to restore it to normal. Figure 1 A flowchart illustrating an embodiment of the manual repair method according to this application is shown, such as... Figure 1 As shown, this method can utilize some repair tools, such as Blender (a 3D graphics software) and Maya (a 3D modeling and animation software), and then manually perform keyframe repair, such as repairing the rotation angle of the corresponding keyframes, and then performing smooth transition of the keyframes. Figure 2 A schematic diagram of an artificial repair method according to an embodiment of this application is shown, such as... Figure 2 As shown, the corresponding keyframes can be manually repaired using software tools. Manual repair is highly controllable and effective, but it requires professionals to use specialized tools and spend a significant amount of time on the repair, resulting in a slow, labor-intensive, and costly process.

[0076] (2) Optimization method for molding: The molding can be optimized by the molding optimization method to obtain non-molding results. Figure 3 A flowchart illustrating a molding optimization method according to an embodiment of this application is shown, as follows: Figure 3 As shown, for a certain frame rotation angle of the clipping part, a 3D mesh network is obtained after linear blending skinning (LBS) operation. The vertex sequence and coordinates of the 3D human model within the skin are obtained. Then, the non-clipping vertex that is closest to the clipping vertex is found. This distance is used as a clipping penalty constraint, and the rotation angle is iteratively optimized. Through the above optimization, a non-clipping result is obtained. However, this method has the following drawbacks: first, it does not consider the temporal discontinuity caused by clipping optimization; second, it lacks semantic constraints and only optimizes the clipping part to the skin contact surface. Figure 4 A flowchart illustrating another method for optimizing patterning according to an embodiment of this application is shown, such as... Figure 4 As shown, the original rotation angle information is passed through a redirection model to obtain the redirected rotation angle information. Then, LBS operations are used to determine the 3D mesh information of the target 3D human body model, defining the clipping probability and the similarity of the rotation angle information before and after redirection. The redirection model is then trained, and based on the converged redirection model, clipping optimization and spatial constraints are performed to obtain a non-clipping result. However, this method has the following drawbacks: first, it does not consider the temporal discontinuity caused by clipping optimization; second, it lacks semantic constraints, which may lead to semantic loss during the optimization process. Figure 5 A flowchart illustrating another method for optimizing patterning according to an embodiment of this application is shown, such as... Figure 5 As shown, the original rotation angle information is passed through a variational autoencoder (VAE) and then subjected to LBS operation to determine the 3D mesh information of the target 3D human body model. Then, clipping optimization, spatial constraints, and contact constraints are performed to finally obtain a non-clipping result. However, this method has the following drawbacks: first, it does not consider the temporal discontinuity caused by clipping optimization; second, it lacks semantic constraints, which can lead to semantic loss during the optimization process.

[0077] To address the aforementioned problems in the related technologies, this application provides a repair method.

[0078] The repair method provided in this application embodiment can be applied to the repair of 3D human models; for example, it can be applied to the repair of digital human models that have been clipped. Figure 6 This diagram illustrates the architecture of a digital human system according to an embodiment of this application, as shown below. Figure 6As shown, the system may include a terminal, a virtual character (i.e., a digital human) brain unit, and a virtual character driving unit. During application, the terminal first receives input from the user. For example, the terminal could be a smart assistant, and the user could ask the smart assistant via voice, "Are there any good movies lately?" After receiving this input, the terminal transmits the voice to the virtual character brain unit. The recognition, perception, and understanding module in the virtual character brain unit can recognize, perceive, and understand the voice, converting it into text and understanding the user's true intention. After understanding the user's question, the virtual character brain unit can provide an answer through the intelligent analysis and decision-making module, such as, "Recently..." The movies currently showing include "XXX", "XX", and "XXXX". I wonder if you like a certain celebrity? I recommend watching their new movie "XXX"! After receiving this response text, it can be input into the character speech generation module and character animation generation module in the virtual character driving unit to generate corresponding speech, digital human face, and digital human animation parameters, including lip shape, facial expressions, body movements, and other animation parameters. The character animation generation module can also perform clipping repair on the generated digital human's movements. Based on the speech and animation parameters obtained in the previous stage, the audio-visual synthesis explicit module synthesizes the final display effect and displays it to the user through the terminal. The repair method provided in this application embodiment can repair clipping actions generated by the digital human during movement in the character animation generation module, thereby generating a clipping-free, semantically preserved, and naturally flowing digital human animation.

[0079] It should be noted that the application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will know that the technical solutions provided by the embodiments of this application are also applicable to similar technical problems in the face of other similar or new scenarios; for example, they can be applied to other 3D model repair scenarios.

[0080] The repair method provided in the embodiments of this application will be described in detail below.

[0081] For example, the repair method provided in the embodiments of this application can be derived from the above. Figure 6 The method can be executed on a terminal device, such as a smartphone, tablet, computer, smart screen, smart glasses, smart vehicle system, robot, in-vehicle mobile device, personal digital assistant, etc. For example, the repair method provided in this application embodiment can also be executed by a cloud server.

[0082] Figure 7 This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 7 As shown, the method may include:

[0083] S701. Obtain first data; the first data includes motion data of a three-dimensional 3D human body model.

[0084] For example, a 3D human body model may include at least one skeletal point, at least one skinned point, relationships between skeletal points, relationships between skinned points, and associations between skeletal points and skinned points. For instance, the motion data of a 3D human body model may be motion data corresponding to skeletal points and / or motion data corresponding to skinned points.

[0085] For example, motion data may include one or more of the following: rotation angle information, displacement information, motion trajectory, motion speed, motion acceleration, etc.

[0086] For example, the first data may be time-series data. For example, the first data may include motion data of the 3D human model during the time period when it experiences clipping.

[0087] S702. Based on the 3D human body model and the first constraint, the first data is corrected to obtain the first corrected data; the first constraint is used to constrain semantic consistency and / or motion consistency during the correction process; the semantic consistency refers to the consistency of the action semantics of the human body parts in the 3D human body model; the motion consistency includes at least one of the following: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration.

[0088] As an example, the first constraint can be used to constrain semantic consistency during the correction process, that is, to ensure that the action semantics of the human body parts in the 3D human body model remain consistent. For example, it can constrain the consistency of the direction of the lines connecting bone points, skin points, or skin patches within the collision area of ​​the 3D human body model. As another example, the first constraint can be used to constrain the consistency of the amplitude and direction of motion of bone points in the 3D human body model during the correction process. As yet another example, the first constraint can be used to constrain the consistency of the velocity and acceleration of motion of bone points in the 3D human body model during the correction process. See below for a detailed description.

[0089] This application embodiment corrects the motion data of the 3D human body model according to the 3D human body model and the first constraint condition to obtain the first corrected data; the motion data of the 3D human body model can be corrected; wherein, the first constraint condition can be used to constrain semantic consistency and / or motion consistency during the correction process, so as to maintain the semantic consistency, motion trajectory consistency and temporal continuity of the original action during the correction process, and obtain semantically preserved and naturally coherent motion data, so that the action of the corrected 3D human body model is natural, coherent and smooth and stable.

[0090] For example, after performing step S702, the following can also be performed: the first correction data is corrected according to the 3D human body model and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the intersecting skin patches in the 3D human body model during the correction process. For example, the intersecting skin patches in the 3D human body model can be obtained using the Bounding Volume Hierarchies (BVH) method.

[0091] As an example, the second constraint can be used to constrain the distance between intersecting skin patches and / or the orientation of the intersecting skin patches during the correction process. See below for a detailed description.

[0092] In one embodiment, the first data may include motion data of the time period during which the 3D human model experiences clipping; the first data can be corrected according to the 3D human model and the first constraint to obtain first corrected data; the first corrected data can be corrected according to the 3D human model and the second constraint to obtain second corrected data; thus, the repair method provided in this application embodiment can repair the motion data of the 3D human model that experiences clipping during movement. While repairing the clipping action, it can maintain the semantic consistency, motion trajectory consistency and temporal continuity of the original action, so that the repaired 3D human model has no clipping and the action is natural and coherent.

[0093] The following is an exemplary description of a possible implementation method for correcting the first data based on the 3D human body model and the first constraint conditions in step S702 above to obtain the first corrected data.

[0094] In one possible implementation, the first constraint is used to constrain semantic consistency during the correction process, that is, to keep the semantics of the actions formed by the human body parts in the 3D human body model consistent. The human body parts can include hands, legs, feet, head, etc. For example, when two hands make a heart-shaped gesture, the first constraint is used to constrain the two hands so that the shape formed by the two hands remains a "heart" shape. Figure 8This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 8 As shown, the following steps may be included:

[0095] S801. Divide the 3D human body model into regions to obtain multiple regions.

[0096] For example, the region can be a body region in a 3D human model, such as the head region, left hand region, right leg region, etc. For example, the 3D human model can be automatically divided into regions using a human body segmentation algorithm. For example, the 3D human model can be manually divided into regions; for example, those skilled in the art can divide the 3D human model into regions according to actual needs. Methods for dividing the 3D human model into regions can refer to related technologies, and this application embodiment does not limit them.

[0097] S802. Perform collision detection on the multiple regions to determine the first time period and the regions where collisions occur within the multiple regions; the first time period is the time period during which collisions occur.

[0098] For example, performing collision detection on the plurality of regions to determine the regions where collisions occur within a first time period and among the plurality of regions may include:

[0099] (1) Obtain the bounding boxes corresponding to the multiple regions.

[0100] Bounding boxes can be designed for the multiple regions obtained in step S801 to obtain bounding boxes corresponding to the multiple regions. The shape of the bounding box can be designed according to actual needs. For example, the bounding box can be square, spherical, or other three-dimensional, and this application does not limit it in this regard.

[0101] (2) Perform collision detection on the bounding boxes corresponding to the multiple regions to determine the regions where collisions occur in the first time period and the multiple regions.

[0102] Bounding box collision detection can be performed during the movement of a 3D human model to determine the time period in which a collision occurs (i.e., the first time period) and the bounding box where the collision occurs. The region corresponding to the bounding box where the collision occurs is the collision region. If a collision is detected, it can be assumed that a geometric semantic action occurred in the collision region, which can be called the geometric semantic region. Other regions outside the geometric semantic region can be called motion semantic regions.

[0103] In this way, by using bounding box design and bounding box collision detection to determine the time period and area of ​​collision, it is possible to quickly determine whether a geometric semantic action has occurred, and to quickly identify the action segments with geometric semantics (i.e., the actions corresponding to the time period of collision) and the body regions (i.e., the areas of collision), thereby shortening the repair time and improving the repair efficiency.

[0104] It should be noted that it is possible to perform collision detection on multiple regions directly without bounding box design, but the computational load and time will increase compared to the bounding box collision detection method.

[0105] S803. During the first time period, the first data is corrected based on the skeletal points in the area where the collision occurred and the first constraint condition to obtain the first corrected data.

[0106] For example, the first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

[0107] As an example, the first constraint could be as follows:

[0108]

[0109] Among them, S m For any geometric semantic region (i.e., the first collision region), S n To be with S m Any geometric semantic region where a collision occurs (i.e., the second collision region), Bone i For S m Any bone point (i.e., the first bone point) in the bone j For S n Any bone point in the bone (i.e., the second bone point), Direction(Bone) i Bone j ) indicates Bone i With Bone j The direction of the connection can be called the semantic direction. Formula (1) indicates that for any two colliding regions in the geometric semantic region, the semantic direction between the skeletal points of one region and the skeletal points of the other region is consistent. The semantic direction between skeletal points in the geometric semantic region can be called geometric semantic information. The process of correcting the first data according to the skeletal points in the colliding regions and the constraints shown in Formula (1) can be called geometric semantic consistency constraint.

[0110] It is understandable that step S803 uses the constraint on the consistency of the connection direction between bone points in the collision area of ​​the 3D human body model as an example for illustration. Similarly, the consistency of the connection direction between skin points or skin patches in the collision area of ​​the 3D human body model can also be constrained, without limitation.

[0111] Figure 9 A flowchart illustrating geometric semantic consistency constraints according to an embodiment of this application is shown, as follows: Figure 9 As shown, a 3D human body model can be divided into regions to obtain multiple regions; bounding boxes can be designed for these regions to obtain corresponding bounding boxes; bounding box collision detection can be performed during the movement of the 3D human body model to determine whether a geometric semantic action has occurred. If a collision is detected, a geometric semantic action can be considered to have occurred, and the time period and region of the geometric semantic action can be determined; within the time period of the geometric semantic action, the first data can be corrected. For any two regions that collide in the geometric semantic region, the direction of the line connecting the bone points of one region and the bone points of the other region is called the semantic direction. Consistency constraints can be applied to the semantic direction to correct the motion data corresponding to the bone points within the geometric semantic region, thus obtaining the first corrected data.

[0112] Figure 10 A schematic diagram illustrating geometric semantic consistency constraints according to an embodiment of this application is shown, as follows: Figure 10 As shown, the 3D human body model is divided into regions and a bounding box is assigned to each region. The shape of the bounding box can be square. Bounding box collision detection can be performed during the movement of the 3D human body model to determine whether a geometric semantic action has occurred. Figure 10 As shown, during the heart-making gesture of the 3D human model, the bounding boxes corresponding to the left and right hand regions collide. This indicates that the 3D human model has generated a geometric semantic action, and the left and right hand regions can be referred to as geometric semantic regions. Consistency constraints can be applied to the semantic directions between the skeletal points in the left and right hand regions, thereby correcting the motion data corresponding to the skeletal points in both regions and obtaining corrected motion data. It can be seen that after the geometric semantic consistency constraint, the semantic directions between the skeletal points in the left and right hand regions remain consistent, meaning that the left and right hand regions form a "heart shape." Thus, this embodiment of the application, by applying consistency constraints to the semantic directions between skeletal points within the geometric semantic regions during the correction process, can retain the original geometric semantic information of the action and maintain the geometric semantic consistency of the action before and after correction.

[0113] In one possible implementation, the first constraint is used to constrain motion consistency during the correction process; motion consistency includes consistency of motion amplitude and consistency of motion direction. Figure 11 This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 11 As shown, the following steps may be included:

[0114] S1101. Obtain the motion amplitude and direction of the skeletal points of the 3D human body model; wherein the motion amplitude and direction of the skeletal points are obtained based on the first data.

[0115] For example, the motion trajectory of the skeletal points of the 3D human body model can be obtained based on the first data, and the motion trajectory of the skeletal points can be decomposed into the motion amplitude and motion direction.

[0116] As an example, a skeleton can include joints, and the motion trajectory of a joint can be decomposed into the amplitude and direction of motion. Figure 12 This diagram illustrates a motion trajectory decomposition according to an embodiment of the present application, as shown below. Figure 12 As shown, the motion trajectory of the leg joints A, B, C, and D of the 3D human body model can be decomposed into motion amplitude and motion direction.

[0117] S1102. Based on the movement amplitude and direction of the skeletal points and the first constraint conditions, the first data is corrected to obtain the first corrected data.

[0118] For example, consistency constraints can be applied to both the amplitude and direction of motion to correct the amplitude and direction of motion of the skeletal points. For example, the first constraint can be obtained based on the L2 loss function. As an example, the first constraint can be as follows:

[0119]

[0120] Among them, Loss tra The additional loss is represented by N, the number of skeleton points is represented by T, and MSE is the mean squared error function. This represents the range of motion of the j-th bone point at time t. This indicates the direction of motion of the j-th bone point at time t. This represents the amplitude of motion of the j-th bone point at time t after correction. This represents the corrected motion direction of the j-th bone point at time t. α and β represent weighting parameters, which can be set by those skilled in the art according to actual needs. Loss can be...tra The corresponding value when the minimum value is obtained or the preset threshold is reached The value and The values ​​of are respectively used as the motion amplitude and direction of the j-th bone point after correction at time t. Thus, by setting the first constraint based on the L2 loss function, the corrected motion amplitude and direction of the bone point are obtained when the L2 loss function reaches its minimum value. This allows for a higher consistency between the motion amplitude and direction before and after correction, resulting in a more coherent and natural movement after repair. The constraint shown in formula (2) can be called the motion amplitude consistency and motion direction consistency constraint. Therefore, by imposing consistency constraints on the motion amplitude and direction of the bone points of the 3D human body model during the correction process, this embodiment of the application can maintain the consistency of the motion trajectory before and after repair, making the repaired movement coherent and natural.

[0121] In one possible implementation, the first constraint is used to constrain motion consistency during the correction process; motion consistency includes consistency of motion velocity and consistency of motion acceleration. Figure 13 This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 13 As shown, the following steps may be included:

[0122] S1301. Determine a first weight and a second weight based on the motion velocity and acceleration of the skeletal points of the 3D human body model; wherein the motion velocity and acceleration of the skeletal points are obtained based on the first data; the first weight is determined based on the motion velocity of the skeletal points in different time periods; and the second weight is determined based on the motion acceleration of the skeletal points in different time periods.

[0123] For example, based on the original motion of the skeletal points of a 3D human model, different weights can be assigned to motion velocity and motion acceleration at different time intervals. As an example, the weights of motion velocity and motion acceleration for a given time interval can be determined using the following formula:

[0124] Wight(x)=(x-min(x)) / (max(x)-min(x)+esp) (3)

[0125] Where x represents motion velocity (e.g., instantaneous motion velocity at a certain moment, or average motion velocity over a certain period of time), then Wight(x) represents the weight of motion velocity over that period of time, min(x) represents the minimum value of motion velocity over that period of time, and max(x) represents the maximum value of motion velocity over that period of time; if x represents motion acceleration (e.g., instantaneous motion acceleration at a certain moment, or average motion acceleration over a certain period of time), then Wight(x) represents the weight of motion acceleration over that period of time, min(x) represents the minimum value of motion acceleration over that period of time, and max(x) represents the maximum value of motion acceleration over that period of time; esp is a hyperparameter that can be set according to actual needs, for example, esp can be set to 0.1. The weights of motion velocity and motion acceleration of the skeletal points in different time periods can be adaptively calculated according to formula (3). Figure 14 This diagram illustrates an adaptive weight determination method according to an embodiment of the present application, as shown below. Figure 14 As shown, the weights of movement speeds in different time periods can be adaptively determined based on the original movement of the skeletal points at different time intervals. For example, in the T1-T2 time interval, the movement amplitude is small and the movement speed is slow; in the T4-T5 time interval, the movement amplitude is large and the movement speed is fast. A larger weight can be assigned to the movement speed in the T1-T2 time interval, and a smaller weight to the movement speed in the T4-T5 time interval. For example, the weight of the movement speed in the T1-T2 time interval could be 1, and the weight of the movement speed in the T4-T5 time interval could be 0. This makes the movement smoother and more stable.

[0126] S1302. Based on the motion velocity and acceleration of the skeletal points, the first weight, the second weight, and the first constraint, the first data is corrected to obtain the first corrected data.

[0127] For example, consistency constraints can be applied to motion velocity and acceleration to correct for the motion velocity and acceleration of bone points. For example, the first constraint can be obtained based on the L2 loss function. As an example, the first constraint can be as follows:

[0128]

[0129] Among them, Loss v&a This represents the loss of motion velocity and acceleration, where N represents the number of skeletal points, T represents time, and V represents the loss of motion velocity and acceleration. t j This represents the corrected velocity of the j-th bone point at time t. Let w1 represent the corrected motion acceleration of the j-th bone point at time t, w2 represent the weight of the motion velocity during the time interval 0-T (i.e., the first weight), and w2 represent the weight of the motion acceleration during the time interval 0-T (i.e., the second weight). The formulas for calculating w1 and w2 are as follows:

[0130] w1 = Wight(V ori (5)

[0131] w2 = Wight(Acc) ori (6)

[0132] Among them, V ori This represents the velocity (V) of the motion within the time interval 0-T in the original motion data (i.e., the first data). ori It can be the instantaneous velocity at a certain moment within the 0-T time interval, or it can be the average velocity within the 0-T time interval. ori This represents the acceleration (Acc) during the time interval 0-T in the original motion data (i.e., the first data). ori It can be the instantaneous acceleration at a certain moment within the 0-T time interval, or it can be the average acceleration within the 0-T time interval. V ori Substituting into formula (3) to calculate w1, Acc ori Substituting into formula (3), we obtain w2. The constraint conditions shown in formula (4) can be called motion velocity consistency and motion acceleration consistency constraints. Loss can be... v&a The V corresponding to obtaining the minimum value or reaching the preset threshold t j The value and The values ​​are respectively used as the motion velocity and acceleration of the j-th bone point at time t after correction. Thus, by setting the first constraint based on the L2 loss function, the corrected motion velocity and acceleration of the bone point are obtained when the L2 loss function reaches its minimum value. This ensures higher consistency between the motion velocity and acceleration before and after correction, resulting in smoother and more stable movements after correction. This embodiment of the application, by constraining the consistency of the motion velocity and acceleration of the bone points of the 3D human model during the correction process, can maintain the stability and continuity of the motion sequence before and after correction, making the corrected movements smooth and stable.

[0133] Motion amplitude, direction, velocity, and acceleration can be referred to as motion semantic information, and consistency constraints for motion amplitude, direction, velocity, and acceleration can be referred to as motion semantic consistency constraints. In one embodiment, motion semantic consistency constraints can be applied to the motion semantic region and / or the geometric semantic region after geometric semantic consistency constraints, thereby preserving the original motion semantic information of the action and maintaining the motion semantic consistency of the action before and after the repair.

[0134] In one possible implementation, the first constraint includes a first sub-constraint and a second sub-constraint; the first sub-constraint is used to constrain the areas in the 3D human body model where collisions occur during the correction process; and the second sub-constraint is used to constrain motion consistency during the correction process. Figure 15 This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 15 As shown, the following steps may be included:

[0135] S1501. Based on the 3D human body model and the first sub-constraint, the first data is corrected to obtain the first intermediate corrected data.

[0136] This step can be referred to above. Figure 8 Steps S801 to S803 are not described in detail here.

[0137] S1502. Based on the 3D human body model and the second sub-constraint, the first intermediate correction data is corrected to obtain the first correction data.

[0138] This step can be referred to above. Figure 11 In steps S1101 to S1102 and / or the above Figure 13 The steps S1301 to S1302 are not described in detail here.

[0139] Thus, by constraining the semantic direction between skeletal points within the geometric semantic region during the correction process, the embodiments of this application can retain the geometric semantic information of the original action and maintain the geometric semantic consistency of the action before and after the repair. By constraining the motion amplitude, motion direction, motion speed and motion acceleration of skeletal points during the correction process, the motion semantic information of the original action can be retained, and the consistency of the motion trajectory, the stability and continuity of the motion sequence before and after the repair can be maintained, making the repaired action coherent, natural, smooth and stable.

[0140] The following is an exemplary description of a possible implementation method for correcting the first corrected data based on the 3D human body model and the second constraint to obtain the second corrected data.

[0141] Figure 16 This diagram illustrates a flowchart of a repair method according to an embodiment of this application; exemplarily, the method can be derived from the above... Figure 6 Executed on a terminal or cloud server; such as Figure 16 As shown, the following steps may be included:

[0142] S1601. Obtain the intersecting skin surface according to the first correction data.

[0143] For example, the BVH algorithm can be used to obtain all intersecting skin patches in a 3D human body model. For example, the obtained intersecting skin patches can be filtered to extract skin patches of interest; for example, intersecting skin patches within the collision region can be filtered out. The filtering process can be implemented with reference to relevant technologies.

[0144] S1602. Based on the intersecting skin panels and the second constraint condition, the first correction data is corrected to obtain the second correction data; wherein, the second constraint condition is used to constrain the distance between the intersecting skin panels and / or the orientation of the intersecting skin panels during the correction process.

[0145] For example, a specific distance between the intersecting skin patches can be corrected according to a second constraint, which can be as follows:

[0146] dis(v t ) = n fs (v t -o fs )r (7)

[0147] Among them, dis(v) t ) represents the distance of the intrusion into the skin panel, n fs v represents the normal vector of the invaded skin patch. t Indicates the vertex position of the intrusion, o fs The circle represents the center of the invaded skin patch, and r represents the penalty coefficient, which can be set according to actual needs.

[0148] For example, the orientation of the intersecting skin patches can be corrected according to a second constraint, which can be as follows:

[0149]

[0150] Among them, E collisionV represents the clipping loss. t Indicates the vertex position of the intrusion, n s Let dis(v) represent the normal vector of the invaded skin patch, ‖·‖ represent the norm, and dis(v) represent the normal vector of the invaded skin patch. t It can be calculated according to formula (7).

[0151] The process of constraining the intersecting skin panels according to the constraints shown in formulas (7) and (8) can be called the mold-through constraint process. Figure 17 This diagram illustrates a molding constraint process according to an embodiment of the present application, as shown below. Figure 17 As shown, the BVH algorithm can be used to obtain all intersecting skin patches in a 3D human body model. Then, the skin patches of interest can be filtered out, and distance and orientation optimization can be performed on these patches to complete the clipping constraint process. Thus, by constraining the distance and orientation of intersecting skin patches during the correction process, this embodiment of the application can appropriately penalize the clipping parts in the 3D human body model, thereby repairing the clipping.

[0152] Figure 18 A schematic diagram of a repair method according to an embodiment of this application is shown, as follows: Figure 18As shown, the system can input motion data of a 3D human model with clipping issues, such as temporal rotation angle data. LBS operations can be used to determine the 3D mesh information of the 3D human model, and the skin position of the current model can be obtained. The input motion data is then corrected through the geometric semantic consistency module, motion semantic consistency module, and clipping constraint module, resulting in clipping-free, semantically preserved, and naturally coherent motion data. Specifically, the geometric semantic consistency module includes a human semantic partitioning submodule, a 3D geometric bounding box module, a geometric overlap detection submodule, and a relative position deviation constraint submodule. The human semantic partitioning submodule is used to divide the 3D human model into multiple regions; the 3D geometric bounding box module can be used to assign bounding boxes to different regions; the geometric overlap detection submodule can be used to detect collisions (i.e., overlaps) of bounding boxes, thereby determining whether geometric semantic actions have occurred during motion and identifying the time period and region of these actions; the relative position deviation constraint submodule can be used to design relative position deviation constraints, such as constraining the semantic direction between bone points within a geometric semantic region; the motion semantic consistency module includes a joint motion amplitude and direction decomposition submodule. The system includes submodules for amplitude and direction-related trajectory constraints, dynamic weighting, and zero-speed / zero-acceleration constraints. The joint motion amplitude and direction decomposition submodule decomposes the motion trajectory of a joint into motion amplitude and direction. The amplitude and direction-related trajectory constraint submodule constrains the decomposed motion amplitude and direction. The dynamic weighting submodule decomposes the original motion velocity and acceleration, calculates the velocity and acceleration at different time intervals, and obtains the weights of velocity and acceleration at different time intervals. The zero-speed / zero-acceleration constraint submodule constrains velocity and acceleration, for example, by applying L2 constraints and incorporating the weights of velocity and acceleration. The clipping constraint module appropriately penalizes clipping. Thus, the repair method provided in this application embodiment, by constraining the geometric semantic consistency and motion semantic consistency of the action during the correction process, and constraining the through-mold part, can retain the geometric semantic information and motion semantic information of the original action while repairing the through-mold action, maintain the geometric semantic consistency and motion semantic consistency of the action before and after the repair, maintain the continuity of the motion sequence, and make the repaired action coherent, natural, smooth and stable.

[0153] Figure 19 A flowchart illustrating a repair method according to an embodiment of this application is shown, as follows: Figure 19As shown, the system can input motion data corresponding to the source skeleton (i.e., the skeleton composed of bone points) and / or motion data corresponding to the skin of a 3D human model. The motion data includes data on the time periods when clipping occurs. The 3D human model can be divided into regions and bounding boxes can be designed. It is important to note that the bounding box designed based on the motion data of the source skeleton and the bounding box designed based on the motion data of the skin are different in size. The bounding box designed based on the motion data of the skin needs to encompass the skin portion. Bounding box collision detection can be performed to determine the geometric semantic region and the motion semantic region. Geometric semantic consistency constraints are applied within the geometric semantic region, and then motion semantic consistency constraints are applied within the motion semantic region and within the geometric semantic region after geometric semantic consistency constraints. Clipping constraints are then applied, ultimately outputting motion data that is clipping-free, semantically preserved, and exhibits natural and coherent motion. This embodiment of the application, by constraining the geometric semantic consistency and motion semantic consistency of the action during the correction process and constraining the clipping portion, can retain the original geometric semantic information and motion semantic information of the action while repairing clipping actions, maintaining the continuity of the motion sequence, resulting in a coherent, natural, smooth, and stable action after repair.

[0154] Figure 20 A flowchart illustrating a repair method according to an embodiment of this application is shown, as follows: Figure 20 As shown, the system can input motion data corresponding to the source skeleton of a 3D human model and / or motion data corresponding to the skin. The motion data includes data on the time periods when clipping occurs. The 3D human model can be divided into regions and bounding boxes can be designed. Bounding box collision detection can be performed to determine the geometric semantic region. Geometric semantic consistency constraints are then applied within the geometric semantic region, followed by clipping constraints. Finally, motion data without clipping and with preserved geometric semantics can be output. This embodiment of the application, by constraining the geometric semantic consistency of the action during the correction process and constraining the clipping portion, can retain the geometric semantic information of the original action while repairing clipping actions, maintaining the geometric semantic consistency of the action before and after repair.

[0155] Figure 21 A flowchart illustrating a repair method according to an embodiment of this application is shown, as follows: Figure 21 As shown, the user can input motion data corresponding to the source skeleton of the 3D human model and / or motion data corresponding to the skin. The motion data includes data on the time period when clipping occurs. Motion semantic consistency constraints can be applied to the 3D human model, followed by clipping constraints. Finally, motion data without clipping and with natural and coherent motion can be output. This embodiment of the application, by constraining the motion semantic consistency of actions and constraining the clipping portion during the correction process, can retain the original motion semantic information of the actions while repairing clipping actions, maintaining the continuity of motion sequence, and making the repaired actions coherent, natural, smooth, and stable.

[0156] Based on the same inventive concept as the above method embodiments, this application also provides a repair device, which can be used to execute the technical solutions described in the above method embodiments. For example, it can execute the above... Figure 7 , Figure 8 , Figure 11 , Figure 13 , Figure 15 or Figure 16 The steps of the repair method shown are illustrated.

[0157] Figure 22 A block diagram of a repair apparatus according to an embodiment of this application is shown, such as Figure 22 As shown, the device may include: an acquisition module 2201, used to acquire first data; the first data includes motion data of a three-dimensional 3D human body model; a first correction module 2202, used to correct the first data according to the 3D human body model and a first constraint condition to obtain first corrected data; the first constraint condition is used to constrain geometric semantic consistency and / or motion consistency during the correction process; the semantic consistency refers to the consistency of the action semantics of the human body parts in the 3D human body model; the motion consistency includes at least one of the following: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration.

[0158] This application embodiment corrects the motion data of the 3D human body model according to the 3D human body model and the first constraint condition to obtain the first corrected data; the motion data of the 3D human body model can be corrected; wherein, the first constraint condition can be used to constrain semantic consistency and / or motion consistency during the correction process, so as to maintain the semantic consistency, motion trajectory consistency and temporal continuity of the original action during the correction process, and obtain semantically preserved and naturally coherent motion data, so that the action of the corrected 3D human body model is natural, coherent and smooth and stable.

[0159] In one possible implementation, the first constraint is used to constrain semantic consistency during the correction process; the first correction module 2202 is further used to: divide the 3D human body model into regions to obtain multiple regions; perform collision detection on the multiple regions to determine the regions where collisions occur in a first time period and among the multiple regions; the first time period is the time period in which collisions occur; within the first time period, the first data is corrected according to the skeletal points in the regions where collisions occur and the first constraint to obtain the first corrected data.

[0160] In one possible implementation, the first correction module 2202 is further configured to: obtain bounding boxes corresponding to the plurality of regions; perform collision detection on the bounding boxes corresponding to the plurality of regions to determine the regions where collisions occur in the first time period and among the plurality of regions.

[0161] In one possible implementation, the first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

[0162] In one possible implementation, the first constraint is used to constrain motion consistency during the correction process; the motion consistency includes consistency of motion amplitude and consistency of motion direction; the first correction module 2202 is further used to: obtain the motion amplitude and motion direction of the skeletal points of the 3D human body model; wherein the motion amplitude and motion direction of the skeletal points are obtained based on the first data; and correct the first data based on the motion amplitude and motion direction of the skeletal points and the first constraint to obtain first corrected data.

[0163] In one possible implementation, the first constraint is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion velocity and the consistency of motion acceleration; the first correction module 2202 is further used to: determine a first weight and a second weight based on the motion velocity and motion acceleration of the skeletal points of the 3D human body model; wherein the motion velocity and motion acceleration of the skeletal points are obtained based on the first data; the first weight is determined based on the motion velocity of the skeletal points in different time periods; the second weight is determined based on the motion acceleration of the skeletal points in different time periods; and the first data is corrected based on the motion velocity and motion acceleration of the skeletal points, the first weight, the second weight, and the first constraint to obtain first corrected data.

[0164] In one possible implementation, the first constraint is obtained based on the L2 loss function.

[0165] In one possible implementation, the device further includes: a second correction module, configured to correct the first correction data according to the 3D human body model and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the intersecting skin patches in the 3D human body model during the correction process.

[0166] In one possible implementation, the second correction module is further configured to: obtain the intersecting skin patches according to the first correction data; and correct the first correction data according to the intersecting skin patches and the second constraint condition to obtain second correction data; wherein the second constraint condition is used to constrain the distance between the intersecting skin patches and / or the orientation of the intersecting skin patches during the correction process.

[0167] The above Figure 22 The technical effects and specific descriptions of the repair device and its various possible implementations can be found in the above-mentioned repair methods, and will not be repeated here.

[0168] It should be understood that the division of modules in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the modules in the device can be implemented by a processor calling software; for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each module in the device. The processor can be, for example, a general-purpose processor, such as a Central Processing Unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the modules in the device can be implemented as hardware circuits. The functionality of some or all modules can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the modules is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between these logic gates are configured through configuration files, thereby achieving the functionality of some or all of the modules. All modules of the above device can be implemented entirely through processor-invoked software, entirely through hardware circuits, or partially through processor-invoked software with the remaining parts implemented through hardware circuits.

[0169] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a CPU, microprocessor, graphics processing unit (GPU), digital signal processor (DSP), neural network processing unit (NPU), tensor processing unit (TPU), etc. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.

[0170] As can be seen, each module in the above apparatus can be one or more processors (or processing circuits) configured to implement the methods of the above embodiments, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types. Furthermore, each module in the above apparatus can be integrated in whole or in part, or can be implemented independently; there is no limitation on this.

[0171] Embodiments of this application also provide a repair apparatus, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of the above embodiments when executing the instructions. Exemplarily, the above can be performed. Figure 7 , Figure 8 , Figure 11 , Figure 13 , Figure 15 or Figure 16 The steps of the repair method shown are illustrated.

[0172] Figure 23 This diagram illustrates the structure of a repair device according to an embodiment of the present application, as shown below. Figure 23 As shown, the repair device may include at least one processor 2301, a communication line 2302, a memory 2303, and at least one communication interface 2304.

[0173] Processor 2301 may be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits for controlling the execution of programs according to the present application. Processor 2301 may also include a heterogeneous computing architecture of multiple general-purpose processors, for example, it may be a combination of at least two of CPU, GPU, microprocessor, DSP, ASIC, and FPGA. As an example, processor 2301 may be CPU+GPU, CPU+ASIC, or CPU+FPGA.

[0174] Communication line 2302 may include a path for transmitting information between the aforementioned components.

[0175] The communication interface 2304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0176] The memory 2303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via communication line 2302. The memory may also be integrated with the processor. The memory provided in this application embodiment is generally non-volatile. The memory 2303 is used to store computer execution instructions for executing the scheme of this application and is controlled by the processor 2301 for execution. The processor 2301 is used to execute computer execution instructions stored in the memory 2303, thereby implementing the method provided in the above embodiments of this application; exemplarily, the above can be implemented. Figure 7 , Figure 8 , Figure 11 , Figure 13 , Figure 15 or Figure 16 The steps of the repair method shown are illustrated.

[0177] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0178] For example, processor 2301 may include one or more CPUs, e.g. Figure 23 CPU0 in the CPU; processor 2301 may also include a CPU, and any one of GPU, ASIC, or FPGA, for example, Figure 23 The CPU0+GPU0, CPU0+ASIC0, or CPU0+FPGA0 are mentioned.

[0179] For example, the repair device may include multiple processors, such as Figure 23 Processors 2301 and 2307 are mentioned in the text. Each of these processors can be a single-core processor, a multi-core processor, or a heterogeneous computing architecture that includes multiple general-purpose processors. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0180] In a specific implementation, as one embodiment, the repair device may further include an output device 2305 and an input device 2306. The output device 2305 communicates with the processor 2301 and can display information in various ways. For example, the output device 2305 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, such as a vehicle-mounted HUD, AR-HUD, or monitor. The input device 2306 communicates with the processor 2301 and can receive user input in various ways. For example, the input device 2306 may be a mouse, keyboard, touchscreen device, or sensing device.

[0181] Embodiments of this application provide a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the methods described in the above embodiments. Exemplarily, the above can be implemented... Figure 7 , Figure 8 , Figure 11 , Figure 13 , Figure 15 or Figure 16 The steps of the repair method shown are illustrated.

[0182] Embodiments of this application provide a computer program product, which may include, for example, computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code; when the computer program product is run on a computer, the computer performs the methods described in the above embodiments. Exemplarily, the above... Figure 7 , Figure 8 , Figure 11 , Figure 13 , Figure 15 or Figure 16 The steps of the repair method shown are illustrated.

[0183] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0184] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0185] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0186] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0187] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0188] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0189] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0190] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A repair method, characterized in that, The method includes: Acquire first data; the first data includes motion data of a three-dimensional 3D human body model; Based on the 3D human body model and the first constraint, the first data is corrected to obtain the first corrected data; the first constraint is used to constrain semantic consistency during the correction process, or to constrain the semantic consistency and motion consistency; the semantic consistency refers to the consistency of the action semantics of the human body parts in the 3D human body model; the motion consistency includes at least one of the following: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration.

2. The method according to claim 1, characterized in that, The first constraint is used to constrain semantic consistency during the correction process; The step of correcting the first data based on the 3D human body model and the first constraint to obtain the first corrected data includes: The 3D human body model is divided into regions to obtain multiple regions; Collision detection is performed on the multiple regions to determine the regions where collisions occur within a first time period; the first time period is the time period during which collisions occur. During the first time period, the first data is corrected based on the skeletal points in the area where the collision occurred and the first constraint condition to obtain the first corrected data.

3. The method according to claim 2, characterized in that, The step of performing collision detection on the multiple regions to determine the regions where collisions occurred within the first time period and among the multiple regions includes: Obtain the bounding boxes corresponding to the multiple regions; Collision detection is performed on the bounding boxes corresponding to the multiple regions to determine the regions where collisions occur in the first time period and among the multiple regions.

4. The method according to claim 2 or 3, characterized in that, The first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

5. The method according to any one of claims 1-4, characterized in that, The first constraint is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion amplitude and the consistency of motion direction. The step of correcting the first data based on the 3D human body model and the first constraint to obtain the first corrected data includes: The amplitude and direction of motion of the skeletal points of the 3D human body model are obtained; wherein the amplitude and direction of motion of the skeletal points are obtained based on the first data; Based on the range and direction of motion of the skeletal points and the first constraint condition, the first data is corrected to obtain the first corrected data.

6. The method according to any one of claims 1-5, characterized in that, The first constraint is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion velocity and the consistency of motion acceleration. The step of correcting the first data based on the 3D human body model and the first constraint to obtain the first corrected data includes: Based on the motion velocity and acceleration of the skeletal points of the 3D human body model, a first weight and a second weight are determined; wherein, the motion velocity and acceleration of the skeletal points are obtained from the first data; the first weight is determined based on the motion velocity of the skeletal points in different time periods; and the second weight is determined based on the motion acceleration of the skeletal points in different time periods. Based on the motion velocity and acceleration of the skeletal points, the first weight, the second weight, and the first constraint, the first data is corrected to obtain the first corrected data.

7. The method according to any one of claims 1-6, characterized in that, The first constraint is obtained based on the L2 loss function.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Based on the 3D human body model and the second constraint, the first correction data is corrected to obtain the second correction data; wherein, the second constraint is used to constrain the intersecting skin patches in the 3D human body model during the correction process.

9. The method according to claim 8, characterized in that, The step of correcting the first correction data according to the 3D human body model and the second constraint to obtain the second correction data includes: Based on the first corrected data, obtain the intersecting skin surface; Based on the intersecting skin panels and the second constraint condition, the first correction data is corrected to obtain the second correction data; wherein, the second constraint condition is used to constrain the distance between the intersecting skin panels and / or the orientation of the intersecting skin panels during the correction process.

10. A repair device, characterized in that, The device includes: The acquisition module is used to acquire first data; the first data includes motion data of a three-dimensional 3D human body model. The first correction module is used to correct the first data according to the 3D human body model and the first constraint condition to obtain the first corrected data; the first constraint condition is used to constrain semantic consistency during the correction process, or to constrain the semantic consistency and motion consistency; the semantic consistency refers to the consistency of the action semantics of the human body parts in the 3D human body model; the motion consistency includes at least one of the following: consistency of motion amplitude, consistency of motion direction, consistency of motion speed, or consistency of motion acceleration.

11. The apparatus according to claim 10, characterized in that, The first constraint is used to constrain semantic consistency during the correction process; The first correction module is further configured to: The 3D human body model is divided into regions to obtain multiple regions; Collision detection is performed on the multiple regions to determine the regions where collisions occur within a first time period; the first time period is the time period during which collisions occur. During the first time period, the first data is corrected based on the skeletal points in the area where the collision occurred and the first constraint condition to obtain the first corrected data.

12. The apparatus according to claim 11, characterized in that, The first correction module is further configured to: Obtain the bounding boxes corresponding to the multiple regions; Collision detection is performed on the bounding boxes corresponding to the multiple regions to determine the regions where collisions occur in the first time period and among the multiple regions.

13. The apparatus according to claim 11 or 12, characterized in that, The first constraint condition is that the line connecting the first bone point and the second bone point has the same direction; wherein, the first bone point is any bone point in the first collision region; the second bone point is any bone point in the second collision region; the first collision region is any region in the plurality of regions where a collision occurs, and the second collision region is any region in the plurality of regions that collides with the first collision region.

14. The apparatus according to any one of claims 10-13, characterized in that, The first constraint is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion amplitude and the consistency of motion direction. The first correction module is further configured to: The amplitude and direction of motion of the skeletal points of the 3D human body model are obtained; wherein the amplitude and direction of motion of the skeletal points are obtained based on the first data; Based on the range and direction of motion of the skeletal points and the first constraint condition, the first data is corrected to obtain the first corrected data.

15. The apparatus according to any one of claims 10-14, characterized in that, The first constraint is used to constrain motion consistency during the correction process; the motion consistency includes the consistency of motion velocity and the consistency of motion acceleration. The first correction module is further configured to: Based on the motion velocity and acceleration of the skeletal points of the 3D human body model, a first weight and a second weight are determined; wherein, the motion velocity and acceleration of the skeletal points are obtained from the first data; the first weight is determined based on the motion velocity of the skeletal points in different time periods; and the second weight is determined based on the motion acceleration of the skeletal points in different time periods. Based on the motion velocity and acceleration of the skeletal points, the first weight, the second weight, and the first constraint, the first data is corrected to obtain the first corrected data.

16. The apparatus according to any one of claims 10-15, characterized in that, The first constraint is obtained based on the L2 loss function.

17. The apparatus according to any one of claims 10-16, characterized in that, The device further includes: The second correction module is used to correct the first correction data according to the 3D human body model and the second constraint condition to obtain the second correction data; wherein, the second constraint condition is used to constrain the intersecting skin patches in the 3D human body model during the correction process.

18. The apparatus according to claim 17, characterized in that, The second correction module is also used for: Based on the first corrected data, obtain the intersecting skin surface; Based on the intersecting skin panels and the second constraint condition, the first correction data is corrected to obtain the second correction data; wherein, the second constraint condition is used to constrain the distance between the intersecting skin panels and / or the orientation of the intersecting skin panels during the correction process.

19. A repair device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method described in any one of claims 1-9 when executing the instructions.

20. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-9.

21. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method described in any one of claims 1-9.

Images