Motion data processing method and apparatus, product, device and medium

Abstract

Provided are a motion data processing method and apparatus, a product, a device, and a medium. The motion data processing method comprises: each of N motion points of a second object model corresponding to one or more basic skeletons among M basic skeletons; performing conversion processing on first reference positions of the N motion points and second reference positions of the M skeletons to generate global prediction features, wherein the global prediction features are used for reflecting global position features of the N motion points and the corresponding basic skeletons, position features of the M basic skeletons and structural features of a bone chain where the basic skeletons corresponding to the motion points are located; and predicting motion binding parameters of each motion point and a corresponding basic skeleton on the basis of the global prediction features, wherein any one motion point moves along with a corresponding basic skeleton on the basis of the motion binding parameters of the motion point and the corresponding basic skeleton. The efficiency and accuracy of acquiring the motion binding parameters of each motion point and the corresponding basic skeleton can be improved.

Description

Motion data processing method, device, product, equipment and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 26, 2023, with application number 2023107634825 and application name “Motion data processing method, device, product, equipment and medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a motion data processing method, apparatus, product, equipment, and medium. Background Art

[0003] Skinning generally refers to establishing a binding relationship between clothing movement points and the character's skeleton, so that subsequent clothing can follow the character's corresponding movements based on the binding relationship.

[0004] In existing applications, artists typically establish binding relationships between clothing movement points and the character's skeleton based on their experience with clothing following human motion. However, clothing movement points are often numerous, requiring artists to spend considerable time establishing these binding relationships. This results in low efficiency. Furthermore, the varying skill levels of different artists can make the accuracy of these established binding relationships uncertain.

[0005] Summary of the Invention

[0006] The present application provides a motion data processing method, apparatus, product, device and medium, which can improve the efficiency and accuracy of obtaining motion binding parameters between motion points and corresponding basic bones.

[0007] On one hand, the present application provides a motion data processing method, the method comprising:

[0008] Acquire a first object model, where the first object model includes M basic bones or a portion of the M basic bones, where M is a natural number;

[0009] Obtain a second object model, where the second object model includes N motion points, wherein each of the N motion points corresponds to one or more basic bones of the M basic bones, and the N motion points are used to define a motion of the second object model, where N is a natural number;

[0010] Determine a first reference position of each motion point and a second reference position of each base bone in the M base bones;

[0011] Converting the first reference position and the second reference position to generate global prediction features corresponding to N motion points, where the global prediction features are used to reflect the global position features between the N motion points and the corresponding base bones, the position features of the M base bones, and the structural features of the bone chains of the base bones corresponding to the N motion points;

[0012] Based on the global prediction features, the motion binding parameters between each motion point and the corresponding basic skeleton are determined, and any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0013] In one aspect, the present application provides a motion data processing device, the device comprising:

[0014] A first acquisition module is configured to acquire a first object model, where the first object model includes M basic bones or a portion of the M basic bones, where M is a natural number;

[0015] a second acquisition module, configured to acquire a second object model, the second object model comprising N motion points, wherein each of the N motion points corresponds to one or more basic bones of the M basic bones, the N motion points being used to define motion of the second object model, where N is a natural number;

[0016] A first determining module is configured to determine a first reference position of each motion point and a second reference position of each of the M basic bones;

[0017] A conversion module is used to convert the first reference position and the second reference position to generate global prediction features corresponding to N motion points. The global prediction features are used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chains where the basic bones corresponding to the N motion points are located;

[0018] The second determination module is used to determine the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features. Any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0019] In one aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor executes the method in one aspect of the present application.

[0020] In one aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0021] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in various optional embodiments such as the above-mentioned aspect.

[0022] The present application can obtain a first object model and a second object model, wherein the first object model may include M basic bones or some basic bones among the M basic bones, and the second object model may include N motion points, each of which may correspond to one or more basic bones among the M basic bones, and the N motion points may be used to define the motion of the second object model. Therefore, the present application can perform feature embedding processing on the first reference positions of the N motion points and the second reference positions of the M bones to generate global prediction features corresponding to the N motion points, and the global prediction features can be used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chains where the basic bones corresponding to the N motion points are located; thus, the present application can determine the motion binding parameters between each motion point and the corresponding basic bones based on the global prediction features; any of the N motion points can follow the corresponding basic bones to move according to the motion binding parameters between the N motion points and the corresponding basic bones. It can be seen that the method proposed in the present application can perform feature embedding processing on the first reference position of each motion point in the first object model and the second reference position of each motion point and the corresponding basic bone in the second object model to obtain global prediction features corresponding to N motion points. The global prediction features can include multiple features, such as the global position features between the motion point and the corresponding basic bone, the position features of M bones, and the structural features of the bone chain where the M bones are located. Subsequently, the motion binding parameters between each motion point and the corresponding basic bone can be quickly and accurately determined through the global prediction features. Any motion point can follow the corresponding basic bone to perform accurate movement according to the motion binding parameters between the motion point and the corresponding basic bone. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a schematic diagram of a network architecture provided in an embodiment of the present application;

[0024] FIG2a is a first schematic diagram of a scene of object following motion provided by an embodiment of the present application;

[0025] FIG2 b is a second schematic diagram of a scene of object following motion provided by an embodiment of the present application;

[0026] FIG3 is a flow chart of a motion data processing method provided in an embodiment of the present application;

[0027] FIG4 is a schematic diagram of a skin configuration interface provided by an embodiment of the present application;

[0028] FIG5a is a first schematic diagram of an interface of a skin configuration provided by an embodiment of the present application;

[0029] FIG5 b is a second schematic diagram of an interface of a skin configuration provided by an embodiment of the present application;

[0030] FIG6 a is a first schematic diagram of a post-processing configuration interface provided in an embodiment of the present application;

[0031] FIG6 b is a second schematic diagram of a post-processing configuration interface provided in an embodiment of the present application;

[0032] FIG6 c is a third schematic diagram of a post-processing configuration interface provided in an embodiment of the present application;

[0033] FIG6 d is a fourth schematic diagram of a post-processing configuration interface provided in an embodiment of the present application;

[0034] FIG7 is a schematic diagram of the effect of a post-processing provided in an embodiment of the present application;

[0035] FIG8 is a flow chart of a position smoothing method provided in an embodiment of the present application;

[0036] FIG9 is a schematic diagram of a scenario of position smoothing processing provided by an embodiment of the present application;

[0037] FIG10 is a schematic diagram showing the effect of smoothing a second object model according to an embodiment of the present application;

[0038] FIG11 is a flow chart of a feature generation method provided in an embodiment of the present application;

[0039] FIG12a is a structural diagram 1 of a skin network provided in an embodiment of the present application;

[0040] FIG12b is a second structural diagram of a skin network provided in an embodiment of the present application;

[0041] FIG12c is a third structural diagram of a skin network provided in an embodiment of the present application;

[0042] FIG12d is a fourth structural diagram of a skin network provided in an embodiment of the present application;

[0043] FIG12e is a fifth structural diagram of a skin network provided in an embodiment of the present application;

[0044] FIG12f is a sixth structural diagram of a skin network provided in an embodiment of the present application;

[0045] FIG12g is a seventh structural diagram of a skin network provided in an embodiment of the present application;

[0046] FIG13 is a flow chart of another feature generation method provided in an embodiment of the present application;

[0047] FIG14 is a flow chart of a network training method provided in an embodiment of the present application;

[0048] FIG15 is a schematic flow chart of a partial skinning method provided in an embodiment of the present application;

[0049] FIG16 is a schematic diagram of an interface for configuring a partial skin provided by an embodiment of the present application;

[0050] FIG17a is a structural diagram 1 of a local skin network provided by an embodiment of the present application;

[0051] FIG17b is a second structural diagram of a local skin network provided by an embodiment of the present application;

[0052] FIG17c is a third structural diagram of a local skin network provided by an embodiment of the present application;

[0053] FIG17d is a fourth structural diagram of a local skin network provided in an embodiment of the present application;

[0054] FIG18a is a schematic diagram of a partial skin effect provided by an embodiment of the present application;

[0055] FIG18b is a second schematic diagram of the effect of a partial skin provided by an embodiment of the present application;

[0056] FIG19 is a schematic structural diagram of a motion data processing device provided in an embodiment of the present application;

[0057] FIG20 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] This application relates to technologies related to artificial intelligence. Artificial Intelligence (AI) refers to theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is the study of the design principles and implementation methods of various intelligent machines, enabling them to have the capabilities of perception, reasoning, and decision-making.

[0059] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0060] This application mainly involves machine learning in artificial intelligence. Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by formula.

[0061] In this application, machine learning (such as model deep learning) can be used to automatically and accurately generate motion binding parameters between each motion point of the second object model and the basic skeleton of the first object model, so that the subsequent second object model can follow the corresponding basic skeleton to perform corresponding movements based on the motion binding parameters between each motion point and the corresponding basic skeleton. For details, please refer to the description in the following embodiments.

[0062] First of all, it should be noted that all data collected in this application (such as all data related to the first object model and the second object model) are collected with the consent and authorization of the object to which the data belongs (such as a user, organization or enterprise), and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0063] Here, the relevant technical concepts involved in this application are described.

[0064] Mesh: A mesh can be used to represent digital assets in 3D games (or other 3D scenes), such as clothing, scenes, etc. In this application, a mesh can refer to a second object model. In other words, the second object model can be constructed and represented based on a mesh.

[0065] Maya: A 3D modeling and animation software.

[0066] 3ds Max: Another 3D modeling and animation software.

[0067] Skinning: Establish the binding weights between the 3D Mesh (3D mesh) and the bones, that is, establish the binding relationship between the 3D Mesh and the bones.

[0068] LBS (Linear Blend Skinning): Linear blend skinning, an algorithm that uses bones to drive mesh vertices to move.

[0069] MLP (Multi-Layer Perceptron): Multi-layer perceptron, a deep learning module.

[0070] Embedding: Embed the features (i.e., information) of the input data into a vector, and use the learned vector to represent the features of the input data.

[0071] Please refer to Figure 1, which is a structural diagram of a network architecture provided by an embodiment of the present application. As shown in Figure 1, the network architecture may include a server 200 and a terminal device cluster. The terminal device cluster may include one or more terminal devices, and the number of terminal devices will not be restricted here. As shown in Figure 1, the multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n, where n is a positive integer; as shown in Figure 1, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all be connected to the server 200 through a network connection, so that each terminal device can exchange data with the server 200 through the network connection.

[0072] The server 200 shown in FIG1 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smart terminal such as a smartphone, tablet computer, laptop computer, desktop computer, smart TV, etc. The following describes the embodiments of the present application in detail, taking the communication between the terminal device 1 and the server 200 as an example.

[0073] Please refer to Figures 2a and 2b. Figure 2a is a schematic diagram of a scene of an object following motion provided by an embodiment of the present application. Figure 2b is a schematic diagram of a scene of an object following motion provided by an embodiment of the present application. The above-mentioned terminal device 1 can be a terminal device of an animator (user). The animator can initiate a request for skinning between a first object model and a second object model through the terminal device 1. Then, the terminal device 1 can give the request to the server 200, and the server 200 can implement the skinning between the first object model and the second object model. The first object model can be an object that can actively move in the animation, such as the first object model can be a virtual character or virtual monster modeled in a game, and the second object model can be an object that moves with the first object model, such as the second object model can be a piece of clothing that the first object model can wear or an accessory that the first object model can carry.

[0074] As shown in Figure 2a, the first object model can be a virtual nude model. The first object model can have several basic skeletons modeled, and the second object model can be clothing that the first object model can wear. Therefore, the second object model can be worn on the first object model. The second object model can be constructed based on a mesh grid, and each vertex of the mesh can be referred to as a motion point of the second object model. The movement of the motion point of the second object model can realize the movement of the entire second object model. Each motion point of the second object model can follow the basic skeleton of the first object model to perform corresponding movements.

[0075] Among them, the first object model can initially have a static posture (also called a starting posture or a standard posture, etc., such as a standing posture), and the first object model can move from a standing posture to a leaning posture. At this time, the various motion points of the second object model can also follow the basic skeleton of the second object model to perform corresponding movements, such as the movement of the motion point y100 here.

[0076] The corresponding movement of the second object model following the first object model can be achieved by the server 200 performing a skinning operation between the first object model and the second object model.

[0077] The process of skinning between the first object model and the second object model by the server 200 can be shown in Figure 2b: when the first object model wears the second object model and is in a static posture, the server 200 can obtain the reference position of each movement point of the second object model at this time (the reference position of the movement point can be referred to as the first reference position) and the reference position of each basic skeleton of the first object model (the reference position of each basic skeleton can be referred to as the second reference position). The first reference position of each movement point and the second reference position of each basic skeleton are the initial reference positions for skinning between the movement point and the basic skeleton.

[0078] Furthermore, the server 200 can perform conversion processing (which can be feature embedding processing) on ​​the second reference positions of each basic skeleton of the first object model and the first reference positions of each motion point of the second object model to obtain a global prediction feature, which is a feature used to predict the motion binding parameters (i.e., binding weights) between the basic skeleton of the first object model and the motion point of the second object model. For details on how to perform conversion processing on the second reference positions of each basic skeleton of the first object model and the first reference positions of each motion point of the second object model to generate the global prediction feature, please refer to the relevant description in the embodiment corresponding to Figure 3 below.

[0079] Among them, the process of predicting the motion binding parameters between the basic skeleton of the first object model and the motion points of the second object model is the process of skinning the first object model and the second object model. Subsequently, the motion points of the second object model can follow the basic skeleton of the first object model to perform corresponding movements based on the motion binding parameters between them and the basic skeleton of the first object model.

[0080] In the present application, feature embedding processing can be performed on the second reference position of the basic skeleton of the first object model and the first reference position of the motion point of the second object model to generate a global prediction feature for the motion point of the second object model. Subsequently, the motion binding parameters between each motion point and the corresponding basic skeleton can be quickly and accurately predicted through the global prediction feature, thereby realizing rapid skinning between the first object model and the second object model.

[0081] Please refer to Figure 3, which is a flow chart of a motion data processing method provided in an embodiment of the present application. The execution subject in the embodiment of the present application can be a motion data processing device, which can be a computer device or a computer device cluster composed of multiple computer devices. The computer device can be a server, a terminal device, or other devices, without limitation. The motion data processing device can be referred to as a processing device in the following. As shown in Figure 3, the method can include:

[0082] Step S101: Acquire a first object model, where the first object model includes M basic skeletons, where M is a natural number.

[0083] Optionally, the processing device can obtain the first object model of modeling, and the first object model can be an object that can actively move in an animation or game (or other scenes). For example, the first object model can be a virtual character or monster (such as a 3D character model or monster model) built in the game. The first object model can have several basic skeletons obtained by modeling, and the first object model can be moved by its basic skeleton. The number of basic skeletons that the first object model has can be determined according to actual application scenario, and this is not limited. In the present application, the first object model can include (i.e., have) M basic skeletons or include some basic skeletons in the M basic skeletons, where M is a natural number, and the specific value of M can be determined according to actual application scenario.

[0084] In other words, the above-mentioned M basic skeletons may all be the basic skeletons of the first object model; or, in some special scenarios, the M skeletons may also include auxiliary skeletons, which may not be the basic skeletons of the first object model, and the auxiliary skeletons may be the basic skeletons used to assist the second object model in moving. For example, if the second object model is a skirt with a puffy skirt, the auxiliary skeleton may be the basic skeleton used to support the skirt, that is, the auxiliary skeleton may be used to make the skirt puff up. The number of the auxiliary skeletons can also be determined according to the actual application scenario, and there is no restriction on this. It is understandable that in some scenarios, the second object model may also have no auxiliary skeletons, and in this case the auxiliary skeleton of the second object model may not be included in the M skeletons.

[0085] In summary, the above-mentioned M basic skeletons can include all the bones in the first object model that will affect the movement of N motion points (therefore, the M basic skeletons can include all or part of the basic skeletons of the first object model). In addition, according to the needs of the actual application scenario, the M basic skeletons may also include or exclude auxiliary bones used to assist the second object model in movement.

[0086] Step S102: Obtain a second object model, where the second object model includes N motion points, wherein each of the N motion points corresponds to one or more basic bones in the M basic bones, and the N motion points are used to define the motion of the second object model, where N is a natural number.

[0087] Optionally, the processing device may also acquire a second object model. This second object model can be any object that can follow the movement of the first object model in an animation or game (or other scenarios). For example, the second object model can be clothing or accessories worn by the first object model, or any detachable item that the first object model can carry. The second object model can be constructed and represented based on a mesh.

[0088] The second object model may include (i.e., have) N motion points, where N is a natural number. The specific value of N may be determined based on the actual application scenario. The motion points of the second object model may be vertices (referred to as mesh vertices) of a mesh representing the second object model. The mesh may be a polygonal mesh, such as a triangular mesh. The second object model may move based on its motion points. In other words, by moving each motion point of the second object model, the second object model as a whole can achieve corresponding motion.

[0089] It can be understood that the N motion points of the second object model define (or characterize) the movement of the second object model. In other words, the N motion points of the second object model can be used to describe / refer to the movement of the second object model (such as the motion trajectory). Therefore, it can be understood that the N motion points of the second object model can follow the above-mentioned M basic skeletons to move.

[0090] Furthermore, each of the N motion points may correspond to one or more basic skeletons among the M basic skeletons. The basic skeletons corresponding to any motion point may include all basic skeletons among the M basic skeletons that may affect the movement of any motion point. In other words, any motion point may move with reference to the movement of its corresponding basic skeleton. Any motion point may have one or more corresponding basic skeletons. The number of basic skeletons corresponding to different motion points may be the same or different, and there is no limitation on this.

[0091] Optionally, the basic skeleton corresponding to any movement point may be part of the M basic skeletons. For example, the basic skeleton corresponding to any movement point may be a plurality of basic skeletons in the M basic skeletons that are relatively close to the movement point (e.g., the 30 basic skeletons closest to the movement point). Alternatively, the basic skeleton corresponding to any movement point may also include all of the M basic skeletons.

[0092] However, it is understandable that, since most of the time not all of the M basic bones will have an impact on the movement of a certain motion point, by selecting some basic bones from the M basic bones as the basic bones corresponding to a certain motion point, only the motion binding parameters between the motion point and its corresponding basic bones can be predicted and determined subsequently, thereby reducing the amount of calculation for predicting the motion binding parameters between the motion point and the basic bones.

[0093] Optionally, the basic skeleton corresponding to each of the N motion points in the M basic skeletons may be pre-configured, so that when the first object model and the second object model are obtained, the basic skeleton corresponding to each motion point may be directly obtained (also referred to as identified); or, the basic skeleton corresponding to each of the N motion points in the M basic skeletons may be configured (such as selected) in real time by a technician who skins the first object model and the second object model and provided to a processing device. This application does not impose any restrictions on this.

[0094] For example, the picture that the first object model carries the second object model can be carried out visual display in animation software, and therefore, above-mentioned N motion points and M basic skeletons all can be visualization.Therefore, optionally, the basic skeleton corresponding to each above-mentioned motion point can be selected by corresponding technical personnel in this animation software for each motion point of visualization and obtains.As can be by the overall circle selection operation of some basic skeletons of display or the single-choice operation of each basic skeleton is selected, perhaps, can also be by the relevant selection strategy (such as selecting the basic skeleton of the target quantity with the minimum distance between the motion point as the strategy of the basic skeleton corresponding to the motion point etc.) of setting by processing equipment, automatically select.Optional, in fact, can select the basic skeleton corresponding to each motion point by any feasible mode, specifically how to select the basic skeleton corresponding to each motion point, can determine according to actual application scenario, do not limit this.

[0095] Through the above process, the processing device can obtain one or more basic skeletons corresponding to each of the N motion points in the M basic skeletons, and each motion point can move along with its own basic skeleton.

[0096] Step S103: Determine a first reference position of each motion point and a second reference position of each of the M basic skeletons.

[0097] Optionally, each of the N motion points and each of the M basic skeletons has its own starting position. A motion point can have a starting position, and a basic skeleton can have a starting position. The starting position of a basic skeleton can be the starting position of a specific point on the specified basic skeleton (such as the center point or vertex of the basic skeleton). The starting position can be the position when the first object model carries the second object model and maintains it in a starting posture (also known as a static posture or a standard posture). If the second object model is the clothing worn by the first object model, the starting posture is the standard standing posture of the first object model. Then, the starting positions of the N motion points and the starting positions of the M basic skeleton can be the positions of the second object model when it stands wearing the first object model. Optionally, the starting positions of the N motion points and the M basic skeleton can be position coordinates in a pre-established standard coordinate system (which can be 3D), such as the world coordinate system.

[0098] Each of the N movement points and each of the M basic bones may have a respective reference position. Optionally, the reference position of each basic bone may be the starting position of each basic bone.

[0099] If the second object model is clothing, the second object model may be very complex in many scenarios (because the types of clothing are very complicated). The clothing may have many wrinkles or multiple layers. Some overly detailed shapes of the clothing (such as wrinkles or layers) will affect the skinning effect, making the skinning between the first object model and the second object model uneven and inaccurate.

[0100] Therefore, the present application can also smooth the starting positions of the N moving points (i.e., smoothing operation) without changing the overall shape of the second object model to obtain the smoothed starting positions of each moving point, and then use the smoothed starting positions of each moving point as the reference positions of each moving point. Of course, in some cases, if the shape of the second object model is very regular (i.e., very regular) without too many wrinkles or details, then the starting positions of the N moving points can also be directly used as the reference positions of the N moving points. The specific method of determining the reference positions of the N moving points can be determined according to the actual application scenario.

[0101] It can be understood that the above-mentioned smoothing process is mainly to make the more detailed parts of the second object model smoother and easier to skin without changing the overall shape of the second object model. Therefore, the difference (i.e., the distance) between the smoothed starting position of each moving point and the starting position of each moving point is often very small. Therefore, regardless of whether the reference position of each moving point is the starting position of each moving point, or the position obtained after smoothing the starting position of each moving point, it can be understood that the reference position of each moving point can be used to reflect the starting position of each moving point to a certain extent.

[0102] The reference positions of the above-mentioned N motion points can be referred to as first reference positions, and the reference positions of the above-mentioned M basic bones can be referred to as second reference positions. Subsequently, the present application can realize the skinning between the first object model and the second object model based on the first reference position of each motion point and the second reference position of each bone, that is, generate motion binding parameters between each motion point and each basic bone, and the motion binding parameters characterize the binding relationship between each motion point and each basic bone. The motion point can follow the corresponding basic bone to move based on the motion binding parameters with the corresponding basic bone. The motion binding parameters between the motion point and the basic bone can be the binding weight between the motion point and the basic bone, and the binding weight can be used to determine how the motion point follows the corresponding basic bone to move.

[0103] The first reference position of each movement point and the second reference position of each basic skeleton can be represented by corresponding position coordinates in the above-mentioned standard coordinate system.

[0104] By smoothing the starting positions of the N motion points, some overly detailed local forms in the second object model can be smoothed, which helps to learn the smoother morphological features of the second object model through subsequent deep learning. Subsequently, the second object model can be skinned more accurately and stably as a whole, so that the skinning effect (such as the generated motion binding parameters) can also be smoother and more accurate, that is, there will not be too large a mutation between the generated motion binding parameters, because the mutated motion binding parameters are usually random and inaccurate.

[0105] It is understandable that the reference positions of the N moving points obtained by smoothing the starting positions of the N moving points are used to subsequently implement skinning between the first object model and the second object model. After the skinning is completed, each moving point still moves based on its own starting position combined with the corresponding motion binding parameters and follows the corresponding basic skeleton. In other words, smoothing the starting positions of the N moving points is to enable more accurate and stable skinning of the first object model and the second object model through deep learning (such as generating more accurate motion binding parameters) without changing the form of the second object model when it moves after the skinning is completed.

[0106] For the specific process of smoothing the starting positions of the N moving points to obtain the first reference positions of the N moving points, please refer to the description in the embodiment corresponding to FIG. 8 below.

[0107] In order to enable users (such as artists or animators) to use the technical solution of the present application more conveniently, the present application has also developed plug-ins for Maya and 3ds Max (which can be called skinning plug-ins, which can belong to applications or web programs, etc.). The skinning plug-in can be moved on the front end (such as a terminal device), and then the user can configure the relevant requirements of the skinning between the first object model and the second object model on the skinning plug-in.

[0108] Please refer to Figure 4, which is a schematic diagram of an interface for a skinning configuration interface provided by an embodiment of the present application. In the configuration interface shown in Figure 4, the overall skinning (i.e., global skinning) between the first object model and the second object model can be configured, such as the type of clothing (i.e., the type of clothing worn by the first object model, the second object model can be all the clothing worn by the first object model or part of the clothing worn by the first object model (such as only a top)) and the body type (i.e., the type of the first object model), and the clothing that needs to be skinned as a whole can be selected in the editing area (such as adding it at the model selection site in the scene, which can be the above-mentioned second object model that needs to be skinned), and the basic skeleton that affects the movement of the selected second object model (such as the above-mentioned M basic skeletons). Then, click Start Skinning to request the above-mentioned processing device to perform skinning on the selected second object model and the basic skeleton.

[0109] When the skinning-related requirements are configured in the skinning plug-in, the skinning plug-in can automatically generate a configuration file, which contains the corresponding data of the user's configuration of the skinning-related requirements in the skinning plug-in. Then, the skinning plug-in can submit the configuration file to the processing device for the processing device to perform skinning between the first object model and the second object model.

[0110] Referring again to Figures 5a and 5b, Figure 5a is a schematic diagram of a skinning configuration interface provided in an embodiment of the present application, and Figure 5b is a schematic diagram of a skinning configuration interface provided in an embodiment of the present application. In the configuration interface shown in Figure 5a, the user can import a configuration file generated by the skinning plug-in, or import a pre-generated configuration file, without limitation.

[0111] Furthermore, the configuration file may also include information related to a custom configuration, which may be configured in the configuration interface shown in FIG5 b. The information related to the custom configuration may include a service name (e.g., the first object model and the second object model to be skinned may be objects in a game, and the service name may be the name of the service to which the first object model and the second object model belong), a skinning type (e.g., a global skinning type), a URL (which may be a backend (e.g., a processing device) address, which may be used to access the backend skinning service), and the like.

[0112] Step S104, converting the first reference position and the second reference position to generate global prediction features corresponding to N motion points; the global prediction features are used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chain where the basic bones corresponding to the N motion points are located.

[0113] Optionally, the processing device can perform fusion embedding processing (belonging to conversion processing) on ​​the first reference positions of the above-mentioned N motion points and the second reference positions of the basic bones corresponding to the N motion points to generate global features of the motion points corresponding to the N motion points. This process can be understood as encoding (i.e., fusing) the information of the basic bones (such as position information) into the information of the motion points (such as position information) through the second reference positions of the basic bones corresponding to the motion points, and then obtaining the global representation corresponding to the N motion points. This global representation is also the global feature of the motion points.

[0114] Therefore, the global feature of the motion point can be used to reflect the global position features between N motion points and the corresponding basic skeleton. In other words, the global feature of the motion point can not only include the position features of each motion point, but also include the relative position features between each motion point and the corresponding basic skeleton.

[0115] The specific process of generating the above-mentioned global features of the moving points can be referred to the relevant description in the embodiment corresponding to FIG11 below.

[0116] In addition, the processing device can also perform feature embedding processing (also a conversion process) on the position features of the above-mentioned M basic bones to generate bone position features of the M basic bones, and the bone position features can be used to reflect the position features of the M basic bones.

[0117] The process of generating the above-mentioned global features of the motion points interacts with the process of generating the bone position features. The specific process of generating the bone position features can also be referred to the description in the embodiment corresponding to FIG. 11 below.

[0118] Furthermore, the processing device can also generate bone chain structural features associated with the basic bones corresponding to N motion points based on the bone chains of each basic bone in the M basic bones. As the name suggests, the bone chain structural features can be used to reflect the structural features of the bone chains of the basic bones corresponding to the N motion points.

[0119] The specific process of generating the above-mentioned bone chain structural features can be found in the relevant description in the embodiment corresponding to FIG13 below.

[0120] The above-generated global features of motion points, bone position features and bone chain structure features can all be feature matrices. Then, the processing device can fuse the above-generated global features of motion points, bone position features and bone chain structure features to generate global prediction features. The global prediction features are features used to ultimately predict the motion binding parameters between N motion points and the corresponding basic bones. The N motion points can correspond to the same global prediction feature.

[0121] For example, the above-generated global features of the motion points, the bone position features, and the bone chain structure features can be spliced ​​together (a feature fusion processing method, which can be horizontal splicing, and the order of splicing can be unlimited) to generate global prediction features. Alternatively, the above-generated global features of the motion points, the bone position features, and the bone chain structure features can be fused together, or the global features of the motion points, the bone position features, and the bone chain structure features can be summed up (such as adding the feature values ​​at the same element position (such as the same row and column position) in the global features of the motion points, the bone position features, and the bone chain structure features) or other processing can be performed, which can be determined according to the actual application scenario.

[0122] The present application may preferably perform splicing processing on the global features of the motion points, the bone position features and the bone chain structure features to obtain the global prediction features. This method can retain and reflect the global features of the motion points, the bone position features and the bone chain structure features in the global prediction features to the greatest extent. The global prediction features subsequently obtained in this way can also more accurately predict the motion binding parameters between the motion points and the corresponding basic bones.

[0123] Therefore, the global prediction features generated above can include the global features of the motion points, the bone position features and the bone chain structure features. The global prediction features can be used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones and the structural features of the bone chain where the basic bones corresponding to the N motion points are located.

[0124] It can be seen that the above-mentioned global prediction features obtained in the present application can include a series of complete features, including the position information of N motion points, the position information of M basic bones, the position information between the N motion points and the corresponding basic bones, and the structural features of the bone chain where the M basic bones are located (also called the association features of the bone chain where the M basic bones are located). Subsequently, the global prediction features can be used to realize the global prediction of the motion binding parameters between the N motion points and the corresponding basic bones.

[0125] Among them, the above-mentioned global features of the motion points, bone position features and bone chain structure features can all be generated by calling the skinning network. The skinning network (also called the skinning model) can be a trained deep learning network for predicting (i.e., determining) the motion binding parameters between the motion points and the corresponding basic bones.

[0126] For example, the global features of the motion points mentioned above can be generated by calling the skinning network and fusing and embedding the first reference positions of the N motion points and the second reference positions of the basic skeletons corresponding to the N motion points. The above-mentioned bone position features can be generated by calling the skinning network and fusing and embedding the second reference positions of the M basic skeletons. The above-mentioned bone chain structure features can also be generated by calling the skinning network based on the bone chains where the basic skeletons corresponding to each motion point are located. Furthermore, the above-mentioned global prediction network can also be generated by calling the skinning network and fusing the above-mentioned global features of the motion points, bone position features and bone chain structure features. The feature processing operations involved in this series of processes can all belong to the operations of converting the first reference position and the second reference position.

[0127] The specific process of training the skinning network can be found in the description of the embodiment corresponding to FIG14 . The skinning network can be a network that can be used to perform skinning between the first object model and the second object model. For example, skinning can refer to a process in which the skinning network can predict motion binding parameters between the motion points of the second object model and the M basic skeletons.

[0128] Optionally, the present application can not only convert the first reference positions of all the above-mentioned N motion points and the second reference positions of the basic skeletons corresponding to all the motion points to generate global prediction features corresponding to all the motion points; the present application can also only convert the first reference positions of some of the N motion points (such as 1 or more) and the second reference positions of the basic skeletons corresponding to these motion points to generate global prediction features corresponding to these motion points. In this case, the method provided by the present application can be used to determine the motion binding parameters between these motion points and the corresponding basic skeletons. Among them, the specific motion binding parameters between which motion points of the second object model and the corresponding basic skeletons need to be determined can be determined according to the actual application scenario, and the present application does not impose any restrictions on this.

[0129] Step S105, based on the global prediction features, determine the motion binding parameters between each motion point and the corresponding basic skeleton; any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0130] Optionally, the processing device can determine (predict) the motion binding parameters (also called binding weights) between each motion point and the corresponding basic skeleton through the global prediction features generated above. Any one of the N motion points can be used to follow the basic skeleton corresponding to the any motion point to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0131] The processing device may call upon the skinning network to predict the motion binding parameters between each motion point and the corresponding base skeleton based on the generated global prediction features. A motion binding parameter may be predicted between a motion point and a base skeleton corresponding to the motion point.

[0132] It can be understood that usually a motion point will have corresponding multiple basic bones, and among the predicted motion binding parameters between the motion point and the corresponding multiple basic bones, usually only a few (such as 3, 4, etc.) motion binding parameters are greater than 0 (this part of the motion binding parameters is the motion binding parameters that ultimately determine how the motion point moves), and the other motion binding parameters are predicted to be 0 (this part of the motion binding parameters will not affect the movement of the motion point). Which specific motion binding parameters between the motion point and the corresponding basic bones are not 0 are all determined by the adaptive learning process of the skinning network.

[0133] The sum of the motion binding parameters between any predicted motion point and the corresponding basic bones (ie, the sum value) may be equal to 1.

[0134] Optionally, the motion binding parameters between the motion points predicted by the skinning network and the corresponding basic bones can be used as the initial motion binding parameters, that is, the skinning network can be called based on the global prediction features to predict the initial motion binding parameters between each motion point and the corresponding basic bones.

[0135] In many cases, the positions of some motion points in the second object model need to be collinear. For example, if the second object model is clothing, the motion points on the same clothing line of the clothing need to be collinear (i.e., they need to be on the same line). Therefore, in the present application, the initial motion binding parameters between each of the above-mentioned predicted motion points and the corresponding basic skeleton can also be post-processed. The post-processing can be based on the collinearity optimization method for N motion points to optimize the initial motion binding parameters between each motion point and the corresponding basic skeleton, as described below.

[0136] The processing device can calculate the predicted motion positions of N motion points under multiple target object poses of the first object model (which can be multiple preset object poses, such as all or some object poses that the first object model can make) by using the initial motion binding parameters predicted for each motion point. Each motion point can have a predicted motion position under each target object pose. The predicted motion position of a motion point under a target object pose is the position of the first object model after moving from the current object pose to the target object pose using the initial motion binding parameters predicted for each motion point. This position can be calculated using the LBS algorithm defined in the following formula (1).

[0137] The processing device may also obtain straight wiring patterns formed by the N moving points in the aforementioned multiple target object postures. The N moving points may form a straight wiring pattern in a single target object posture. The straight wiring pattern formed by the N moving points in a single target object posture is, when the first object model is in the target object posture, the distribution of the N moving points obtained by arranging the N moving points that need to be collinear in a relatively standard manner on a line.

[0138] Among them, the process of generating a straight wiring of N moving points under a target object posture may include: first, the mesh of the second object model may be composed of a number of triangles, and when the first object model is in the target object posture, the triangles in the mesh of the second object model that is also currently in the target object posture may be merged (i.e., the hypotenuse is filtered out or the hypotenuse of the triangle is removed) to obtain a merged mesh, such as two triangles with the same side can be merged into a quadrilateral. Since straight wiring can usually be represented by quadrilaterals, the mesh can be merged with triangles, which can also simplify the structure of the mesh, reduce the difficulty of generating straight wiring, and reduce the amount of calculation for generating straight wiring. In the above process, only some edges in the mesh will be eliminated, but the moving points in the mesh will not be eliminated, and all the moving points will be retained.

[0139] Furthermore, a half-edge structure can be established for the merged mesh (the edges in the merged mesh have directions, that is, an edge can have an outgoing edge and / or an incoming edge, and two connected moving points in the mesh can form an edge) to traverse the moving points (that is, vertices) that need to be collinear in the merged mesh. For example, the edge with the smallest angle traversed to a certain edge and tending to 0 (such as less than a certain angle threshold, the angle threshold is a value tending to 0) can be regarded as the edge that needs to be collinear.

[0140] Furthermore, the moving points on the edges that need to be collinear can be regarded as the moving points that need to be collinear, and the positions of the moving points that need to be collinear can be smoothed so that the moving points that need to be collinear are on a smoother line, thereby generating a straight wiring for N moving points under the posture of the target object.

[0141] Therefore, the processing device can also obtain the target posture positions of the N moving points under the above-mentioned multiple target object postures based on the straight wiring formed by the N moving points under the multiple target object postures. That is, the position of the moving point determined by the straight wiring can be called the target posture position, and one moving point can have one target posture position under one target object posture.

[0142] Furthermore, the processing device can optimize the initial motion binding parameters between the N motion points and the corresponding basic bones through the difference between the predicted motion positions of the N motion points under the above-mentioned multiple target object postures and the target posture positions of the N motion points under the multiple target object postures, and then the optimized initial motion binding parameters can be used as the final motion binding parameters between the motion points and the corresponding basic bones.

[0143] Among them, the process of optimizing the initial motion binding parameters between the N motion points and the corresponding basic skeleton through the difference between the predicted motion positions of the N motion points in the above-mentioned multiple target object postures and the target posture positions of the N motion points in the multiple target object postures can include: the processing device can calculate the corresponding loss function (which can be called the target loss function) through the predicted motion positions of the N motion points in the above-mentioned multiple target object postures and the target posture positions of the N motion points in the multiple target object postures. The loss function can be the L2-loss (square loss function) or mse loss (mean square error, a type of loss function) between the predicted motion positions of the N motion points in the above-mentioned multiple target object postures and the target posture positions of the N motion points in the multiple target object postures. The loss function can be used to reflect the error of the predicted motion positions of the N motion points in the above-mentioned multiple target object postures compared with the target posture positions of the N motion points in the multiple target object postures.

[0144] Therefore, the processing device can adjust (i.e., optimize) the initial motion binding parameters between each motion point and the corresponding basic skeleton through the above-mentioned target loss function, so that the target loss function tends to a minimum value (such as tends to 0). By making the target loss function tend to a minimum value, that is, making the predicted motion positions of the N motion points under the above-mentioned multiple target object postures as a whole approach (i.e., approximate) the target posture positions of the N motion points under the multiple target object postures.

[0145] Therefore, after optimizing the initial motion binding parameters between each motion point and the corresponding basic skeleton through the above process, the accuracy of the motion binding parameters between each motion point and the corresponding basic skeleton can be improved. Through the accurate motion binding parameters, after the first object model moves, the N motion points of the second object model that need to be collinear will be more standardly collinear, thereby achieving more accurate skinning between the first object model and the second object model.

[0146] Optionally, in addition to the above-mentioned collinear optimization method, the post-processing method of the motion binding parameters obtained by the above prediction may also include other additional processing, such as vertex stitching processing, patch penetration processing, and weight constraint processing at seams, etc., and any feasible processing. The post-processing method can be arbitrarily selected according to actual application requirements.

[0147] Among them, vertex stitching processing can refer to attaching other object models (such as a small flower or any other object) to the second object model for movement. Therefore, the vertex weight of the other object model (that is, the binding weight between the movement point of the other object model and the basic skeleton of the first object model) can be set to be consistent with the vertex weight of the local part (that is, the place where the other object model needs to be attached to the second object model) (that is, the binding weight between the local movement point of the second object model and the corresponding basic skeleton).

[0148] Since the second object model may have multiple overlapping layers (such as overlapping layers of clothing, which can be called overlapping multi-layer local objects), the patch penetration processing can refer to setting the weights of overlapping vertices in the multiple overlapping layers of the second object model (i.e., the weights between the vertices and the corresponding basic bones) to be consistent.

[0149] The above-mentioned weight constraint processing at the seam may refer to setting the vertex weights of the movement points of the local part (such as the neck of the second object model) of the second object model that completely follows the local part (such as the neck of the first object model) of the first object model to fit the movement, so that the position of the local part (i.e., the neck) of the second object model is the same as the vertex weight of the local part (i.e., the neck) of the first object model.

[0150] Please refer to Figures 6a-6d, Figure 6a is a schematic diagram of the interface of a post-processing configuration interface provided in an embodiment of the present application, Figure 6b is a schematic diagram of the interface of a post-processing configuration interface provided in an embodiment of the present application, Figure 6c is a schematic diagram of the interface of a post-processing configuration interface provided in an embodiment of the present application, and Figure 6d is a schematic diagram of the interface of a post-processing configuration interface provided in an embodiment of the present application. The configuration interface shown in Figures 6a-6d can also be the interface in the above-mentioned skinning plug-in. Specifically, the user can select the vertices (i.e., motion points) that need to be optimized for collinear vertices in the configuration interface shown in Figure 6a, can select the vertices that need to be stitched for vertices in the configuration interface shown in Figure 6b, can select the patches that need to be processed for patch penetration (such as overlapping multi-layer clothing) in the configuration interface shown in Figure 6c, and can select clothing (such as a neck scarf) that needs to be weighted at the seams in the configuration interface shown in Figure 6d, etc.

[0151] Please refer to Figure 7, which is a schematic diagram of the effect of a post-processing provided by an embodiment of the present application. As shown in Figure 7, in some scenarios, if the motion binding parameters between the predicted motion points of the second object model and the corresponding basic bones are not collinearly optimized, then after the second object model moves based on the predicted motion binding parameters, the lines of the second object model may be messy. However, after the motion binding parameters between the predicted motion points of the second object model and the corresponding basic bones are collinearly optimized, the lines of the second object model are arranged neatly.

[0152] After predicting the motion binding parameters between each moving point and the corresponding base skeleton, when the base skeleton corresponding to the moving point moves, the moving point can follow the corresponding base skeleton based on the motion binding parameters between the moving point and the corresponding base skeleton to move. In this application, an LBS algorithm can be used to make the movement of the base skeleton drive the moving point to move, as shown in the following formula.

[0153] Wherein, i represents the i-th moving point among N moving points, and i can be a positive integer less than or equal to N, that is, the i-th moving point can be any moving point among the N moving points. i It can be the starting position of the i-th motion point, j represents the j-th basic skeleton corresponding to the i-th motion point, j is a positive integer less than or equal to J, J can be the total number of basic skeletons corresponding to the i-th motion point, T j It can be the motion parameter of the j-th basic bone, which can be the translation and rotation motion matrix of the j-th basic bone, that is, the motion parameter is used to indicate how the j-th basic bone needs to move, such as the direction of translation distance and rotation angle. ij It is the motion binding parameter between the i-th motion point and the corresponding j-th basic bone, that is, the control weight of the j-th basic bone on the i-th motion point. It is the position that the i-th motion point needs to reach after following the movement of the corresponding basic skeleton.

[0154] It can be understood that by setting the object posture to which the first object model needs to move, the above T j It can be automatically determined that if the first object model is required to move from a stationary object posture (such as a standing object posture) to a jumping object posture, then T j These are the motion parameters of the j-th basic bone in the process of the first object model moving from a standing object posture to a jumping object posture, such as the distance the j-th basic bone needs to translate and the angle it needs to rotate when the first object model moves from a standing object posture to a jumping object posture.

[0155] Among them, the various parameters in the above formula (1) can be obtained on the basis of the first object model being in a stationary object posture (i.e., a standard object posture, also called a starting object posture). If the first object model has multiple posture transformations (i.e., multiple movements) in a process, then the various parameters in formula (1) can be obtained by continuously superimposing relevant parameters of the transformed posture on the basis of the stationary object posture of the first object model. In other words, the various parameters in the above formula (1) can be parameters for moving the first object model to a desired posture on the basis of the first object model being in a stationary object posture (i.e., a standard object posture, also called a starting object posture).

[0156] The above process describes the overall process of updating the positions of N motion points based on the motion binding parameters between the M basic skeletons and the N motion points as the M basic skeletons move, thereby achieving the effect that the N motion points respectively follow the corresponding basic skeletons to move. The above process is the process of performing overall skinning between the first object model and the second object model. As can be seen from the above, in this application, when the motion parameters of the motion point corresponding to the basic skeleton are known, the motion position of the motion point can be determined by the motion binding parameters between the motion point and the corresponding basic skeleton. In other words, when the motion parameters of the motion point corresponding to the basic skeleton are known, the motion binding parameters between the motion point and the corresponding basic skeleton can be used to allow the basic skeleton corresponding to the motion point to drive the motion point to perform corresponding movements.

[0157] The present application can realize the rapid and accurate prediction of the motion binding parameters between the motion points and the corresponding basic skeletons through the skinning network obtained by deep learning, thereby reducing the manpower overhead of establishing the motion binding parameters between the motion points and the corresponding basic skeletons by artists.

[0158] The present application can obtain a first object model and a second object model, wherein the first object model may include M basic bones or some basic bones among the M basic bones, and the second object model may include N motion points, each of which may correspond to one or more basic bones among the M basic bones, and the N motion points may be used to define the motion of the second object model. Therefore, the present application can perform feature embedding processing on the first reference positions of the N motion points and the second reference positions of the M bones to generate global prediction features corresponding to the N motion points, and the global prediction features can be used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chains where the basic bones corresponding to the N motion points are located; thus, the present application can determine the motion binding parameters between each motion point and the corresponding basic bones based on the global prediction features; any of the N motion points can follow the corresponding basic bones to move according to the motion binding parameters between the N motion points and the corresponding basic bones. It can be seen that the method proposed in the present application can perform feature embedding processing on the first reference position of each motion point in the first object model and the second reference position of each motion point and the corresponding basic bone in the second object model to obtain global prediction features corresponding to N motion points. The global prediction features can include multiple features, such as the global position features between the motion point and the corresponding basic bone, the position features of M bones, and the structural features of the bone chain where the M bones are located. Subsequently, the motion binding parameters between each motion point and the corresponding basic bone can be quickly and accurately determined through the global prediction features. Any motion point can follow the corresponding basic bone to perform accurate movement according to the motion binding parameters between the motion point and the corresponding basic bone.

[0159] Please refer to Figure 8, which is a flow chart of a position smoothing method provided by an embodiment of the present application. As shown in Figure 8, the method may include:

[0160] Step S201 , calculating the discrete curvature of each moving point based on the starting position of each moving point and the starting positions of its adjacent moving points; the discrete curvature of any moving point is used to reflect the smoothness of the surface around the moving point.

[0161] Optionally, the N moving points each have their own starting position, which is the position of the N moving points of the second object model when the first object model carries the second object model in a static posture (such as a standard static posture).

[0162] The processing device can obtain the adjacent moving points (also called adjacent vertices or neighbor vertices) of each moving point among the N moving points. The adjacent moving points of a moving point can include all the moving points that are directly connected to the moving point (i.e., have a connection relationship) in the mesh of the second object model.

[0163] The processing device can calculate the discrete curvature (also known as the discrete average curvature) of any moving point using the starting position of the moving point and the starting positions of its adjacent moving points. This discrete curvature can be used to reflect the smoothness (or degree of smooth curvature) of the surface surrounding the moving point (such as the surface of the surrounding mesh). The larger the discrete curvature, the less smooth the curve surrounding the moving point. Conversely, the smaller the discrete curvature, the smoother the curve surrounding the moving point. Each moving point has one discrete curvature. Through the above process, the discrete curvatures of N moving points can be obtained.

[0164] Among them, the starting position of the above-mentioned moving point can be a three-dimensional position in a three-dimensional coordinate system (such as including the x-axis, y-axis, and z-axis), and the discrete curvature of any moving point can be calculated by substituting the starting position of any of the above-mentioned moving points and the starting position of the adjacent moving point of any of the moving points into the mathematical formula for calculating the discrete curvature.

[0165] Step S202 : obtaining multiple reference curvatures, and smoothing the starting positions of the N moving points based on the multiple reference curvatures and the discrete curvatures of the N moving points to generate smoothed positions of the N moving points under each reference curvature.

[0166] Optionally, the present application can achieve smoothing of the starting positions of N moving points by referring to the curvature. The goal of smoothing the starting positions of the moving points in the present application can be to moderately smooth and collapse the wrinkled and fluffy detail areas in the second object model (such as clothing), and to maintain the original shape of the areas in the second object model that originally have a relatively smooth surface as much as possible without making any changes.

[0167] Therefore, when the present application smoothes the starting positions of N moving points through a reference curvature, the starting positions of the moving points among the N moving points whose discrete curvature is greater than or equal to the reference curvature (i.e., the moving points whose surrounding surfaces are not very smooth) can be smoothed, while the starting positions of the moving points among the N moving points whose discrete curvature is less than the reference curvature (i.e., the moving points whose surrounding surfaces are relatively smooth) remain as they are and are not smoothed.

[0168] On this basis, since the shapes and styles of different second object models may vary greatly (such as the shapes and styles of complex clothing with a large number of styling designs may vary greatly), it is difficult to adapt to all clothing shapes by relying on a given reference curvature alone. Therefore, the present application can use a set of set reference curvatures to smooth the starting position of the motion point of the second object model in turn, and then can adaptively select the most suitable reference curvature for a specific second object model, so as to smooth the starting position of the motion point of the second object model through the most suitable reference curvature.

[0169] Therefore, the processing device can obtain multiple reference curvatures, which can be multiple reference curvatures of different sizes. The number and values ​​of the multiple reference curvatures can be arbitrarily set according to the actual application scenario, such as setting multiple suitable curvatures with uniform numerical distribution (such as numerical values ​​in an arithmetic progression) as multiple reference curvatures, and there is no restriction on this.

[0170] Then, the processing device can smooth the starting positions of the N moving points using the multiple reference curvatures and the discrete curvatures of the N moving points to generate smoothed positions of the N moving points under each reference curvature. One moving point has one smoothed position under one reference curvature.

[0171] Among them, any one of the above-mentioned multiple reference curvatures can be called the target reference curvature. The following example describes the process of smoothing the starting positions of N moving points based on the target reference curvature and the discrete curvatures of N moving points to obtain the smooth position of each moving point under the target reference curvature, as described below.

[0172] The processing device may treat a motion point among the N motion points whose discrete curvature is greater than the target reference curvature as a motion point that needs to be adjusted (which may be simply referred to as a motion point that needs to be adjusted). Furthermore, the processing device may smooth the starting position of the motion point that needs to be adjusted to generate a smoothed starting position of the motion point that needs to be adjusted, while the starting positions of the motion points among the N motion points other than the motion point that needs to be adjusted are not smoothed and remain unchanged.

[0173] Furthermore, the processing device can use the starting positions of the N moving points except the moving point to be adjusted, and the smoothed starting position of the moving point to be adjusted, as the smoothed positions of the N moving points under the target reference curvature, that is, a moving point has a smooth position under the target reference curvature, and the smoothed position can be the starting position of the moving point or the smoothed starting position of the moving point.

[0174] Optionally, the present application may use a Laplace smoothing algorithm to smooth the starting position of the moving point to be adjusted. The idea of ​​the Laplace smoothing algorithm is to continuously adjust the position of the vertex through multiple iterations based on the position relationship of neighboring vertices in the mesh to improve the shape and quality of the mesh. As shown in the following formula. (d) =P (d-1) -λLP (d-1) (2)

[0175] Among them, any one of the moving points to be adjusted can be the moving point s, d represents the number of iterative adjustments to the position of the moving point s (the number can be set according to the actual application scenario, and can be the iterative update of the starting position of the moving point s), p s P (d-1) An element in p s represents the position of the moving point s after iterative update d-1 times, P (d-1) Contains the positions of all moving points that need to be adjusted after d-1 iterations of updating the starting positions, P (d) Contains the positions of all moving points that need to be adjusted after d iterations of updating their starting positions. λ is a hyperparameter that can range from [0, 1]. A larger λ value results in a greater influence on the position update (i.e., smoothing) of the moving point's neighboring vertices (or adjacent moving points).

[0176] L is the Laplace matrix, which determines the update direction of the position of each moving point to be adjusted, p c It represents the position of any neighboring moving point c (which can be an adjacent moving point or a neighboring vertex) of the moving point s after iterative update d-1 times, that is, c represents any neighboring moving point of the moving point s, N s is the set of neighboring moving points of moving point s, w sc L(p is the weight of the influence of the neighboring moving point c on the position update of the moving point s during the Laplace smoothing process. The larger the weight, the greater the influence of the neighboring moving point c on the position update of the moving point s. s ) is included in the Laplace matrix L, L(p s ) represents the Laplace vector representation of the moving point s.

[0177] Therefore, according to the principle described above, after smoothing the starting positions of the N moving points using each reference curvature, the smoothed positions of the N moving points under each reference curvature can be obtained.

[0178] Please refer to Figure 9, which is a schematic diagram of a scene for position smoothing provided by an embodiment of the present application. As shown in Figure 9, vertex 1 can be any moving point of the second object model, and vertices 2 to 5 can be adjacent vertices (i.e., adjacent moving points) of vertex 1. Therefore, after smoothing the starting position of vertex 1 through the starting positions of vertices 2 to 5, a smoothed vertex 1 can be obtained, and the position of vertex 1 after smoothing can become more collapsed compared to the position of vertex 1 before smoothing.

[0179] Please refer again to Figure 10, which is a schematic diagram of the effect of smoothing a second object model provided by an embodiment of the present application. By adopting the above-mentioned method of adaptively selecting a reference curvature based on the shape of the second object model (i.e., the shape of the clothing) of the present application, the first short skirt with more wrinkles in Figure 10 can be smoothed to make its curved surface smoother (i.e., smooth). Furthermore, after smoothing the second skirt with a relatively smooth slit curved surface and the third belt with a thin sheet structure in Figure 10, the skirt and belt essentially maintain their original shapes. Therefore, the above-mentioned method of the present application can achieve moderate smoothing of clothing without causing excessive deformation of the clothing after smoothing.

[0180] Step S203 : Based on the smooth positions of the N moving points at each reference curvature, respectively calculate the average curvature of the N moving points at each reference curvature.

[0181] Optionally, the processing device may calculate the average curvature of the N moving points at each reference curvature based on the smoothed positions of the N moving points at each reference curvature.

[0182] Here, the process of calculating the average curvature of N moving points under the target reference curvature is taken as an example for explanation, as described below.

[0183] The processing device can calculate the discrete curvature of any moving point under the target reference curvature based on the smooth position of any moving point under the target reference curvature and the smooth position of the adjacent moving points of any moving point under the target reference curvature (the calculation method is the same as the above-mentioned method of calculating the discrete curvature through the starting position). In this way, the discrete curvature of each moving point under the target reference curvature can be calculated.

[0184] Furthermore, the processing device may average the discrete curvatures of each moving point under the target reference curvature, and thus obtain the average curvature of the N moving points under the target reference curvature.

[0185] According to the above principle, the processing device can calculate the average curvature of the N moving points under each reference curvature. The average curvature of the N moving points under a reference curvature can be called the average curvature corresponding to the reference curvature.

[0186] Step S204 : taking the smoothed positions of the N moving points at the reference curvature corresponding to the minimum average curvature as the first reference positions of the N moving points.

[0187] Optionally, the processing device may use the smoothed positions of the N moving points under the reference curvature with the smallest corresponding average curvature (i.e., the starting positions of the N moving points after smoothing reach the smoothest (smoothest) reference curvature as a whole) as the final first reference positions of the N moving points.

[0188] In other words, the processing device can use the reference curvature with the smallest corresponding average curvature as the reference curvature for ultimately smoothing the starting positions of the N moving points, and the processing device can use the smoothed positions of the N moving points under the reference curvature as the first reference positions of the N moving points.

[0189] Through the above process, the starting positions of the N motion points can be smoothed respectively through multiple reference curvatures, and then the reference curvature (i.e., the reference curvature corresponding to the minimum average curvature) finally used for smoothing the N motion points can be adaptively selected from the multiple reference curvatures based on the results of smoothing the starting positions of the N motion points, so as to obtain the first reference position after the overall optimal smoothing of the starting positions of the N motion points. Subsequently, the first reference position obtained after smoothing the starting positions of the N motion points can also be used to more accurately predict the motion binding parameters between the N motion points and the corresponding basic bones.

[0190] Referring to Figure 11, Figure 11 is a schematic flow diagram of a feature generation method provided by an embodiment of the present application. Figure 11 corresponds to the embodiment and mainly describes the process of generating the global feature of the moving point. However, since the process of generating the global feature of the moving point interacts with the process of generating the skeleton position feature of M basic skeletons, the process of generating the skeleton position feature of M basic skeletons is also described in the following process (specifically described in the following step S304). As shown in Figure 11, the method may include:

[0191] Step S301 : Based on the first reference position of each motion point and the second reference position of the base skeleton corresponding to each motion point, position association information between each motion point and the corresponding base skeleton is generated respectively.

[0192] Optionally, the processing device can generate position association information between each motion point and the corresponding basic bone through the first reference position of each motion point and the second reference position of the basic bone corresponding to each motion point. There can be one position association information between a motion point and a basic bone corresponding to the motion point.

[0193] Any one of the N motion points can be the target motion point, and the position association information between the target motion point and any basic bone corresponding to the target motion point can include at least one of the following: the relative position information between the target motion point and the basic bone, the absolute distance between the target motion point and the basic bone, and the shortest path distance between the target motion point and the basic bone.

[0194] Among them, the relative position information can be the difference between the first reference position of the target motion point and the second reference position of the corresponding basic skeleton. Since the starting position and the first reference position of the motion point can be three-dimensional, the relative position information can also be three-dimensional. The relative position information can be used to reflect the relative position between the target motion point and the corresponding basic skeleton.

[0195] The absolute distance may refer to the Euclidean distance between the first reference position of the target motion point and the second reference position of the corresponding base skeleton. The Euclidean distance may be one-dimensional. The Euclidean distance may be used to reflect the true distance between the target motion point and the corresponding base skeleton in three-dimensional space.

[0196] The shortest path distance may be the geodesic distance between the target motion point and the corresponding base skeleton. The geodesic distance may be the shortest distance from the target motion point to the corresponding base skeleton. The geodesic distance may also be one-dimensional.

[0197] Therefore, a 5-dimensional positional association information can be formed between the target motion point and the corresponding basic skeleton. If the number of basic skeletons corresponding to each motion point is R, where R is a positive integer, then the total dimension of the positional association information between N motion points and their corresponding basic skeletons can be expressed as (N, R, 5).

[0198] Step S302 : performing feature embedding processing on the position association information between the N motion points and the corresponding basic skeletons to generate skeleton association features of the N motion points.

[0199] Optionally, the processing device can call the above-mentioned skinning network to perform feature embedding processing on the position association information (a total of ((N, R, 5) dimensions) between the N motion points and the corresponding basic skeletons, so as to encode the information of the basic skeleton corresponding to each motion point (such as position information) into the information of each motion point, thereby generating the bone association features of the N motion points, that is, the features obtained after the feature embedding processing of the position association information (a total of ((N, R, 5) dimensions) between the N motion points and the corresponding basic skeletons are called bone association features.

[0200] The dimension of the skeleton-associated feature finally obtained can be (N, 64), that is, the position association information between a motion point and the corresponding basic skeleton is embedded, and finally a 64-dimensional feature can be obtained. Each motion point can correspond to a 64-dimensional feature in the skeleton-associated feature (in fact, it can also be set to other dimensions, there is no restriction on this).

[0201] Step S303 : performing feature transformation processing on the skeleton-related features and the first reference positions of the N motion points to generate motion point transformation features of the N motion points.

[0202] Optionally, the processing device may obtain normal information of each moving point in a standard coordinate system (such as a world coordinate system, including three dimensions, namely, an x-axis, a y-axis, and a z-axis), and the normal information may also be three-dimensional.

[0203] Furthermore, the processing device can combine the first reference position (3-dimensional) of each moving point and the normal information (3-dimensional) of each moving point to obtain the basic features of N moving points. The dimension of the basic features can be (N, 6), that is, one moving point corresponds to a 6-dimensional feature in the basic features, and the 6-dimensional feature can include the first reference position of the moving point and the normal information of the moving point.

[0204] Furthermore, the processing device can fuse the above-mentioned skeleton-related features and basic features to generate initial representation features of N motion points, that is, the features obtained after fusing the above-mentioned skeleton-related features and basic features can be referred to as the initial representation features of N motion points (the initial representation features can be feature matrices). Optionally, the fusion process can be a splicing process, or it can be other processes, which can be specifically determined according to the actual application scenario. The dimension of the initial representation feature can be (N, 70), that is, one motion point can correspond to a 70-dimensional feature in the initial representation feature.

[0205] Furthermore, the processing device can perform feature transformation processing on the above-mentioned initial representation features (mainly stretching and rotating the initial representation features) to generate motion point transformation features of N motion points, that is, the features obtained after the feature transformation processing of the initial representation features can be called motion point transformation features (which can be feature matrices).

[0206] Optionally, the dimension of the motion point transformation feature may also be (N, 70), that is, a motion point may correspond to a 70-dimensional feature in the motion point transformation feature, and a 70-dimensional feature of a motion point in the motion point transformation feature may be called the transformation feature of the motion point. In other words, the motion point transformation feature may contain the transformation feature of each motion point.

[0207] By performing feature transformation processing on the initial representation features, the posture of the first object model can be transformed (such as transforming the features of a standing posture into the features of a jumping posture). This method can increase the accuracy of the features, allowing the skinning network to adapt to a variety of transformed features, thereby making the skinning network more robust to the posture of the first object model.

[0208] Step S304: Generate a global feature of the moving point based on the moving point transformation feature.

[0209] Optionally, the processing device may obtain neighbor features of each moving point to perform information exchange between each moving point and corresponding neighboring moving points, as described below.

[0210] Any one of the N moving points can be referred to as a target moving point. The following is a specific description of the process of obtaining neighbor features of the target moving point, using multiple neighbor selection methods (which can be multiple preset methods) to select multiple neighbor moving points for the target moving point from the N moving points, so that multiple neighbor features of the target moving point can be subsequently obtained. Specifically, using one neighbor selection method can obtain a single neighbor moving point of the target moving point.

[0211] For example, the multiple neighbor motion point selection methods may include at least two of the following:

[0212] The first neighbor motion point selection method may be: the target motion point may have several connected motion points in the mesh, and therefore, multiple motion points (which may be adjacent motion points belonging to the target motion point) may be selected (e.g., randomly selected) from several motion points connected to the target motion point as neighbor motion points of the target motion point.

[0213] A second neighboring motion point selection method may include selecting a plurality (a specified number) of motion points whose transformation features have the smallest difference from the transformation features of the target motion point as neighboring motion points of the target motion point. Optionally, the transformation features of the motion points may be represented as feature vectors, and the difference between the transformation features of different motion points may be represented by vector distance. A larger vector distance between different transformation features indicates a greater difference between the transformation features.

[0214] The third method for selecting neighbor motion points can be: clustering N motion points to treat motion points that are clustered into the same class as neighbor motion points. Optionally, the N motion points can be clustered based on the distance between each motion point and each basic skeleton (such as geodesic distance). For example, multiple motion points (which can be a specified number) that are closest to the same basic skeleton can be clustered into the same class. In this way, various classes based on the basic skeletons can be obtained. Different classes can contain some of the same motion points. Therefore, other motion points in the class where a certain motion point is located can be treated as neighbor motion points of the motion point.

[0215] The processing device may perform feature embedding processing on the transformation features of the target moving point and the transformation features of each type of neighboring moving point of the target moving point, thereby generating multiple neighbor-related features of the target moving point. The multiple neighbor-related features constitute the neighbor features of the target moving point. In other words, by performing feature embedding processing on the transformation features of the target moving point and the transformation features of one type of neighboring moving point of the target moving point, a single neighbor-related feature of the target moving point may be generated.

[0216] Among them, the process of performing feature embedding processing on the transformation features of the target motion point and the transformation features of a neighbor motion point of the target motion point to generate a neighbor association feature of the target motion point can include: the processing device can call the skinning network to perform splicing processing on the transformation features of the target motion point and the transformation features of a neighbor motion point of the target motion point to obtain a spliced ​​neighbor feature, and then the processing device can call the skinning network to perform feature embedding processing on the spliced ​​neighbor feature to generate a neighbor association feature of the target motion point.

[0217] Using the same principle as above, multiple neighbor association features can be obtained for each moving point, that is, a moving point can have multiple neighbor association features. The processing device can fuse all the neighbor association features of N moving points to generate target neighbor features of the N moving points. The target neighbor features contain the neighbor features of the N moving points. Optionally, the fusion process can be a splicing process or other processing, which can be determined according to the actual application scenario.

[0218] Then, the processing device can perform feature reorganization processing (also called feature transformation processing) on ​​the target neighbor features to generate global features of the moving points.

[0219] Among them, the process of performing feature reorganization processing on the target neighbor features to generate the global features of the motion points may include: the processing device may also obtain the motion point interaction features, which may be obtained after feature interaction processing of the position features of N motion points (such as the features of the first reference positions of the N motion points and the relative position features between the N motion points and the corresponding basic skeletons) and the position features of the basic skeletons corresponding to the N motion points (such as the features of the reference positions of the M basic skeletons).

[0220] The processing device can fuse the aforementioned motion point interaction features and target neighbor features to generate a target motion point feature. Specifically, the feature obtained by fusing the motion point interaction features and target neighbor features can be referred to as the target motion point feature. This fusion process can be a concatenation process or other process, depending on the actual application scenario. Furthermore, the processing device can perform feature renormalization on the target motion point feature to generate a global motion point feature.

[0221] The above-mentioned motion point interaction features can be generated in the interactive process of the process of generating motion point global features and the process of generating bone position features. The process of generating the motion point interaction features is described as follows.

[0222] The processing device can select the neighbor bones of each basic bone from the above-mentioned M basic bones. Optionally, a bone topology map can be formed between the M basic bones, and the bone topology map reflects the structural relationship (such as connection relationship) between the M basic bones. The processing device can select the neighbor bones of each basic bone based on the bone topology map. For example, the bone topology map can be used to select multiple other basic bones that are closest to a basic bone as the neighbor bones of the basic bone.

[0223] Furthermore, the processing device can perform splicing processing on the second reference position of each base bone with the second reference position of each base bone's neighboring bones to generate splicing position information for each base bone. In other words, by splicing the second reference position of a base bone with the second reference position of its neighboring bones, the splicing position information of the base bone can be obtained.

[0224] The processing device can call the skinning network to perform feature embedding processing on the splicing position information of M basic bones (which can be placed in the same feature matrix) to generate transition bone features of the M basic bones. That is, the features obtained after the feature embedding processing of the splicing position information of the M basic bones can be called transition bone features. The dimension of the transition bone feature can be (M, C1), where C1 is a positive integer, that is, one basic bone can correspond to a C1-dimensional feature in the transition bone feature.

[0225] The processing device may obtain the above-mentioned basic features of the N moving points, and the basic features may be obtained by combining and processing the first reference positions of the N moving points and the normal information of the N moving points.

[0226] The processing device can perform feature interaction processing on the basic features and transition bone features of the N motion points, thereby generating the above-mentioned motion point interaction features of the N motion points and the bone interaction features of the M basic bones. The bone interaction features are features generated by interacting with the features of the basic features of the N motion points on the basis of the transition bone features. The motion point interaction features can be features generated by interacting with the features in the transition bone features on the basis of the basic features of the N motion points.

[0227] Optionally, the present application may be a Cross Attention mechanism using Transformer (a sequence model based on an attention mechanism) to perform feature interaction processing on the basic features and transition skeleton features of N motion points. The working principle of the Cross Attention may include: first, the query, key, and value of the N motion points can be decomposed from the basic features of the N motion points, and the query, key, and value of the M basic skeletons can be decomposed from the transition skeleton features. The query represents the query vector of the key, the key represents the key value, and the value represents the feature value corresponding to the key. Therefore, in Cross Attention, the corresponding key can be queried through the query of N motion points and M basic skeletons (the query result is the result of Cross Attention self-learning), and then the attention mechanism is processed on the value of the motion point and the value of the skeleton based on the queried key (i.e., feature interaction), and the above-mentioned motion point interaction feature and skeleton interaction feature can be finally generated.

[0228] The bone interaction feature is the above-mentioned bone position feature of the generated M basic bones.

[0229] Please refer to Figures 12a-12g, Figure 12a is a structural schematic diagram of a skin network provided in an embodiment of the present application, Figure 12b is a structural schematic diagram of a skin network provided in an embodiment of the present application, Figure 12c is a structural schematic diagram of a skin network provided in an embodiment of the present application, Figure 12d is a structural schematic diagram of a skin network provided in an embodiment of the present application, Figure 12e is a structural schematic diagram of a skin network provided in an embodiment of the present application, Figure 12f is a structural schematic diagram of a skin network provided in an embodiment of the present application, and Figure 12g is a structural schematic diagram of a skin network provided in an embodiment of the present application. As shown in Figure 12a, the skin network may include Vertex Feature Representation (vertex feature representation branch), Joint Feature Representation (joint feature representation branch) and Bone-Chain Feature Representation (bone chain feature representation branch).

[0230] Among them, the symbol with a plus sign in the circle in the figure represents splicing. First, the Vertex Feature Representation is explained: the Vertex Feature Representation is used to generate the above-mentioned global features of the motion points. The input of BoneNet in the Vertex Feature Representation can be the position association information between the above-mentioned N motion points and the corresponding basic bones. The output of BoneNet (a neural network for encoding the position information of the basic bones into the position information of the vertices) can be the above-mentioned bone association features. The input of T-Net (a neural network for feature transformation) in the Vertex Feature Representation can be the initial representation features of the N motion points obtained by splicing the bone association features and the basic features of the N motion points. The output of T-Net can be the above-mentioned motion point transformation features.

[0231] Next, the Vertex Feature Representation may include two EdgeConv modules (neural networks for extracting domain features). The first EdgeConv module may include Mesh EdgeConv (a neural network for embedding the transformation features of the motion point with the transformation features of the neighboring motion points of the motion point obtained based on the first method above), KNN EdgeConv (a neural network for embedding the transformation features of the motion point with the transformation features of the neighboring motion points of the motion point obtained based on the second method above), and ClusterEdgeConv (a neural network for embedding the transformation features of the motion point with the transformation features of the neighboring motion points of the motion point obtained based on the third method above). The Mesh EdgeConv, KNN EdgeConv, and ClusterEdgeConv are each used to generate an initial neighbor association feature corresponding to each of the N motion points.

[0232] Furthermore, the second EdgeConv module can also include Mesh EdgeConv (a neural network for embedding the initial neighbor association features of the moving point with the initial neighbor association features of the neighbor moving point of the moving point obtained based on the above first method), KNN EdgeConv (a neural network for embedding the initial neighbor association features of the moving point with the initial neighbor association features of the neighbor moving point of the moving point obtained based on the above second method), and ClusterEdgeConv (a neural network for embedding the initial neighbor association features of the moving point with the initial neighbor association features of the neighbor moving point of the moving point obtained based on the above third method). The Mesh EdgeConv, KNN EdgeConv and ClusterEdgeConv are each used to finally generate a neighbor association feature corresponding to N moving points. Through the above method, a moving point can have three neighbor association features. Here, the initial neighbor association features of the moving point output by the first EdgeConv module and the neighbor association features of the moving point output by the second EdgeConv module can be spliced ​​to obtain the above target neighbor features. The input of the second EdgeConv module can be the feature obtained by splicing the input of the first EdgeConv module and the initial interaction feature of the motion point output by the first InterModule module in the following Joint Feature Representation module.

[0233] Since most of the position features of vertices and basic skeletons have been learned through the above process, we can focus on the structural features between motion points (such as the features between motion points and adjacent motion points). Therefore, a Mesh EdgeConv can be connected after the above second EdgeConv module. Through this Mesh EdgeConv, the features between motion points and adjacent motion points can be extracted at a deeper level. The input of this Mesh EdgeConv can be the target motion point feature obtained by splicing the above target neighbor features and the motion point interaction features output by the second InterModule module in the Joint Feature Representation module.

[0234] Starting from the Mesh EdgeConv after the second EdgeConv module in the Vertex Feature Representation, to the data processing of the target motion point features by the upper and lower MLP network layers connected later (the lower MLP network layer is also connected to a Max Pooling (maximum pooling)), it can be understood as the feature reorganization of the target motion point features. Here, the MLP connected above is used to unify the features output by the EdgeConv in the feature space (because after splicing, it can be understood as a space containing multiple neighbor features), and the MLP and Max Pooling connected below are used to transform and reorganize the features output by the EdgeConv. Finally, the output of the LMP connected above and the output of the Max Pooling connected below are spliced ​​to obtain the above-mentioned global features of the motion point.

[0235] Next, the Joint Feature Representation module is explained: the JointFeature Representation module can be used to generate the above-mentioned bone interaction features. The input of the JointFeature Representation module can be the second reference position of the base bone corresponding to the motion point. The Joint Feature Representation module can include Joint EdgeConv. Through the Joint EdgeConv (a neural network for feature association embedding of base bones and their neighboring bones), the splicing position information of each base bone can be feature embedded to generate transition bone features of M base bones.

[0236] Next, two InterModule modules (neural networks for feature interaction processing) are connected after Joint EdgeConv. The working principles of these two InterModule modules can be the same. Both InterModule modules adopt the working principle of the Cross Attention mechanism of the above-mentioned Transformer. The input of the first InterModule module can be the basic features of N motion points and the transition skeleton features output by the above-mentioned Joint EdgeConv. The output of the first InterModule module can include the initial interaction features of the motion points and the initial interaction features of the basic skeleton. The input of the second InterModule module can be the output of the first InterModule module, and then the above-mentioned motion point interaction features and the above-mentioned skeleton interaction features can be output through the second InterModule module. The motion point interaction features can flow into the above-mentioned Vertex Feature Representation, and the skeleton interaction features are retained in the Joint Feature Representation module. The skeleton interaction features are also the generated skeleton position features.

[0237] Here, for the sake of ease of description, the generation process of the following bone chain structure features is explained. The above-mentioned Bone-Chain Feature Representation is used to generate the bone chain structure features. The Bone-Chain Feature Representation can include two BoneChainNets (neural networks for extracting bone chain features). The input of the BoneChainNet connected above is the first bone chain information described below, and the output of the BoneChainNet can be the first bone chain embedding feature described below; the input of the BoneChainNet connected below is the second bone chain information described below, and the output of the BoneChainNet can be the second bone chain embedding feature described below. The first bone chain embedding feature and the second bone chain embedding feature are fused to generate the bone chain structure features. The fusion process can be a splicing process or other processes, which can be determined according to the actual application scenario.

[0238] Furthermore, the global prediction features generated by the Vertex Feature Representation, the bone position features generated by the Joint Feature Representation, and the bone chain structure features generated by the Bone-Chain Feature Representation can be combined to generate the global prediction features. This global prediction feature is then input into the last connected MLP network layer in the skinning network to predict the kinematic binding parameters between each motion point and the corresponding underlying skeleton.

[0239] The network structure of the BoneNet network in the skinned network in Figure 12a can be shown in Figure 12b. The BoneNet network can perform multi-dimensional transformations on the input features (part of the feature learning process), and ultimately output features of dimension N*64 (or other dimensions). The transformations between the various feature dimensions in the BoneNet can be achieved through linear network layers (linear layers).

[0240] The network structure of the T-Net in the skinning network is shown in Figure 12c. The Vertex Spatial Transform-Net is a neural network that performs feature space transformation. This T-Net transforms the input features into different dimensions, ultimately outputting features with a dimension of N*70 (or other dimensions). Transformations between various feature dimensions in this T-Net can also be implemented using linear layers.

[0241] The structures of the above-mentioned Mesh EdgeConv, KNN EdgeConv and ClusterEdgeConv are the same, and the principle of data processing is the same, except that the types of neighbor motion points used are different. Optionally, the structure can be the structure shown in Figure 12d, which can include Conv2d (convolution layer), Max Pooling (maximum pooling layer) and Graph attention (attention mechanism layer).

[0242] The network structures of the two Inter Module networks in the above-mentioned skinning network can be as shown in Figure 12e. The Inter Module network can include two MLP network layers and two Attention network layers (attention mechanism layers). The input of the upper MLP network layer is the relevant features of the vertices, such as the input of the upper MLP in the first Inter Module network above is the basic features of N vertices, and the input of the lower MLP in the first Inter Module network is the bone transition features; the input of the lower MLP network layer is the relevant features of the basic skeleton, such as the input of the upper MLP in the second Inter Module network above is the initial interaction features of N vertices, and the input of the lower MLP in the second Inter Module network is the initial interaction features of the basic skeleton.

[0243] Optionally, the structure of the Attention network layer in the above-mentioned Inter Module network can be as shown in Figure 12f. The Attention network layer can include MatMul (a matrix multiplication function layer), Scale (scale transformation layer), Mask (mask layer) and softmax (normalization layer).

[0244] The network structure of the BoneChainNet network in the skinning network described above can be shown in Figure 12g. Through this BoneChainNet network, multi-dimensional transformations can be performed on the input features (part of the feature learning process), and ultimately features with dimensions of N*M*64 (or other dimensions) can be output. The transformations between the various feature dimensions in this BoneChainNet are achieved through linear network layers (linear layers).

[0245] Through the above method of the present application, feature interactive learning can be performed on the position features of N motion points and the position features of M basic bones, and ultimately accurate motion point global features for characterizing the global position features between N motion points and the corresponding basic bones, as well as bone position features for characterizing the global position features of M basic bones can be generated.

[0246] Please refer to Figure 13, which is a flow chart of another feature generation method provided in an embodiment of the present application. The embodiment of the present application describes the process of generating the bone chain structure features associated with the above-mentioned M basic skeletons. As shown in Figure 13, the method may include:

[0247] Step S401, obtain the associated joints of each basic bone on the bone chain; any bone chain is composed of multiple basic bones connected to each other, and the joints of any bone chain refer to the bone connections between the basic bones contained in the bone chain. The associated joints of any basic bone include one or more adjacent joints of any basic bone on the bone chain.

[0248] Optionally, a bone chain may be composed of multiple interconnected basic bones, and a bone chain may contain one or more joints. A joint of a bone chain may refer to the bone connection between the basic bones contained in the bone chain (such as the connection between the basic bone vertices).

[0249] The processing device can obtain the associated joints (which can be called upper-level joints) of each basic bone among the M basic bones on the bone chain. The associated joints of any basic bone may include one or more adjacent joints of the basic bone on the bone chain. The number of associated joints of a basic bone can be determined according to the actual application scenario.

[0250] For example, if a base bone has two associated joints, the two associated joints of the base bone can include the parent joint (also called the parent node) and the grandparent joint (also called the grandparent node, i.e. the parent joint of the parent joint of the base bone) of the base bone on the same bone. If the number of associated nodes of the base bone is other numbers, the same principle can be used to obtain the associated nodes of each base bone.

[0251] Step S402: Combine the second reference position of the basic skeleton corresponding to each motion point with the third reference position of the associated joint of the basic skeleton corresponding to each motion point to obtain the first bone chain information associated with each motion point.

[0252] Optionally, it is understood that each joint in the bone chain formed by the aforementioned M basic bones can also have its own reference position (which can also be three-dimensional, and the reference position of the joint can be referred to as a third reference position). This third reference position can also be the starting position of the joint when the first object model and the second object model are in a standard object pose (resting pose). Therefore, the processing device can obtain the third reference position of the joints in the bone chain formed by the M basic bones.

[0253] The processing device can combine the second reference position of the basic skeleton corresponding to each motion point with the third reference position of the associated joint of the basic skeleton corresponding to each motion point (such as splicing) to obtain the first bone chain information associated with each motion point. In other words, by combining the second reference position of the basic skeleton corresponding to any motion point with the third reference position of the associated joint of the basic skeleton, the first bone chain information associated with the motion point can be obtained. If there are 2 associated nodes of a basic skeleton, and the number of basic skeletons corresponding to a motion point is R, the dimension of the first bone chain information associated with a motion point can be (R, 9), that is, the second reference position of a basic skeleton corresponding to a motion point and the third reference position of the 2 associated joints of the basic skeleton can constitute a 9-dimensional information in the first bone chain information.

[0254] Therefore, the total dimension of the first bone chain information associated with N motion points can be expressed as (N, R, 9).

[0255] Step S403: Combine and process the position distance information between each motion point and the corresponding basic skeleton, and the position distance information between each motion point and the associated joint of the corresponding basic skeleton to obtain the second bone chain information associated with each motion point.

[0256] Optionally, the processing device may obtain and calculate the position distance information between each motion point and the corresponding base skeleton. A motion point and a base skeleton corresponding to the motion point may have position distance information, and optionally, the position distance information may include the Euclidean distance (1 dimension) and the geodesic distance (1 dimension) between the first reference position of the motion point and the second reference position of the corresponding base skeleton.

[0257] The processing device may also calculate the positional distance information between each motion point and the associated joint of the corresponding base skeleton. A motion point may have positional distance information with an associated joint of a base skeleton corresponding to the motion point. Optionally, the positional distance information may also include the Euclidean distance (1-dimensional) and the geodesic distance (1-dimensional) between the reference position of the motion point and a third reference position of the associated joint of the corresponding base skeleton.

[0258] The processing device can combine (e.g., concatenate) the positional distance information between each motion point and the corresponding base skeleton, and the positional distance information between each motion point and the associated joint of the corresponding base skeleton, to obtain the second bone chain information associated with each motion point. A motion point can have a second bone chain information, and the second bone chain information can include the positional distance information between the motion point and the corresponding base skeleton, and the positional distance information between the motion point and the associated joint of the corresponding base skeleton.

[0259] Therefore, if a base skeleton has two associated joints, the positional distance information between a motion point and the corresponding base skeleton, as well as the positional distance information between the motion point and the associated joint of the base skeleton, constitutes a 3*2 6-dimensional information. If the number of base skeletons corresponding to each motion point is R, then the dimension of the second bone chain information associated with any motion point can be (R, 6), and the total dimension of the second bone chain information associated with N motion points is (N, R, 6).

[0260] Step S404: Generate bone chain structural features based on the first bone chain information and the second bone chain information associated with each motion point.

[0261] Optionally, the processing device can perform feature embedding processing on the first bone chain information associated with N motion points (the dimensions can be (N, R, 9) in total, and the first bone chain information associated with N motion points can be placed in the same feature matrix) to obtain the first bone chain embedding features associated with the N motion points.

[0262] Similarly, the processing device can perform feature embedding processing on the second bone chain information associated with N motion points (the dimensions can be (N, R, 6) in total, and the second bone chain information associated with N motion points can also be placed in the same feature matrix) to obtain the embedded features of the second bone chain associated with N motion points.

[0263] Furthermore, the processing device can splice the first bone chain embedding feature and the second bone chain embedding feature obtained above to generate the above-mentioned bone chain structural feature. Since the bone chain structural feature is obtained based on the second reference position of the basic skeleton corresponding to the N motion points, the third reference position of the associated joint of the basic skeleton corresponding to the N motion points, and the position distance between the N motion points and the associated joint of the corresponding basic skeleton (which can be reflected by the above-mentioned position distance information), the bone chain structural feature contains the joint information of the bone chain where the basic skeleton corresponding to the N motion points is located. The position of the basic skeleton contained in the bone chain and the position of the joint contained in the bone chain can be used to reflect the structure of the bone chain. Therefore, it can be understood that the bone chain structural feature can be used to reflect the structural characteristics of the bone chain where the basic skeleton corresponding to the N motion points is located.

[0264] Through the above process, feature embedding processing is performed by associating the first reference position of N motion points, the second reference position of the basic skeleton corresponding to the N motion points, and the third reference position of the associated joints of the basic skeleton corresponding to the N motion points, so as to accurately generate the bone chain structure features associated with the N motion points.

[0265] Please refer to Figure 14, which is a flow chart of a network training method provided by an embodiment of the present application. This embodiment of the present application describes the specific process of training to obtain a skinned network. As shown in Figure 14, the method may include:

[0266] Step S501: Acquire a first sample object model, where the first sample object model includes G sample basic bones or part of the G sample basic bones, where G is a natural number.

[0267] Optionally, the processing device may obtain a first sample object model, the concept of which is the same as the concept of the above-mentioned first object model. The first sample object model may include G sample basic bones or include some sample basic bones among the G sample basic bones, the concept of the G sample basic bones is the same as the concept of the above-mentioned M basic bones, and G is a natural number.

[0268] Step S502: Obtain a second sample object model, where the second sample object model includes K sample motion points, each sample motion point corresponding to one or more sample basic bones among G sample basic bones, and the K sample motion points are used to define the motion of the second sample object model. K is a natural number, and the K sample motion points all have sample labels. The sample label of any sample motion point is used to indicate the real motion binding parameters between any sample motion point and the corresponding sample basic bone.

[0269] Optionally, the processing device may also acquire a second sample object model. The concept of this second sample object model is the same as that of the second object model described above, and the second sample object model may follow the motion of the first sample object model. This second sample object model may have K sample motion points, where K is a natural number. The concept of these K sample motion points is the same as that of the N motion points described above. These K sample motion points are used to define (or characterize) the motion of the second sample object model, and are not further described here.

[0270] Therefore, it can be understood that the above-mentioned G sample basic skeletons can include all bones in the first object model that will affect the movement of the K sample motion points, and, according to the requirements of the actual application scenario, the G sample basic skeletons may also include or not include auxiliary bones used to assist the second sample object model in movement.

[0271] Similarly, each sample motion point can correspond to one or more sample basic skeletons in the G sample basic skeletons, and a sample motion point can move along with the corresponding sample basic skeleton. Therefore, the processing device can obtain the sample basic skeleton corresponding to each sample motion point in the G sample basic skeletons.

[0272] Furthermore, each sample motion point can have a sample label. The sample label of any sample motion point is used to indicate the real motion binding parameter between the sample motion point and the corresponding sample basic skeleton. A sample motion point and a corresponding sample basic skeleton can have a real motion binding parameter, and the real motion binding parameter is the correct target that the final predicted motion binding parameter needs to achieve.

[0273] Among them, the real motion binding parameters between the sample motion points and the corresponding sample basic skeletons can be manually marked by professional artists based on the actual motion effects that need to be achieved when each sample motion point of the second sample object model follows the corresponding sample basic skeleton to move.

[0274] Step S503: Determine the fourth reference position of each sample motion point and the fifth reference position of each sample basic skeleton in the G sample basic skeletons.

[0275] Optionally, each sample motion point and each sample base skeleton may have its own reference position. The reference position of the sample motion point may be referred to as the fourth reference position, and the reference position of the sample base skeleton may be referred to as the fifth reference position. The fourth reference position of each sample motion point and the fifth reference position of each sample base skeleton are determined in the same manner as the reference positions of the N motion points and the M base skeletons described above, and are not further described herein.

[0276] Therefore, the processing device can determine the fourth reference position of each sample motion point and the fifth reference position of each sample basic skeleton. The fourth reference position and the fifth reference position can be obtained together when the first sample object model and the second sample object model are obtained.

[0277] Step S504, call the skinning network to be trained, convert the fourth reference position and the fifth reference position, and generate sample global prediction features corresponding to K sample motion points. The sample global prediction features are used to reflect the global position features between the K sample motion points and the corresponding sample basic bones, the position features of the G sample basic bones, and the structural features of the bone chain where the sample basic bones corresponding to the K sample motion points are located.

[0278] Optionally, the processing device can obtain a skinning network to be trained, which is a network used for overall skinning. The processing device can call the skinning network to be trained, and perform feature embedding processing on the fourth reference position of the K sample motion points and the fifth reference position of the G sample basic bones to generate sample global prediction features for the K sample motion points. The sample global prediction features can be used to reflect the global position features between the K sample motion points and the corresponding sample basic bones, the position features of the G sample basic bones, and the structural features of the bone chain where the basic bones corresponding to the K sample motion points are located.

[0279] Among them, the process of calling the skinning network to be trained and performing conversion processing (which can be feature embedding processing) on ​​the fourth reference positions of the K sample motion points and the fifth reference positions of the G sample basic skeletons to generate the global prediction features of the samples is the same as the above-mentioned process of calling the skinning network and performing conversion processing on the first reference positions of the N motion points and the second reference positions of the M basic skeletons to generate the global prediction features, and will not be repeated here.

[0280] The concept of sample global prediction features here is the same as the concept of global prediction features mentioned above.

[0281] Step S505 : calling the skinning network to be trained to predict the sample motion binding parameters between each sample motion point and the corresponding sample basic skeleton based on the sample global prediction features.

[0282] Optionally, the processing device may call the skinning network to be trained to predict the sample motion binding parameters between each sample motion point and the corresponding sample base skeleton based on the above-mentioned sample global prediction features. The process of calling the skinning network to be trained to predict the sample motion binding parameters between each sample motion point and the corresponding sample base skeleton is the same as the process of calling the skinning network to predict the motion binding parameters between each motion point and the corresponding base skeleton, and will not be repeated here.

[0283] Step S506 , based on the difference between the sample motion binding parameters predicted for each sample motion point and the actual motion binding parameters indicated by the sample label of each sample motion point, the network parameters of the skinned network to be trained are corrected to obtain the skinned network.

[0284] Optionally, the sample motion binding parameters between a predicted sample motion point and the corresponding sample basic skeleton can be referred to as the sample motion binding parameters corresponding to the sample motion point (the sample motion binding parameters between the sample motion point and all corresponding sample basic skeletons can be represented by a sequence or vector), and the real motion binding parameters indicated by the sample label of the sample motion point can be referred to as the real motion binding parameters corresponding to the sample motion point (the real motion binding parameters between the sample motion point and all corresponding sample basic skeletons can also be represented by a sequence or vector).

[0285] The processing device can generate the global parameter prediction deviation of the skin network to be trained for the sample motion binding parameters and the motion position prediction deviation for the second sample object model based on the sample motion binding parameters and the real motion binding parameters corresponding to each sample motion point.

[0286] The global parameter prediction deviation directly reflects the difference between the sample motion binding parameters corresponding to each sample motion point and the true motion binding parameters. The larger the global parameter prediction deviation, the greater the difference between the sample motion binding parameters corresponding to each sample motion point and the true motion binding parameters. Conversely, the smaller the global parameter prediction deviation, the smaller the difference between the sample motion binding parameters corresponding to each sample motion point and the true motion binding parameters. The global parameter prediction deviation can be KL-Loss (a divergence loss used to measure the similarity between two distributions), as shown in the following formula:

[0287] Among them, Loss KL That is, the global parameter prediction deviation, G is the number of sample basic bones, K is the number of sample motion points, k is a positive integer less than or equal to K, k is used to represent any one of the K sample motion points, g is a positive integer less than or equal to G, g is used to represent any one of the G sample basic bones. represents the sample motion binding parameter between the predicted sample motion point k and the sample base skeleton g, p kg It represents the true motion binding parameter between the sample motion point k and the sample basic skeleton g (the true motion binding parameter indicated by the sample label of the sample motion point k), and log represents the logarithm.

[0288] The motion position prediction deviation is used to reflect the difference between the positions of the K sample motion points after movement based on the predicted sample motion binding parameters and the positions of the K sample motion points after movement based on the actual motion binding parameters. The motion position prediction deviation can also be called the deformation loss of the second sample object model. As described below, the process of generating the motion position prediction deviation may include:

[0289] The processing device may pre-set several sample object postures for the first sample object model. The sample object postures may be several object postures to which the first sample object model needs to move. The number of the sample object postures may be determined according to the actual application scenario and is not limited thereto.

[0290] Furthermore, the processing device can calculate the predicted motion position of each sample motion point by using the sample motion binding parameters predicted for each sample motion point, the starting position of each sample motion point, and the motion parameters of the sample basic skeleton corresponding to each sample motion point (the motion parameters can be determined based on the sample object posture set above, and the motion parameters can be the translation and transfer matrix of the sample basic skeleton when the first sample object model moves to the sample object posture, and the motion parameters can be used to determine the distance the sample basic skeleton needs to translate and the angle of rotation, etc.), using the LBS algorithm in the above formula (1). The predicted motion position is the position of each sample motion point calculated by using the sample motion binding parameters predicted for each sample motion point, the starting position of each sample motion point, and the motion parameters of the sample basic skeleton corresponding to each sample motion point after the first sample object model moves to the sample object posture. The concept of the starting position of the sample motion point is the same as the concept of the starting position of the above motion point.

[0291] Similarly, the processing device can also use the LBS algorithm in the above formula (1) to calculate the real motion position of each sample motion point through the real motion binding parameters indicated by the sample label of each sample motion point, the starting position of each sample motion point and the motion parameters of the sample basic skeleton corresponding to each sample motion point (determined based on the sample object posture set above).

[0292] Furthermore, the processing device can generate the above-mentioned motion position prediction deviation based on the predicted motion position and the actual motion position of each sample motion point. The motion position prediction deviation can be the MSE (mean square error) between the predicted motion position and the actual motion position of each sample motion point. The mean square error is used to reflect the difference between the predicted motion position and the actual motion position of each sample motion point. The larger the mean square error, the greater the difference, as shown in the following formula:

[0293] Among them, Lossdeformation is the motion position prediction deviation, and pos can represent the true motion position of K sample motion points. It can represent the predicted motion positions of K sample motion points.

[0294] Furthermore, the processing device can also generate a neighborhood parameter prediction deviation for the K sample motion points through the difference between the sample motion binding parameters corresponding to adjacent sample motion points among the above-mentioned K sample motion points and the difference between the real motion binding parameters corresponding to adjacent sample motion points among the K sample motion points. The neighborhood parameter prediction deviation can be used to reflect the deviation between the difference between the sample motion binding parameters predicted for adjacent sample motion points among the K sample motion points and the difference between the real motion binding parameters corresponding to adjacent sample motion points among the K sample motion points.

[0295] Optionally, the process of generating the neighborhood parameter prediction deviation may include:

[0296] First, the processing device can calculate the parameter distance between the sample motion binding parameters corresponding to each sample motion point (the sample motion binding parameters between a sample motion point and all corresponding sample basic bones can be represented by a sequence or a vector) and the sample motion binding parameters corresponding to the adjacent sample motion points of each sample motion point, such as the parameter distance can be L1 distance (Manhattan distance) or vector distance, etc., and the sample motion binding parameters corresponding to a sample motion point and the sample motion binding parameters corresponding to an adjacent sample motion point of the sample motion point can have a parameter distance. The adjacent sample motion points of a sample motion point may include other sample motion points that have a connection relationship with the sample motion point in the mesh of the second sample object model, and the adjacent sample motion points of the sample motion point may belong to the neighboring sample motion points of the sample motion point.

[0297] Furthermore, the processing device can obtain the predicted parameter distances corresponding to the K sample motion points through the parameter distances between the sample motion binding parameters corresponding to the above-mentioned sample motion points and the sample motion binding parameters corresponding to the adjacent sample motion points of each sample motion point. The predicted parameter distances also include the parameter distances between the sample motion binding parameters corresponding to the K sample motion points and the sample motion binding parameters corresponding to the adjacent sample motion points of the K sample motion points. The predicted parameter distances can be understood as a set composed of the parameter distances between the sample motion binding parameters corresponding to the K sample motion points and the sample motion binding parameters corresponding to the adjacent sample motion points of the K sample motion points.

[0298] Similarly, the processing device can also calculate the parameter distance between the real motion binding parameters corresponding to each sample motion point and the real motion binding parameters corresponding to the adjacent sample motion points of each sample motion point. The parameter distance can also be L1 distance (Manhattan distance) or vector distance, etc. There can also be a parameter distance between the real motion binding parameters corresponding to a sample motion point and the real motion binding parameters corresponding to an adjacent sample motion point of the sample motion point.

[0299] Furthermore, the processing device can obtain the real parameter distances corresponding to the K sample motion points through the parameter distances between the real motion binding parameters corresponding to the above-mentioned sample motion points and the real motion binding parameters corresponding to the adjacent sample motion points of each sample motion point. The real parameter distances also include the parameter distances between the real motion binding parameters corresponding to the K sample motion points and the real motion binding parameters corresponding to the adjacent sample motion points of the K sample motion points. The real parameter distances can be understood as a set composed of the parameter distances between the real motion binding parameters corresponding to the K sample motion points and the real motion binding parameters corresponding to the adjacent sample motion points of the K sample motion points.

[0300] The above-mentioned neighborhood parameter prediction deviation can be the MSE (mean square error) between the above-mentioned prediction parameter distance and the true parameter distance. The mean square error can be used to reflect the difference between the prediction parameter distance and the true parameter distance. The larger the mean square error, the greater the difference between the prediction parameter distance and the true parameter distance. Therefore, it can also be understood that the neighborhood parameter prediction deviation can be generated by the difference between the prediction parameter distance and the true parameter distance. As shown in the following formula, the neighborhood parameter prediction deviation can be:

[0301] Among them, Loss ngb is the neighborhood parameter prediction deviation, D represents the above true parameter distance, Represents the above prediction parameter distance.

[0302] Then, the processing device can add the above-mentioned global parameter prediction deviation, motion position prediction deviation and neighborhood parameter prediction deviation (which can be weighted and then added) to obtain the target prediction deviation for the K sample motion points. The target prediction deviation is the final overall loss (which can be a loss function) of the sample motion binding parameters predicted by the skin network to be trained for the K sample motion points. As shown in the following formula: LOSS mb =w1*Loss KL +w2*Lossdeformation+w3*Loss ngb (7)

[0303] Among them, LOSS mb That is the target prediction deviation, Loss KL is the global parameter prediction deviation, Lossdeformation is the motion position prediction deviation, Loss ngb is the prediction deviation of the neighborhood parameter. w1 is Loss KL The weight coefficient of w2 is the weight coefficient of Lossdefoemation, w3 is the weight coefficient of Loss ngb The weight coefficient of .

[0304] Among them, the specific values ​​of the above w1, w2 and w3 can be determined according to actual application requirements, such as the relative importance between the global parameter prediction deviation, the motion position prediction deviation and the neighborhood parameter prediction deviation. The more important the prediction deviation, the higher the weight coefficient can be set. Alternatively, the weight coefficients of the global parameter prediction deviation, the motion position prediction deviation and the neighborhood parameter prediction deviation can all be set to 1 or to a specific value less than 1 (such as 0.3). The specific value can be determined according to the actual application scenario and is not limited to this.

[0305] The processing device can correct the network parameters of the skin network to be trained by the target prediction deviation obtained above. The goal of the correction is to make the target prediction deviation tend to the minimum value (such as tend to 0). When the network parameters of the skin network to be trained are corrected (such as correcting the network parameters of the skin network to be trained to a convergence state, or the number of iterative training times of the skin network to be trained using the above principle reaches a specified threshold), the trained skin network (which can be referred to as the skin network) can be obtained.

[0306] By adopting the above-mentioned method of the present application, a variety of prediction deviations of the skin network to be trained can be generated through the difference between the sample motion binding parameters corresponding to the sample motion points and the real motion binding parameters, so as to realize the evaluation and consideration of the sample motion binding parameters predicted by the skin network to be trained from multiple dimensions, and finally the network parameters of the skin network to be trained can be accurately corrected to obtain an accurate skin network through training. The skin network obtained by the training can realize accurate skinning between the above-mentioned first object model and the second object model.

[0307] Please refer to Figure 15, which is a flow chart of a local skinning method provided in an embodiment of the present application. The embodiment of the present application describes the specific process of performing specific skinning on a part of the second object model. Since the second object model is skinned as a whole through the above-mentioned skinning network, in some specific scenarios, in order to meet the skinning of the local motion points of the second object model with specific effects, the present application also provides the following local skinning method, which can meet the animator's more specific needs for skinning the object model. As shown in Figure 15, the method may include:

[0308] Step S601: Select multiple local motion points from N motion points, and select multiple local basic bones associated with the multiple local motion points from M basic bones.

[0309] Optionally, the processing device may select multiple local motion points from the N motion points, where the multiple local motion points are the motion points from the N motion points that require local skinning. Optionally, the image of the first object model carrying the second object model may be visualized on an animator's terminal device, and the multiple local motion points may be selected by the animator from the visualized image of the first object model carrying the second object model. The multiple local motion points selected by the animator may then be sent by the terminal device to the processing device.

[0310] The processing device may also select multiple local basic bones associated with the multiple local motion points from the M basic bones. The multiple local basic bones are basic bones from the M basic bones that affect the motion of the multiple local motion points. Similarly, the multiple local basic bones may also be selected by an animator in a visualization of the first object model carrying the second object model. The multiple local basic bones selected by the animator may then be sent by the terminal device to the processing device.

[0311] Subsequently, specific local skinning can be performed between the above-mentioned multiple local motion points and multiple local basic bones.

[0312] Users can also configure local skinning in the skinning plug-in. See Figure 16, which is a schematic diagram of an interface for configuring local skinning provided by an embodiment of the present application. In the configuration interface shown in Figure 16, users can select vertices (such as the aforementioned multiple local motion points) and base bones (such as the aforementioned multiple local base bones) to be locally skinned.

[0313] Step S602 : performing feature embedding processing on the first reference positions of the plurality of local motion points to generate initial motion point features of the plurality of local motion points.

[0314] Optionally, the processing device can perform feature embedding processing on the first reference positions of the above-mentioned multiple local motion points (which can be placed in the same feature matrix) to generate initial motion point features of the multiple local motion points, that is, the features obtained after performing feature embedding processing on the first reference positions of the multiple local motion points can be called initial motion point features.

[0315] Step S603: performing feature embedding processing on the second reference positions of the multiple local basic bones to generate initial bone features of the multiple local basic bones.

[0316] Optionally, the processing device may also perform feature embedding processing on the second reference positions of the above-mentioned multiple local basic bones (which may be placed in the same feature matrix) to generate initial bone features of the multiple local basic bones, that is, the features obtained after performing feature embedding processing on the second reference positions of the multiple local basic bones may be called initial bone features.

[0317] Step S604 , performing feature interaction processing on the initial features of the skeleton and the initial features of the motion points to generate first interaction features of multiple local motion points and second interaction features of multiple local basic skeletons.

[0318] Optionally, the processing device may perform feature interaction processing on the above-mentioned initial features of the skeleton and the initial features of the motion points to generate interaction features of the above-mentioned multiple local motion points (which may be referred to as first interaction features) and interaction features of the above-mentioned multiple local basic bones (which may be referred to as second interaction features).

[0319] Among them, the principle of feature interaction processing is the same as that of the initial representation features of the N motion points and the transition bone features of the M basic bones. Here, the principle of Cross Attention of Transformer can also be used to perform feature interaction processing on the initial features of the bones and the initial features of the motion points.

[0320] The first interactive feature is a feature obtained by interacting with the features in the initial features of the skeleton on the basis of the initial features of the motion points. The second interactive feature is a feature obtained by interacting with the features in the initial features of the motion points on the basis of the initial features of the skeleton.

[0321] Step S605 : predicting local motion binding parameters between each local motion point and each local basic skeleton based on the first interaction feature and the second interaction feature.

[0322] Optionally, the processing device can generate a global feature (which can be called a first global feature, and the first global feature can be a feature vector) of the above-mentioned multiple local motion points through the above-mentioned first interaction feature and the second interaction feature, that is, multiple local motion points can share this first global feature.

[0323] The processing device can also generate a global feature of the above-mentioned multiple local basic bones (which can be called a second global feature, and the second global feature can also be a feature vector) through the above-mentioned first interaction feature and the second interaction feature, that is, multiple local basic bones can share this second global feature.

[0324] Furthermore, the processing device generates a first local feature of the above-mentioned multiple local motion points (the first local feature can be a feature matrix) through the above-mentioned first interaction feature and the second interaction feature. The first local feature can include the local feature of each local motion point. In other words, each local motion point can have its own local feature in the first local feature instead of sharing the same local feature.

[0325] Similarly, the processing device generates a second local feature of the above-mentioned multiple local basic bones (the second local feature can also be a feature matrix) through the above-mentioned first interaction feature and the second interaction feature. The second local feature can include the local feature of each local basic bone. In other words, each local basic bone can have its own local feature in the second local feature instead of sharing the same local feature.

[0326] The processing device may further calculate an interaction feature distance between the first interaction feature and the second interaction feature. Optionally, the interaction feature distance may be a square distance (Square Distance) between the first interaction feature and the second interaction feature.

[0327] Furthermore, the processing device can predict the motion binding parameters (which may be called local motion binding parameters) between each local motion point and each local basic skeleton through the above-mentioned interaction feature distance, the above-mentioned first global feature, the above-mentioned first local feature, the above-mentioned second global feature and the above-mentioned second local feature, as described below.

[0328] The dimension of the first global feature may be (1, C2). If the number of the plurality of local motion points is Y1, the dimension of the first local feature may be (Y1, C2), where both Y1 and C2 are positive integers. Therefore, in order to keep the dimension of the first global feature consistent with the dimension of the first local feature, the present application may also repeat the first global feature Y1 times to obtain a global feature of dimension (Y1, C2) (which may be referred to as the repeated first global feature). The repeated first global feature then contains Y1 identical first global features.

[0329] Furthermore, the processing device may perform a splicing process on the repeated first global feature and the first local feature to generate a splicing feature of multiple local motion points. The splicing feature may be referred to as a first splicing feature.

[0330] Similarly, the dimension of the second global feature can also be (1, C2). If the number of the multiple local basic bones is Y2, the dimension of the second local feature can be (Y2, C2). Therefore, in order to keep the dimension of the second global feature consistent with the dimension of the second local feature, the present application can also repeat the second global feature Y2 times to obtain a global feature with a dimension of (Y2, C2) (which can be called the repeated second global feature). The repeated second global feature contains Y2 identical second global features.

[0331] Furthermore, the processing device may perform splicing processing on the repeated second global feature and the second local feature to generate a splicing feature of multiple local basic skeletons, which may be referred to as a second splicing feature.

[0332] The processing device may also obtain an adjacency matrix (which may be referred to as a first adjacency matrix) of the plurality of local motion points. The first adjacency matrix may be used to reflect the connection relationship between the plurality of local motion points in the mesh of the second object model. If two local motion points are connected in the mesh of the second object model, the value of the same position of the two local motion points in the first adjacency matrix is ​​1. Conversely, if the two local motion points are not connected in the mesh of the second object model, the value of the same position of the two local motion points in the first adjacency matrix is ​​0.

[0333] Similarly, the processing device can also obtain the adjacency matrix of the above-mentioned multiple local basic bones (which can be called the second adjacency matrix), and the second adjacency matrix can be used to reflect the connection relationship of the multiple local basic bones in the first object model. If two local basic bones are connected in the first object model, the value of the same position of the two local basic bones in the second adjacency matrix is ​​1. Conversely, if the two local basic bones are not connected in the first object model, the value of the same position of the two local basic bones in the second adjacency matrix is ​​0.

[0334] Furthermore, the processing device can predict the local motion binding parameters between each local motion point and each local basic skeleton using the first splicing feature, the second splicing feature, the first adjacency matrix, the second adjacency matrix, and the interaction feature distance, as described below.

[0335] The processing device may perform a product operation on the first splicing feature and the first adjacency matrix to generate an adjacency feature of the multiple local motion points. This adjacency feature may be referred to as the first adjacency feature. Optionally, after performing the product operation on the first splicing feature and the first adjacency matrix, an average pooling operation may be performed on the result of the product operation to generate the first adjacency feature.

[0336] The processing device may also perform a product operation on the second splicing feature and the second adjacency matrix to generate an adjacency feature of the multiple local basic bones. The adjacency feature may be referred to as the second adjacency feature. Optionally, after performing the product operation on the second splicing feature and the second adjacency matrix, an average pooling operation (Avg Pooling) may be performed on the result of the product operation to generate the second adjacency feature.

[0337] Then, the processing device can generate relative adjacency features between multiple local motion points and multiple local basic bones through the difference between the above-mentioned first adjacency feature and the second adjacency feature. Optionally, the relative adjacency feature can include the square distance (Square Distance) between the first adjacency feature and the second adjacency feature and / or the difference between the first adjacency feature and the second adjacency feature (the first adjacency feature and the second adjacency feature can both be expressed as feature matrices, and the difference can be a feature matrix obtained by subtracting the two feature matrices). The relative adjacency feature can be obtained by splicing the square distance between the first adjacency feature and the second adjacency feature and the difference between the first adjacency feature and the second adjacency feature.

[0338] The processing device can splice the above-mentioned relative adjacent features and the above-mentioned interactive feature distances to generate a feature that is ultimately used to predict the local motion binding parameters between the above-mentioned multiple local motion points and multiple local basic bones. This feature can be called a local prediction feature.

[0339] Finally, the processing device can predict (i.e., determine) the local motion binding parameters between each local motion point and each local basic bone through the above-mentioned local prediction features. A local motion binding parameter can be predicted between a local motion point and a local basic bone.

[0340] Among them, it can be understood that the above-mentioned process of generating related features such as initial features of motion points, initial features of bones, first interaction features, second interaction features, first local features, first global features, second local features, second global features, first splicing features, second splicing features, first adjacent features, second adjacent features, relative adjacent features and local prediction features, as well as the process of predicting local motion binding parameters between local motion points and local basic bones based on local prediction features, can all be obtained by calling the trained local skinning network by the processing device, and the principle of training the local skinning network can be the same as the principle of training the skinning network mentioned above.

[0341] For example, multiple local sample motion points (the same concept as the multiple local motion points mentioned above, which can be some motion points selected from the K sample motion points of the above-mentioned second sample object model) and multiple local sample basic bones (the same concept as the multiple local basic bones mentioned above, which can be some sample basic bones selected from the G sample basic bones of the above-mentioned first sample object model) can be used to train the local skinning network to be trained. The multiple local sample motion points can have local sample labels, and the local sample label of any local sample motion point can be used to indicate the real local motion binding parameters between the any local sample motion point and each local sample basic bone. The real local motion binding parameters indicated by the local sample label of the local sample motion point can be different from the real motion binding parameters indicated by the sample label of the local sample motion point.

[0342] Therefore, the processing device can call the local skinning network to be trained, and predict the local motion binding parameters between each local sample motion point and each local sample basic skeleton based on the same principle as predicting the local motion binding parameters between each local motion point and each local basic skeleton.

[0343] Furthermore, the processing device can generate the target prediction deviation of the skin network to be trained based on the difference between the sample motion binding parameters predicted by the skin network to be trained and the real motion binding parameters indicated by the sample labels of the sample motion points. The same principle can be used to generate the local prediction deviation of the local skin network to be trained based on the difference between the local motion binding parameters predicted by the local skin network to be trained and the real local motion binding parameters indicated by the local sample labels of the local sample motion points. That is, the principle of generating the local prediction deviation is the same as the principle of generating the above-mentioned target prediction deviation. Furthermore, the processing device can correct the network parameters of the local skin network to be trained through the local prediction deviation, and obtain a trained local skin network. Through this local skin network, the prediction of the local motion binding parameters between the above-mentioned multiple local motion points and multiple local basic bones can be realized.

[0344] It can be understood that since the local skin network obtained by the above training can achieve the purpose of the animator to make the local motion points in the second object model follow the local basic skeleton to perform specific movements, the above-mentioned local skin network to be trained can be trained based on the actual needs of the animator (such as the need for the local motion points in the second object model to follow the local basic skeleton to perform specific movements to achieve a certain specific movement effect). For example, specific training of the local skin network to be trained can be achieved by setting specific local sample labels for local sample motion points.

[0345] For example, the animator can negotiate with the artist so that the artist can set corresponding local real motion binding parameters between the local sample motion point and the local sample base bone according to the specific effect that the animator wants the local motion point in the second object model to follow the local base bone to perform a specific movement, that is, set a specific local sample label for the local sample motion point.

[0346] Through the above-described method, the local skinning network obtained by the above-described specific training method can be used to achieve a specific movement with a specific effect by allowing the local motion points in the second object model to follow the local base skeleton. Since the above-described local skinning network is used to perform local skinning between the first object model and the second object model, and the above-described skinning network is used to perform overall skinning (i.e., global skinning) between the first object model and the second object model, the computational complexity of the local skinning network is less than that of the skinning network. Therefore, the network parameters of the local skinning network can also be less than the network parameters of the skinning network.

[0347] Please refer to Figures 17a-17d, Figure 17a is a structural schematic diagram of a local skin network provided in an embodiment of the present application, Figure 17b is a structural schematic diagram of a local skin network provided in an embodiment of the present application, Figure 17c is a structural schematic diagram of a local skin network provided in an embodiment of the present application, and Figure 17d is a structural schematic diagram of a local skin network provided in an embodiment of the present application. As shown in Figure 17a, the local skin network can include 6 EdgeConvMoudle (i.e., EdgeConv modules), 2 InterMoudle networks, and 2 T-Net networks, wherein the EdgeConv modules, InterMoudle networks, and T-Net networks in the local skin network can be the same as the EdgeConv modules, InterMoudle networks, and T-Net networks in the above-mentioned skin network (the structures can be the same), and the principles of their data processing are also the same, which will not be repeated here.

[0348] Among them, the output of the T-Net network connected above in the local skin network can be the above-mentioned initial features of the motion points, and the output of the T-Net network connected below in the local skin network can be the above-mentioned initial features of the skeleton.

[0349] The six EdgeConvModules described above have three combined parts. The first combined part has two EdgeConvModules, one above and one below. The second combined part also has two EdgeConvModules, and the third combined part also has two EdgeConvModules. The upper EdgeConvModule in each combined part is used to extract neighbor features related to local motion points, and the lower EdgeConvModule in each combined part is used to extract neighbor features related to local basic bones.

[0350] The square distance 1 here can be the square distance between the neighbor features of the local motion point obtained from the first combination and the neighbor features of the local basic skeleton, the square distance 2 can be the square distance between the neighbor features of the local motion point obtained from the second combination and the neighbor features of the local basic skeleton, and the square distance 3 can be the square distance between the neighbor features of the local motion point obtained from the third combination and the neighbor features of the local basic skeleton.

[0351] InterMoudle in the local skinning network can be used to perform feature interaction between the neighbor features of the local basic skeleton and the neighbor features of the local motion point. Here, the features related to the local motion point output of the part of the third combination mentioned above can be called the first interaction feature, and the features related to the local basic skeleton output of the part of the third combination can be called the second interaction feature.

[0352] The local skinning network can also include GlobalModule (a neural network for extracting global features), LocalModule (a neural network for extracting local features), and RelationUnit (a neural network for feature mixing (fusion)).

[0353] The features related to the local motion points obtained from the first combination, the features related to the local motion points obtained from the second combination, and the features related to the local motion points obtained from the third combination (i.e., the first interactive features) can be spliced ​​together to obtain a spliced ​​feature of the local motion point. The spliced ​​feature can be input into the GlobalModule, and the above-mentioned first global feature of the local motion point can be generated through the GlobalModule.

[0354] Similarly, the features related to the local basic skeleton obtained from the first combination, the features related to the local basic skeleton obtained from the second combination, and the features related to the local basic skeleton obtained from the third combination (i.e., the second interactive features) can be spliced ​​together to obtain a spliced ​​feature of the local basic skeleton. The spliced ​​feature can be input into the GlobalModule, and the above-mentioned second global feature of the local basic skeleton can be generated through the GlobalModule.

[0355] In addition, the features related to the local motion points obtained from the first combination, the features related to the local motion points obtained from the second combination, and the features related to the local motion points obtained from the third combination (i.e., the first interactive features) can be spliced ​​together to obtain a spliced ​​feature of the motion point. The spliced ​​feature can be input into the LocalModule, and the above-mentioned first local feature of the local motion point can be generated through the LocalModule.

[0356] Similarly, the local basic skeleton-related features obtained from the first combination, the local basic skeleton-related features obtained from the second combination, and the local basic skeleton-related features obtained from the third combination (i.e., the second interactive features) can be spliced ​​together to obtain a spliced ​​feature of the local basic skeleton. The spliced ​​feature can be input into the LocalModule, and the above-mentioned second local feature of the local basic skeleton can be generated through the LocalModule.

[0357] Furthermore, the process of generating the above-mentioned relative adjacency features through the above-mentioned first local features, second local features, first global features, second global features, first adjacency matrix and second adjacency matrix can be realized by the RelationUnit network layer, that is, the input of the RelationUnit network layer can be the above-mentioned first local features, second local features, first global features, second global features, first adjacency matrix and second adjacency matrix, and the output of the RelationUnit network layer can be the above-mentioned relative adjacency features.

[0358] Furthermore, the above-mentioned relative adjacency features, square distance 1, square distance 2 and square distance 3 (the square distance 3 can be called the above-mentioned interaction feature distance) can be spliced ​​to generate the final local prediction feature, and the local prediction feature is input into the last connected MLP network layer in the local skinning network to predict the local motion binding parameters between each local motion point and each local basic skeleton.

[0359] Among them, the network structure of the above-mentioned GlobalModule network layer can be shown in Figure 17b. The GlobalModule network layer can include two MLP network layers (network parameters can be shared) and two max pooling network layers (maximum pooling layers).

[0360] The network structure of the above-mentioned LocalModule network layer can be shown in Figure 17c. The LocalModule network layer can also include two MLP network layers (network parameters can be shared), but does not include a max pooling network layer.

[0361] The network structure of the above-mentioned RelationUnit network layer can be shown in Figure 17d. The RelationUnit network layer can include 3 Avg pooling network layers (average pooling layers) and an MLP. The output of the first Avg pooling network layer above the RelationUnit network layer can be called the first adjacent feature, the output of the lower Avg pooling network layer below the RelationUnit network layer can be called the second adjacent feature, and the feature after splicing the output of the second Avg pooling network layer above the RelationUnit network layer and the square distance in the lower branch of the RelationUnit network layer (which can be the square distance between the first adjacent feature and the second adjacent feature) can be called the relative adjacent feature.

[0362] Optionally, the above-mentioned skin network and local skin network in the present application can also be a network based on the Transformer (a neural network with a multi-head attention mechanism) structure.

[0363] Step S606: Update the predicted motion binding parameters between each local motion point and the corresponding basic skeleton into local motion binding parameters between each local motion point and each local basic skeleton.

[0364] Optionally, the processing device may update (i.e., replace) the motion binding parameters (such as initial motion binding parameters) between each local motion point and the corresponding base bone predicted by the skinning network with local motion binding parameters (also referred to as motion binding parameters) between each local motion point and each local base bone predicted by the local skinning network.

[0365] It can be understood that if it is also necessary to optimize the motion binding parameters predicted by the network (such as the skin network and the local skin network) through the corresponding straight wiring (that is, the relevant description of the collinear optimization part in the embodiment corresponding to Figure 3 above) to achieve collinear optimization of the motion points, then it can also be the local motion binding parameters between each local motion point predicted by the local skin network and each local basic bone, and the motion binding parameters after replacing (updating) the motion binding parameters between each local motion point predicted by the skin network and the corresponding basic bone are optimized.

[0366] Please refer to Figures 18a and 18b. Figure 18a is a schematic diagram of a partial skinning effect provided in an embodiment of the present application, and Figure 18b is a schematic diagram of a partial skinning effect provided in an embodiment of the present application. For example, if the second object model is the clothing in Figure 18a, the left side may be the image effect without partial skinning of the hem within the circle of the second object model, while the right side may be the image effect after partial skinning of the hem within the circle of the second object model, with the hem within the circle on the right being in a more regular drooping state.

[0367] For another example, if the second object model is the clothing in FIG18b , the left side may be the picture effect without partially skinning the hem of the skirt within the circle of the second object model, and the right side may be the picture effect after partially skinning the hem of the skirt within the circle of the second object model, and the hem of the skirt within the circle on the right is also in a relatively regular drooping state.

[0368] Through the above process of the present application, first, global and general accurate skinning can be performed between the first object model and the second object model through the skinning network, and then, specific local skinning can be performed between the first object model and the second object model through the local skinning network. In this way, various requirements for skinning the first object model and the second object model in actual application scenarios can be met, thereby improving the richness, accuracy and skinning effect of the skinning between the first object model and the second object model.

[0369] Optionally, the above method provided by the present application can be applied to any scenario where an object model needs to follow the movement of another object model. The following examples illustrate relevant scenarios in which the present application can be applied, but are not limited to the relevant scenarios described below.

[0370] The above method provided by the present application can be applied in game scenes. The above first object model and the second object model can both be virtual objects modeled in the game, and the second object model can be an object that needs to follow the first object model to move in the game. For example, the first object model can be a virtual game character or game monster in the game, and the second object model can be virtual clothing worn by the game character or game monster, or virtual items (such as accessories) carried by the game character or game monster. Therefore, the above method provided by the present application can be used to realize the skinning operation between the game character in the game and the virtual clothing worn by the game character, and then, the movement of the game character can also drive the movement of the virtual clothing worn by the game character, realizing the overall coordinated movement between the game character and the clothing worn by the game character.

[0371] The above-mentioned method provided by the present application can also be applied in animation generation scenarios. The above-mentioned first object model and second object model can be virtual objects in an animation (such as a cartoon or an animated film, etc.), and the second object model can be an object that needs to follow the first object model to move in the animation. For example, the first object model can be a 3D (3-dimensional) animated character modeled in the animation, and the second object model can be a simulated piece of clothing worn by the animated character, or a special effects costume that follows the animated character to move, etc. The specific operation can be determined according to the actual application scenario. Therefore, the above-mentioned method provided by the present application can also realize the skinning operation between the animated character in the animation and the simulated clothing worn by the animated character, and then, the movement of the animated character can also drive the movement of the simulated clothing worn by the animated character, realizing the overall coordinated movement between the animated character and the clothing worn by the animated character.

[0372] 19 is a schematic diagram of a motion data processing device according to an embodiment of the present invention. The motion data processing device 190 may include: a first acquisition module 1901 , a second acquisition module 1902 , a first determination module 1903 , a conversion module 1904 , and a second determination module 1905 .

[0373] A first acquisition module 1901 is configured to acquire a first object model, where the first object model includes M basic bones or a portion of the M basic bones, where M is a natural number.

[0374] A second acquisition module 1902 is configured to acquire a second object model, where the second object model includes N motion points, wherein each of the N motion points corresponds to one or more basic bones of the M basic bones, and the N motion points are used to define the motion of the second object model, where N is a natural number;

[0375] A first determining module 1903 is configured to determine a first reference position of each motion point and a second reference position of each of the M basic skeletons;

[0376] A conversion module 1904 is configured to convert the first reference position and the second reference position to generate global prediction features corresponding to the N motion points. The global prediction features are configured to reflect the global position features between the N motion points and the corresponding base bones, the position features of the M base bones, and the structural features of the bone chains of the base bones corresponding to the N motion points.

[0377] The second determination module 1905 is used to determine the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features. Any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0378] Optionally, the global prediction features include global features of N motion points, where the global features are used to reflect the global position features between the N motion points and the corresponding basic skeletons. The process of generating the global features of the motion points by the conversion module 1904 includes:

[0379] Based on the first reference position of each motion point and the second reference position of the base skeleton corresponding to each motion point, position association information between each motion point and the corresponding base skeleton is generated respectively;

[0380] Perform feature embedding processing on the positional association information between N motion points and the corresponding basic skeletons to generate skeleton association features of the N motion points;

[0381] Performing feature transformation processing on the skeleton-related features and the first reference positions of the N motion points to generate motion point transformation features of the N motion points;

[0382] Based on the motion point transformation features, the motion point global features are generated.

[0383] Optionally, the conversion module 1904 performs feature transformation processing on the skeleton-related features and the first reference positions of the N motion points to generate motion point transformation features of the N motion points, including:

[0384] Combine the first reference positions of the N moving points and the normal information of the N moving points to generate basic features of the N moving points;

[0385] The basic features and skeleton-related features are fused to generate the initial representation features of N motion points;

[0386] Perform feature transformation processing on the initial representation features to generate motion point transformation features.

[0387] Optionally, any one of the N moving points is a target moving point, and the moving point transformation feature includes a transformation feature of each moving point;

[0388] The conversion module 1904 generates a global feature of a moving point based on the moving point transformation feature, including:

[0389] Using multiple neighbor selection methods, multiple neighboring moving points of the target moving point are selected from N moving points;

[0390] Performing feature embedding processing on the transformation features of the target motion point and the transformation features of each neighbor motion point to generate multiple neighbor association features of the target motion point; performing feature embedding processing on the transformation features of the target motion point and the transformation features of one neighbor motion point to generate one neighbor association feature of the target motion point;

[0391] The neighbor correlation features of N moving points are fused to generate the target neighbor features of N moving points;

[0392] The target neighbor features are reshaped to generate the global features of the moving points.

[0393] Optionally, the conversion module 1904 performs feature reorganization processing on the target neighbor features to generate a global feature of the moving point, including:

[0394] Obtain motion point interaction features, and fuse the target neighbor features and motion point interaction features to generate target motion point features. The motion point interaction features are obtained by performing feature interaction processing on the position features of N motion points and the position features of M bones.

[0395] The target motion point features are reshaped to generate the global features of the motion points.

[0396] Optionally, any one of the N moving points is a target moving point;

[0397] Among them, the position association information between the target motion point and the corresponding basic skeleton includes at least one of the following: relative position information between the target motion point and the corresponding basic skeleton, the absolute distance between the target motion point and the corresponding basic skeleton, and the shortest path distance between the target motion point and the corresponding basic skeleton.

[0398] Optionally, the global prediction feature includes bone position features of M basic bones, and the bone position features are used to reflect the position features of the M bones. The process of generating the bone position features by the conversion module 1904 includes:

[0399] Select the neighbor bones of each basic bone from M basic bones;

[0400] Splicing the second reference position of each basic skeleton with the second reference position of the neighboring skeleton of each basic skeleton to generate splicing position information of each basic skeleton;

[0401] Perform feature embedding processing on the splicing position information of the M basic bones to generate transition bone features of the M basic bones;

[0402] Generate bone position features based on transition bone features.

[0403] Optionally, the conversion module 1904 generates the bone position feature based on the transition bone feature by:

[0404] Obtaining basic features of the N moving points, where the basic features are obtained by combining the first reference positions of the N moving points and the normal information of the N moving points;

[0405] Perform feature interaction processing on the basic features and transition skeleton features to generate motion point interaction features of N motion points and skeleton interaction features of M basic skeletons;

[0406] Among them, the bone position feature is the bone interaction feature.

[0407] Optionally, the global prediction feature includes a bone chain structure feature associated with N motion points. The bone chain structure feature is used to reflect the structure feature of the bone chain of the basic skeleton corresponding to the N motion points. The process of generating the bone chain structure feature by the conversion module 1904 includes:

[0408] Get the associated joints of each basic bone on its bone chain. Any bone chain is composed of multiple interconnected basic bones. The joints of any bone chain refer to the bone connections between the basic bones included in the bone chain. The associated joints of any basic bone include one or more adjacent joints of the basic bone on its bone chain.

[0409] Combine the second reference position of the basic skeleton corresponding to each motion point with the third reference position of the associated joint of the basic skeleton corresponding to each motion point to obtain first bone chain information associated with each motion point;

[0410] Combine and process the position distance information between each motion point and the corresponding basic skeleton, and the position distance information between each motion point and the associated joint of the corresponding basic skeleton, to obtain the second bone chain information associated with each motion point;

[0411] A bone chain structure feature is generated based on the first bone chain information and the second bone chain information associated with each motion point.

[0412] Optionally, the conversion module 1904 generates a bone chain structure feature based on the first bone chain information and the second bone chain information associated with each motion point, including:

[0413] Perform feature embedding processing on the first bone chain information associated with the N motion points to generate embedding features of the first bone chain associated with the N motion points;

[0414] Perform feature embedding processing on the second bone chain information associated with the N motion points to generate embedded features of the second bone chain associated with the N motion points;

[0415] The first bone chain embedding feature and the second bone chain embedding feature are fused to generate a bone chain structure feature.

[0416] Optionally, global prediction features are generated by calling the skinning network;

[0417] The second determination module 1905 determines the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features, including:

[0418] The skinning network is called to predict the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features.

[0419] Optionally, the second determination module 1905 calls the skinning network to predict the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features, including:

[0420] The skinning network is called to predict the initial motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features;

[0421] Calculating predicted motion positions of the N motion points under multiple target object poses of the first object model based on the predicted initial motion binding parameters for each motion point;

[0422] Obtaining a straight wiring formed by N moving points in a plurality of target object postures, and determining target posture positions of the N moving points in the plurality of target object postures based on the straight wiring;

[0423] Based on the differences between the predicted motion positions of the N motion points under various target object postures and the target posture positions, the initial motion binding parameters between the N motion points and the corresponding basic skeletons are optimized to generate motion binding parameters between the N motion points and the corresponding basic skeletons;

[0424] Among them, the process of optimizing the initial motion binding parameters between N motion points and corresponding basic skeletons includes: approximating the predicted motion positions of the N motion points under multiple target object postures to the target posture positions of the N motion points under multiple target object postures.

[0425] Optionally, the motion data processing device 190 further includes a training module 1906, which is configured to:

[0426] Acquire a first sample object model, where the first sample object model includes G sample basic bones or a portion of the G sample basic bones, where G is a natural number;

[0427] Obtain a second sample object model, where the second sample object model includes K sample motion points, wherein each sample motion point corresponds to one or more sample basic bones among G sample basic bones, and the K sample motion points are used to define the motion of the second sample object model, where K is a natural number, and each of the K sample motion points has a sample label, and the sample label of any sample motion point is used to indicate a true motion binding parameter between any sample motion point and the corresponding sample basic bone;

[0428] Determine a fourth reference position of each sample motion point and a fifth reference position of each sample base bone in the G sample base bones;

[0429] Call the skinning network to be trained, convert the fourth reference position and the fifth reference position, and generate sample global prediction features corresponding to K sample motion points. The sample global prediction features are used to reflect the global position features between the K sample motion points and the corresponding sample basic bones, the position features of the G sample basic bones, and the structural features of the bone chain where the sample basic bones corresponding to the K sample motion points are located;

[0430] Call the skinning network to be trained based on the sample global prediction features to predict the sample motion binding parameters between each sample motion point and the corresponding sample basic skeleton;

[0431] Based on the difference between the sample motion binding parameters predicted for each sample motion point and the actual motion binding parameters indicated by the sample label of each sample motion point, the network parameters of the skinning network to be trained are corrected to obtain the skinning network.

[0432] Optionally, the training module 1906 corrects network parameters of the skinned network to be trained based on the difference between the sample motion binding parameters predicted for each sample motion point and the actual motion binding parameters indicated by the sample label of each sample motion point, and obtains the skinned network in a manner including:

[0433] Based on the sample motion binding parameters and the true motion binding parameters corresponding to each sample motion point, generating a global parameter prediction deviation for the sample motion binding parameters and a motion position prediction deviation for the second sample object model;

[0434] Generate a neighborhood parameter prediction deviation based on the difference between sample motion binding parameters corresponding to adjacent sample motion points among the K sample motion points and the difference between true motion binding parameters corresponding to adjacent sample motion points among the K sample motion points;

[0435] The global parameter prediction deviation, motion position prediction deviation and neighborhood parameter prediction deviation are summed to obtain the target prediction deviation for K sample motion points;

[0436] The network parameters of the skin network to be trained are corrected based on the target prediction deviation to obtain the skin network.

[0437] Optionally, the process of the training module 1906 generating the motion position prediction deviation includes:

[0438] Calculate the predicted motion position of each sample motion point based on the sample motion binding parameters predicted for each sample motion point, the starting position of each sample motion point, and the motion parameters of the base skeleton corresponding to each sample motion point;

[0439] Calculate the true motion position of each sample motion point based on the true motion binding parameters indicated by the sample label of each sample motion point, the starting position of each sample motion point, and the motion parameters of the underlying skeleton corresponding to each sample motion point;

[0440] Based on the predicted motion position and the true motion position of each sample motion point, a motion position prediction deviation is generated.

[0441] Optionally, the training module 1906 generates a neighborhood parameter prediction deviation based on the difference between sample motion binding parameters corresponding to adjacent sample motion points among the K sample motion points and the difference between true motion binding parameters corresponding to adjacent sample motion points among the K sample motion points, including:

[0442] Calculate the parameter distances between the sample motion binding parameters corresponding to each sample motion point and the sample motion binding parameters corresponding to the adjacent sample motion points of each sample motion point, and obtain the predicted parameter distances corresponding to K sample motion points;

[0443] Calculate the parameter distances between the true motion binding parameters corresponding to each sample motion point and the true motion binding parameters corresponding to the adjacent sample motion points of each sample motion point, and obtain the true parameter distances corresponding to K sample motion points;

[0444] Generates neighborhood parameter prediction bias based on the difference between the predicted parameter distance and the true parameter distance.

[0445] Optionally, the motion data processing device 190 further includes a local skinning module 1907, which is configured to:

[0446] Selecting multiple local motion points from the N motion points, and selecting multiple local basic bones associated with the multiple local motion points from the M basic bones;

[0447] Performing feature embedding processing on the first reference positions of the plurality of local motion points to generate initial motion point features of the plurality of local motion points;

[0448] Performing feature embedding processing on the second reference positions of the multiple local basic bones to generate initial bone features of the multiple local basic bones;

[0449] Performing feature interaction processing on the initial features of the skeleton and the initial features of the motion points to generate first interaction features of multiple local motion points and second interaction features of multiple local basic skeletons;

[0450] Predicting local motion binding parameters between each local motion point and each local base skeleton based on the first interaction feature and the second interaction feature;

[0451] The predicted motion binding parameters between each local motion point and the corresponding base bone are updated as local motion binding parameters between each local motion point and each local base bone.

[0452] Optionally, the local skinning module 1907 predicts the local motion binding parameters between each local motion point and each local basic bone based on the first interaction feature and the second interaction feature, including:

[0453] Based on the first interaction feature and the second interaction feature, generating a first global feature of the plurality of local motion points, a first local feature of the plurality of local motion points, a second global feature of the plurality of local basic skeletons, and a second local feature of the plurality of local basic skeletons, the first local feature including a local feature of each local motion point, and the second local feature including a local feature of each local basic skeleton;

[0454] Obtaining an interaction feature distance between the first interaction feature and the second interaction feature;

[0455] Based on the interaction feature distance, the first global feature, the first local feature, the second global feature and the second local feature, local motion binding parameters between each local motion point and each local base bone are predicted.

[0456] Optionally, the local skinning module 1907 predicts the local motion binding parameters between each local motion point and each local base bone based on the interaction feature distance, the first global feature, the first local feature, the second global feature, and the second local feature, including:

[0457] Splicing the first global feature and the first local feature to generate first splicing features of multiple local motion points;

[0458] Splicing the second global feature and the second local feature to generate second splicing features of multiple local basic skeletons;

[0459] Obtaining a first adjacency matrix of the plurality of local motion points and a second adjacency matrix of the plurality of local basic bones, wherein the first adjacency matrix is ​​used to reflect the connection relationship between the plurality of local motion points, and the second adjacency matrix is ​​used to reflect the connection relationship between the plurality of local basic bones;

[0460] Based on the first splicing feature, the second splicing feature, the first adjacency matrix, the second adjacency matrix and the interaction feature distance, local motion binding parameters between each local motion point and each local base skeleton are predicted.

[0461] Optionally, the local skinning module 1907 predicts the local motion binding parameters between each local motion point and each local base bone based on the first splicing feature, the second splicing feature, the first adjacency matrix, the second adjacency matrix, and the interaction feature distance, including:

[0462] Performing a product operation on the first splicing feature and the first adjacency matrix to generate first adjacency features of multiple local motion points;

[0463] Performing a product operation on the second splicing feature and the second adjacency matrix to generate second adjacency features of multiple local basic bones;

[0464] generating relative adjacency features between the plurality of local motion points and the plurality of local base bones based on a difference between the first adjacency feature and the second adjacency feature;

[0465] The relative adjacent features and interactive feature distances are spliced ​​to generate local prediction features;

[0466] Based on the local prediction features, the local motion binding parameters between each local motion point and each local base bone are predicted.

[0467] Optionally, each of the N moving points has its own starting position; the first determining module 1903 is configured to:

[0468] Based on the starting position of each moving point and the starting positions of the adjacent moving points of each moving point, the discrete curvature of each moving point is calculated; the discrete curvature of any moving point is used to reflect the smoothness of the surface around the any moving point;

[0469] Acquire multiple reference curvatures, and based on the multiple reference curvatures and the discrete curvatures of the N moving points, smooth the starting positions of the N moving points to generate smoothed positions of the N moving points under each reference curvature;

[0470] Based on the smooth positions of the N moving points at each reference curvature, the average curvature of the N moving points at each reference curvature is calculated respectively;

[0471] The smooth positions of the N moving points under the reference curvature corresponding to the minimum average curvature are used as the first reference positions of the N moving points;

[0472] Among them, any motion point is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton and the starting position of the motion point.

[0473] Optionally, any one of the multiple reference curvatures is a target reference curvature; the first determining module 1903 smoothes the starting positions of the N moving points based on the multiple reference curvatures and the discrete curvatures of the N moving points to generate smoothed positions of the N moving points under each reference curvature, including:

[0474] The moving points whose discrete curvatures are greater than the target reference curvature among the N moving points are regarded as the moving points that need to be adjusted;

[0475] Smoothing the starting position of the motion point to be adjusted to generate a smoothed starting position of the motion point to be adjusted;

[0476] The starting positions of the moving points other than the moving point to be adjusted among the N moving points and the smoothed starting positions of the moving points to be adjusted are determined as the smoothed positions of the N moving points under the target reference curvature.

[0477] According to one embodiment of the present application, the steps involved in the motion data processing method shown in FIG3 can be performed by various modules in the motion data processing device 190 shown in FIG19. For example, step S101 shown in FIG3 can be performed by the first acquisition module 1901 in FIG19, step S102 shown in FIG3 can be performed by the second acquisition module 1902 in FIG19, step S103 shown in FIG3 can be performed by the first determination module 1903 in FIG19, step S104 shown in FIG3 can be performed by the conversion module 1904 in FIG19, and step S105 shown in FIG3 can be performed by the second determination module 1905 in FIG19.

[0478] The present application can obtain a first object model and a second object model, wherein the first object model may include M basic bones or some basic bones among the M basic bones, and the second object model may include N motion points, each of which may correspond to one or more basic bones among the M basic bones, and the N motion points may be used to define the motion of the second object model. Therefore, the present application can perform feature embedding processing on the first reference positions of the N motion points and the second reference positions of the M bones to generate global prediction features corresponding to the N motion points, and the global prediction features can be used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chains where the basic bones corresponding to the N motion points are located; thus, the present application can determine the motion binding parameters between each motion point and the corresponding basic bones based on the global prediction features; any of the N motion points can follow the corresponding basic bones to move according to the motion binding parameters between the N motion points and the corresponding basic bones. It can be seen that the method proposed in the present application can perform feature embedding processing on the first reference position of each motion point in the first object model and the second reference position of each motion point and the corresponding basic bone in the second object model to obtain global prediction features corresponding to N motion points. The global prediction features can include multiple features, such as the global position features between the motion point and the corresponding basic bone, the position features of M bones, and the structural features of the bone chain where the M bones are located. Subsequently, the motion binding parameters between each motion point and the corresponding basic bone can be quickly and accurately determined through the global prediction features. Any motion point can follow the corresponding basic bone to perform accurate movement according to the motion binding parameters between the motion point and the corresponding basic bone.

[0479] According to one embodiment of the present application, the various modules in the motion data processing device 190 shown in Figure 19 can be respectively or all merged into one or several units to constitute, or a certain (some) unit therein can also be split into multiple sub-units that are smaller in function, and the same operation can be achieved without affecting the realization of the technical effect of the embodiment of the present application. The above-mentioned modules are divided based on logical functions. In actual applications, the function of a module can also be realized by multiple units, or the function of multiple modules can be realized by one unit. In other embodiments of the present application, the motion data processing device 190 can also include other units. In actual applications, these functions can also be assisted by other units to achieve, and can be achieved by the collaboration of multiple units.

[0480] According to one embodiment of the present application, a computer program capable of executing the steps involved in the corresponding methods shown in the various embodiments of the present application can be run on a general-purpose computer device (the computer device may include processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM)) to construct a motion data processing device 190 as shown in Figure 19. The computer program can be recorded on a computer-readable recording medium, for example, and can be loaded into the computer device via the computer-readable recording medium and run therein.

[0481] Please refer to Figure 20, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 20, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 20, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0482] In the computer device 1000 shown in FIG20 , the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0483] Acquire a first object model, where the first object model includes M basic bones or a portion of the M basic bones, where M is a natural number;

[0484] Obtain a second object model, where the second object model includes N motion points, wherein each of the N motion points corresponds to one or more basic bones of the M basic bones, and the N motion points are used to define a motion of the second object model, where N is a natural number;

[0485] Determine a first reference position of each motion point and a second reference position of each base bone in the M base bones;

[0486] Converting the first reference position and the second reference position to generate global prediction features corresponding to N motion points, where the global prediction features are used to reflect the global position features between the N motion points and the corresponding base bones, the position features of the M base bones, and the structural features of the bone chains of the base bones corresponding to the N motion points;

[0487] Based on the global prediction features, the motion binding parameters between each motion point and the corresponding basic skeleton are determined, and any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton.

[0488] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the description of the above-mentioned motion data processing method in each embodiment of the present application, and can also execute the description of the above-mentioned motion data processing device 190 in the embodiment corresponding to FIG. 19 above, and will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here.

[0489] In addition, it should be noted that the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the processor executes the computer program, it can perform the description of the motion data processing method in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0490] As an example, the above-mentioned computer program can be deployed and executed on one computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network. Multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain network.

[0491] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped with the computer device. Furthermore, the computer-readable storage medium may include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is about to be output.

[0492] The present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the description of the above-mentioned motion data processing method in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0493] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0494] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0495] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A motion data processing method, characterized in that: The method comprises: Acquire a first object model, wherein the first object model includes M basic bones or includes some basic bones among the M basic bones, where M is a natural number; Acquire a second object model, wherein the second object model includes N motion points, wherein each of the N motion points corresponds to one or more basic bones of the M basic bones, and the N motion points are used to define the motion of the second object model, and N is a natural number; Determine a first reference position of each of the motion points and a second reference position of each of the M basic bones; The first reference position and the second reference position are converted to generate global prediction features corresponding to the N motion points, where the global prediction features are used to reflect global position features between the N motion points and corresponding basic bones, position features of the M basic bones, and structural features of bone chains where the basic bones corresponding to the N motion points are located; Based on the global prediction features, the motion binding parameters between each motion point and the corresponding basic skeleton are determined, and any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion points and the corresponding basic skeleton.

2. The method according to claim 1, characterized in that The global prediction feature includes the global features of the N moving points, and the global features of the moving points are used to reflect the global position features between the N moving points and the corresponding basic skeletons. The process of generating the global features of the moving points includes: Based on the first reference position of each movement point and the second reference position of the basic skeleton corresponding to each movement point, respectively generate position association information between each movement point and the corresponding basic skeleton; Performing feature embedding processing on the position association information between the N motion points and the corresponding basic skeletons to generate skeleton association features of the N motion points; Performing feature transformation processing on the skeleton-related features and the first reference positions of the N motion points to generate motion point transformation features of the N motion points; The moving point global feature is generated based on the moving point transformation feature.

3. The method according to claim 1 or 2, characterized in that: The step of performing feature transformation processing on the skeleton-related features and the first reference positions of the N motion points to generate motion point transformation features of the N motion points includes: Combining and processing the first reference positions of the N moving points and the normal information of the N moving points to generate basic features of the N moving points; Fusing the basic features and the skeleton-related features to generate initial representation features of the N motion points; The initial representation feature is subjected to feature transformation processing to generate the motion point transformation feature.

4. The method according to any one of claims 1 to 3, characterized in that: Any one of the N moving points is a target moving point, and the moving point transformation feature includes the transformation feature of each moving point; The generating the moving point global feature based on the moving point transformation feature comprises: Using multiple neighbor selection methods, select multiple neighbor motion points of the target motion point from the N motion points; Performing feature embedding processing on the transformation features of the target motion point and the transformation features of each neighbor motion point respectively to generate a plurality of neighbor association features of the target motion point, and performing feature embedding processing on the transformation features of the target motion point and the transformation features of one neighbor motion point of the target motion point to generate one neighbor association feature of the target motion point; Performing fusion processing on the neighbor association features of the N moving points to generate target neighbor features of the N moving points; The target neighbor features are subjected to feature reorganization processing to generate the moving point global features.

5. The method according to any one of claims 1 to 4, characterized in that: The step of performing feature reorganization processing on the target neighbor features to generate the moving point global features includes: Obtain the motion point interaction features, and fuse the target neighbor features and the motion point interaction features to generate the target motion point Features, the motion point interaction features are obtained by performing feature interaction processing on the position features of the N motion points and the position features of the M bones; The target motion point features are subjected to feature reorganization processing to generate the motion point global features.

6. The method according to any one of claims 1 to 5, characterized in that: Any one of the N moving points is a target moving point; Among them, the position association information between the target motion point and the corresponding basic skeleton includes at least one of the following: the relative position information between the target motion point and the corresponding basic skeleton, the absolute distance between the target motion point and the corresponding basic skeleton, and the shortest path distance between the target motion point and the corresponding basic skeleton.

7. The method according to any one of claims 1 to 6, characterized in that: The global prediction feature includes the bone position feature of the M basic bones, and the bone position feature is used to reflect the position feature of the M bones. The process of generating the bone position feature includes: Select a neighbor bone of each basic bone from the M basic bones; Performing splicing processing on the second reference position of each basic skeleton and the second reference position of the neighboring skeleton of each basic skeleton respectively, to generate splicing position information of each basic skeleton; Performing feature embedding processing on the splicing position information of the M basic bones to generate transition bone features of the M basic bones; The bone position feature is generated based on the transition bone feature.

8. The method according to any one of claims 1 to 7, characterized in that: The step of generating the bone position feature based on the transition bone feature comprises: Acquire basic features of the N moving points, where the basic features are obtained by combining first reference positions of the N moving points and normal information of the N moving points; Performing feature interaction processing on the basic features and the transitional skeleton features to generate motion point interaction features of the N motion points and skeleton interaction features of the M basic skeletons; Among them, the bone position feature is the bone interaction feature.

9. The method according to any one of claims 1 to 8, characterized in that: The global prediction feature includes the bone chain structure feature associated with the N motion points, and the bone chain structure feature is used to reflect the structure feature of the bone chain where the basic skeleton corresponding to the N motion points is located. The process of generating the bone chain structure feature includes: Obtaining the associated joints of each basic bone on the bone chain where it is located, wherein any bone chain is composed of a plurality of basic bones connected to each other, and the joints of any bone chain refer to the bone connections between the basic bones included in the bone chain, and the associated joints of any basic bone include one or more adjacent joints of the basic bone on the bone chain where it is located; Combining the second reference position of the basic skeleton corresponding to each motion point with the third reference position of the associated joint of the basic skeleton corresponding to each motion point, to obtain the first bone chain information associated with each motion point; Combining and processing the position distance information between each motion point and the corresponding basic skeleton, and the position distance information between each motion point and the associated joint of the corresponding basic skeleton, to obtain the second bone chain information associated with each motion point; The bone chain structure feature is generated based on the first bone chain information and the second bone chain information associated with each motion point.

10. The method according to any one of claims 1 to 9, characterized in that: The generating the bone chain structure feature based on the first bone chain information and the second bone chain information associated with each motion point comprises: Performing feature embedding processing on the first bone chain information associated with the N motion points to generate first bone chain embedding features associated with the N motion points; Performing feature embedding processing on the second bone chain information associated with the N motion points to generate second bone chain embedding features associated with the N motion points; The first bone chain embedding feature and the second bone chain embedding feature are fused to generate the bone chain structure feature.

11. The method according to any one of claims 1 to 10, characterized in that: The global prediction feature is generated by calling the skinning network; The step of determining the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction feature comprises: The skinning network is called to predict the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction feature.

12. The method according to any one of claims 1 to 11, characterized in that: The calling of the skinning network to predict the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction feature includes: Calling the skinning network to predict initial motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction feature; Calculating predicted motion positions of the N motion points under multiple target object postures of the first object model based on the initial motion binding parameters predicted for each motion point; Acquire a straight wiring formed by the N moving points under the various target object postures, and determine the target posture positions of the N moving points under the various target object postures based on the straight wiring; Based on the difference between the predicted motion positions of the N motion points under the multiple target object postures and the target posture positions, the initial motion binding parameters between the N motion points and the corresponding basic bones are optimized to generate the motion binding parameters between the N motion points and the corresponding basic bones; Among them, the process of optimizing the initial motion binding parameters between the N motion points and the corresponding basic skeletons includes: approximating the predicted motion positions of the N motion points under the various target object postures to the target posture positions of the N motion points under the various target object postures.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: Acquire a first sample object model, wherein the first sample object model includes G sample basic bones or includes some sample basic bones among the G sample basic bones, where G is a natural number; Acquire a second sample object model, wherein the second sample object model includes K sample motion points, wherein each sample motion point corresponds to one or more sample basic bones among the G sample basic bones, and the K sample motion points are used to define the motion of the second sample object model, K is a natural number, and the K sample motion points all have sample labels, and the sample label of any sample motion point is used to indicate a real motion binding parameter between any sample motion point and the corresponding sample basic bone; Determine a fourth reference position of each sample motion point and a fifth reference position of each sample basic bone in the G sample basic bones; Calling the skinning network to be trained, converting the fourth reference position and the fifth reference position, generating sample global prediction features corresponding to the K sample motion points, wherein the sample global prediction features are used to reflect the global position features between the K sample motion points and the corresponding sample basic bones, the position features of the G sample basic bones, and the structural features of the bone chain where the sample basic bones corresponding to the K sample motion points are located; Calling the skinning network to be trained to predict sample motion binding parameters between each sample motion point and the corresponding sample basic skeleton based on the sample global prediction feature; Based on the difference between the sample motion binding parameters predicted for each sample motion point and the real motion binding parameters indicated by the sample label of each sample motion point, the network parameters of the skin network to be trained are corrected to obtain the skin network.

14. The method according to any one of claims 1 to 13, characterized in that: The method of correcting the network parameters of the skin network to be trained based on the difference between the sample motion binding parameters predicted for each sample motion point and the real motion binding parameters indicated by the sample label of each sample motion point to obtain the skin network includes: Based on the sample motion binding parameters and the real motion binding parameters corresponding to each sample motion point, generating a global parameter prediction deviation for the sample motion binding parameters and a motion position prediction deviation for the second sample object model; Generate a neighborhood parameter prediction deviation based on differences between sample motion binding parameters corresponding to adjacent sample motion points among the K sample motion points and differences between real motion binding parameters corresponding to adjacent sample motion points among the K sample motion points; The global parameter prediction deviation, the motion position prediction deviation and the neighborhood parameter prediction deviation are summed to obtain a target prediction deviation for the K sample motion points; The network parameters of the skin network to be trained are corrected based on the target prediction deviation to obtain the skin network.

15. The method according to any one of claims 1 to 14, characterized in that: The process of generating the motion position prediction deviation includes: Calculate the predicted motion position of each sample motion point based on the sample motion binding parameters predicted for each sample motion point, the starting position of each sample motion point, and the motion parameters of the basic skeleton corresponding to each sample motion point; Calculate the real motion position of each sample motion point based on the real motion binding parameter indicated by the sample label of each sample motion point, the starting position of each sample motion point and the motion parameter of the basic skeleton corresponding to each sample motion point; The motion position prediction deviation is generated based on the predicted motion position and the actual motion position of each sample motion point.

16. The method according to any one of claims 1 to 15, characterized in that: The generating of the neighborhood parameter prediction deviation based on the difference between the sample motion binding parameters corresponding to the adjacent sample motion points among the K sample motion points and the difference between the real motion binding parameters corresponding to the adjacent sample motion points among the K sample motion points includes: Calculating parameter distances between sample motion binding parameters corresponding to each sample motion point and sample motion binding parameters corresponding to adjacent sample motion points of each sample motion point, to obtain prediction parameter distances corresponding to the K sample motion points; Calculating the parameter distances between the real motion binding parameters corresponding to each sample motion point and the real motion binding parameters corresponding to the adjacent sample motion points of each sample motion point, to obtain the real parameter distances corresponding to the K sample motion points; The neighborhood parameter prediction deviation is generated based on the difference between the predicted parameter distance and the true parameter distance.

17. The method according to any one of claims 1 to 16, characterized in that: The method further comprises: Selecting a plurality of local motion points from the N motion points, and selecting a plurality of local basic bones associated with the plurality of local motion points from the M basic bones; Performing feature embedding processing on first reference positions of the plurality of local motion points to generate initial motion point features of the plurality of local motion points; Performing feature embedding processing on the second reference positions of the multiple local basic bones to generate initial bone features of the multiple local basic bones; Performing feature interaction processing on the initial features of the skeleton and the initial features of the motion points to generate first interaction features of the multiple local motion points and second interaction features of the multiple local basic skeletons; Based on the first interaction feature and the second interaction feature, predicting local motion binding parameters between each local motion point and each local basic skeleton respectively; The predicted motion binding parameters between each local motion point and the corresponding basic skeleton are updated as local motion binding parameters between each local motion point and each local basic skeleton.

18. The method according to any one of claims 1 to 17, characterized in that: The predicting, based on the first interaction feature and the second interaction feature, the local motion binding parameters between each local motion point and each local basic bone respectively, comprises: Based on the first interaction feature and the second interaction feature, generate a first global feature of the multiple local motion points, a first local feature of the multiple local motion points, a second global feature of the multiple local basic bones, and a second local feature of the multiple local basic bones, wherein the first local feature includes a local feature of each local motion point, and the second local feature includes a local feature of each local basic bone; Acquire an interaction feature distance between the first interaction feature and the second interaction feature; Based on the interactive feature distance, the first global feature, the first local feature, the second global feature and the second local feature, local motion binding parameters between each local motion point and each local basic bone are predicted.

19. The method according to any one of claims 1 to 18, characterized in that: The predicting, based on the interactive feature distance, the first global feature, the first local feature, the second global feature and the second local feature, the local motion binding parameter between each local motion point and each local basic bone comprises: Performing splicing processing on the first global feature and the first local feature to generate a first splicing feature of the plurality of local motion points; Performing splicing processing on the second global feature and the second local feature to generate second splicing features of the multiple local basic skeletons; Acquire a first adjacency matrix of the plurality of local motion points, and acquire a second adjacency matrix of the plurality of local basic bones, wherein the first adjacency matrix is ​​used to reflect the connection relationship between the plurality of local motion points, and the second adjacency matrix is ​​used to reflect the connection relationship between the plurality of local basic bones; Based on the first splicing feature, the second splicing feature, the first adjacency matrix, the second adjacency matrix and the interaction feature distance, local motion binding parameters between each local motion point and each local basic skeleton are predicted.

20. The method according to any one of claims 1 to 19, characterized in that: The predicting, based on the first splicing feature, the second splicing feature, the first adjacency matrix, the second adjacency matrix and the interactive feature distance, the local motion binding parameters between each local motion point and each local basic skeleton respectively includes: Performing a product operation on the first splicing feature and the first adjacency matrix to generate first adjacency features of the multiple local motion points; Performing a product operation on the second splicing feature and the second adjacency matrix to generate second adjacency features of the multiple local basic bones; generating relative adjacency features between the plurality of local motion points and the plurality of local base bones based on a difference between the first adjacency feature and the second adjacency feature; Performing splicing processing on the relative adjacent features and the interactive feature distances to generate local prediction features; Based on the local prediction features, local motion binding parameters between each local motion point and each local basic bone are predicted.

21. The method according to any one of claims 1 to 20, characterized in that: The N moving points all have their own starting positions; the method further includes: Based on the starting position of each moving point and the starting positions of the adjacent moving points of each moving point, the discrete curvature of each moving point is calculated; the discrete curvature of any moving point is used to reflect the smoothness of the surface around the any moving point; Acquire multiple reference curvatures, and based on the multiple reference curvatures and the discrete curvatures of the N moving points, smooth the starting positions of the N moving points to generate smooth positions of the N moving points under each reference curvature; Based on the smooth positions of the N moving points under each of the reference curvatures, respectively calculating the average curvature of the N moving points under each of the reference curvatures; Taking the smoothed positions of the N moving points under the reference curvature corresponding to the minimum average curvature as the first reference positions of the N moving points; Among them, any motion point is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion point and the corresponding basic skeleton and the starting position of the motion point.

22. The method according to any one of claims 1 to 21, characterized in that: Any one of the multiple reference curvatures is a target reference curvature; based on the multiple reference curvatures and the discrete curvatures of the N moving points, the starting positions of the N moving points are smoothed to generate the smoothed positions of the N moving points under each reference curvature, including: The moving point whose discrete curvature among the N moving points is greater than the target reference curvature is used as the moving point to be adjusted; Smoothing the starting position of the motion point to be adjusted to generate a smoothed starting position of the motion point to be adjusted; The starting positions of the moving points other than the moving point to be adjusted among the N moving points and the smoothed starting positions of the moving points to be adjusted are determined as the smoothed positions of the N moving points under the target reference curvature.

23. A motion data processing device, characterized in that: The device comprises: A first acquisition module is used to acquire a first object model, wherein the first object model includes M basic bones or some basic bones among the M basic bones, where M is a natural number; A second acquisition module is used to acquire a second object model, wherein the second object model includes N moving points, wherein the N moving points Each motion point in the points corresponds to one or more basic bones in the M basic bones, and the N motion points are used to define the motion of the second object model, where N is a natural number; A first determination module, used for determining a first reference position of each movement point and a second reference position of each basic bone in the M basic bones; A conversion module, used for converting the first reference position and the second reference position to generate global prediction features corresponding to the N motion points, wherein the global prediction features are used to reflect the global position features between the N motion points and the corresponding basic bones, the position features of the M basic bones, and the structural features of the bone chains where the basic bones corresponding to the N motion points are located; The second determination module is used to determine the motion binding parameters between each motion point and the corresponding basic skeleton based on the global prediction features, and any motion point among the N motion points is used to follow the corresponding basic skeleton to move according to the motion binding parameters between the motion points and the corresponding basic skeleton.

24. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 22 when being executed by a processor.

25. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 22.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 22.

Images