A grid model processing method, storage medium and device

Abstract

This specification provides a method, storage medium and device for processing a mesh model. In this method, a joint graph neural network and a skin graph neural network are trained based on multiple preset three-dimensional character models, wherein the joint graph neural network learns the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns the weight of each vertex affected by each joint point based on the skin weight determined in the three-dimensional character model. Then, in the application stage, the joint points and skin weights of the target mesh model to be processed are generated respectively through these two graph neural networks, so that the target mesh model can be used for motion control. In this way, the threshold for creating three-dimensional character animation is lowered, and the cost and time of developing three-dimensional character animation are shortened.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a grid model processing method, storage medium and device. Background Art

[0002] 3D models play a very important role in many fields, such as gaming and live broadcasting. However, in related technologies, the movement of 3D models is generally achieved by professional technicians manually creating the corresponding skeleton and skin of the 3D model, or using scanning equipment to capture the movements of real actors and apply them to 3D model animation through algorithm processing. In this way, creating 3D model animation often requires a large amount of time and manpower costs. Summary of the invention

[0003] In order to overcome the problems existing in the related art, this specification provides a grid model processing method, storage medium and device.

[0004] According to a first aspect of an embodiment of this specification, a method for processing a grid model is provided, comprising:

[0005] Get the target mesh model to be processed;

[0006] Generate joint points of the target mesh model through a joint graph neural network, and then generate skin weights of each joint point of the target mesh model through a skin graph neural network;

[0007] Binding the joint points and the skin weights to the target mesh model so that the bound target mesh model is used for motion control;

[0008] Among them, the joint graph neural network and the skin graph neural network are trained based on multiple preset three-dimensional character models, the three-dimensional character model has multiple vertices, the joint graph neural network learns the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns the weight of each vertex affected by each joint point based on the skin weights determined in the three-dimensional character model.

[0009] In some examples, the plurality of preset three-dimensional character models have the same number of vertices.

[0010] In some examples, the above-mentioned multiple preset 3D character models are obtained by processing as follows:

[0011] Acquire a plurality of initial three-dimensional character models, wherein the plurality of initial three-dimensional character models do not have the same number of vertices;

[0012] For each initial three-dimensional character model, the preset grid template is deformed to fit the initial three-dimensional character model, and the deformed preset grid template is used as the preset three-dimensional character model.

[0013] In some examples, the joint points of the target mesh model are obtained based on the following method:

[0014] Inputting the target mesh model into the joint graph neural network to obtain a feature vector corresponding to each vertex, wherein the feature vector is used to characterize the correlation between the vertex and other vertices;

[0015] Inputting the feature vector into a fully connected network to obtain a predicted joint position of each vertex;

[0016] The predicted joint positions of all vertices are clustered, and the center point of each cluster is used as the joint point of the target mesh model.

[0017] In some examples, each vertex and each joint point of the target mesh model has a corresponding sorting number;

[0018] In the target mesh model, the skin weight of any vertex corresponding to any joint point is determined from the weight matrix output by the skin graph neural network according to the sorting number of the vertex and the sorting number of the joint point.

[0019] In some examples, motion control of the bound target mesh is performed as follows:

[0020] Obtain target motion parameters, control the joints of the target mesh model to move based on the target motion parameters, and control other vertices of the target mesh model to move based on the joints and skin weights; the other vertices refer to vertices in the target mesh model other than the joints.

[0021] In some examples, the above method is applied to a live broadcast client, and the target motion parameter is obtained based on manual settings of the anchor user, or is obtained by analyzing captured anchor movements.

[0022] In some examples, the above method is applied to a live broadcast server;

[0023] The live broadcast server groups multiple preset three-dimensional character models according to crowd types, pre-trains multiple joint graph neural networks and skin graph neural networks, and after the anchor starts broadcasting, identifies the crowd type to which the target mesh model specified by the anchor belongs, and determines the joint graph neural network and skin graph neural network corresponding to the crowd type, so that the anchor obtains the target three-dimensional model for motion control;

[0024] Alternatively, the live broadcast server pre-processes and stores target grid models corresponding to different crowd types, so that after the anchor starts broadcasting, it can identify the crowd type to which the anchor belongs, determine the target grid model that matches the crowd type to which the anchor belongs, and send the target grid model to the anchor's live broadcast client.

[0025] According to a second aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, any method in the embodiments of the specification is implemented.

[0026] According to a third aspect of the embodiments of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the methods in the embodiments of the specification is implemented.

[0027] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0028] In the embodiments of this specification, a method, storage medium and device for processing a mesh model are disclosed. In this method, a joint graph neural network and a skin graph neural network are trained based on multiple preset three-dimensional character models, wherein the joint graph neural network learns the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns the weight of each vertex affected by each joint point based on the skin weight determined in the three-dimensional character model. Then, in the application stage, the joint points and skin weights of the target mesh model to be processed are generated respectively through these two graph neural networks, so that the target mesh model can be used for motion control. In this way, the threshold for creating three-dimensional character animation is lowered, and the cost and time of developing three-dimensional character animation are shortened.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0031] Figure 1 is a flowchart of a method for processing a grid model according to an exemplary embodiment of the present specification;

[0032] Figure 2 is a schematic diagram of a grid model shown in this specification according to an exemplary embodiment;

[0033] Figure 3is a schematic diagram of generating joint positions for a mesh model according to an exemplary embodiment of the present specification;

[0034] Figure 4 is a schematic diagram of generating a skin for a mesh model according to an exemplary embodiment of the present specification;

[0035] Figure 5 is a schematic diagram of a target three-dimensional model generating motion according to an exemplary embodiment of the present specification;

[0036] Figure 6 It is a hardware structure diagram of a computer device where a processing device of a grid model is located according to an exemplary embodiment of this specification;

[0037] Figure 7 It is a block diagram of a grid model processing device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0039] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0040] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0041] Three-dimensional models play a very important role in many fields, such as gaming, live broadcasting, and so on. However, in the related art, the movement of three-dimensional models is generally achieved by professional technicians manually creating the skeleton and skin corresponding to the three-dimensional model, or using scanning equipment to capture the movements of real actors, and applying them to three-dimensional model animations after algorithm processing. In this way, creating three-dimensional model animations often requires a large amount of time and manpower costs. Based on this, the embodiments of this specification provide a method for processing a mesh model to solve the above problems.

[0042] Next, the embodiments of this specification are described in detail.

[0043] like Figure 1 As shown, Figure 1 is a flowchart of a method for processing a grid model according to an exemplary embodiment of the present specification, the method comprising:

[0044] In step 101, a target grid model to be processed is obtained;

[0045] A mesh model is a model that uses a series of polygons of similar size and shape, such as triangles and quadrilaterals, to approximate a three-dimensional object. A mesh model is composed of several vertices. Taking a triangular mesh model as an example, each triangle has three vertices, and each triangle shares edges with other triangles. Each vertex of the mesh model has a corresponding coordinate, which is obtained by establishing a coordinate system with a certain point of the mesh model, such as the center point of the back waist of a human character model, as the origin.

[0046] To visually realize the movement of the constructed mesh model in three-dimensional space, the usual practice is to create a skeleton and skin for the mesh model, that is, for the vertices of the mesh model, determine which joints / bones each vertex is affected by, and specify the influence weight. Among them, the joints refer to the main vertices in the model. The joints are connected according to the character's body structure to form a skeleton. The line between two joints is the skeleton, and skinning refers to binding the vertices in the mesh model to the skeleton. The form of skinning weight can be the controlled joint number of each vertex of the mesh model, and the controlled joint influence weight. For example, a skinning weight can be expressed as M (i, j), where M represents the corresponding weight, i represents the vertex number, and j represents the joint number. This embodiment aims to provide a method for automatically binding joints and skinning weights for a mesh model, so that the processed mesh model can be driven by data.

[0047] The target mesh model mentioned in this embodiment may be a human mesh model, or a mesh model of other images, such as a cartoon character, a robot, an animal, etc., and this specification does not impose any limitation on this.

[0048] In step 102, the joint points of the target mesh model are generated by a joint graph neural network, and the skin weights of the joint points of the target mesh model are generated by a skin graph neural network;

[0049] The joint graph neural network and the skin graph neural network are respectively obtained by training based on a plurality of preset three-dimensional character models, the three-dimensional character model having a plurality of vertices, the joint graph neural network learns to obtain the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns to obtain the weight of each vertex affected by each joint point based on the skin weight determined in the three-dimensional character model;

[0050] In this embodiment, the joint graph neural network and the skin graph neural network are trained based on a plurality of preset three-dimensional character models, wherein each preset three-dimensional character model is composed of a plurality of vertices, has a determined joint point and a determined skin weight. The three-dimensional character model is a three-dimensional model of a certain character image, where the character image includes a real person, a cartoon character, an animal, and the like.

[0051] In order to improve the convergence speed and effect of the neural network, in some examples, these preset three-dimensional role models can be models with the same number of vertices. Optionally, they can be obtained by the following processing: obtaining multiple initial three-dimensional role models, the multiple initial three-dimensional role models do not have the same number of vertices; for each initial three-dimensional role model, deforming the preset mesh template to fit the initial three-dimensional role model, and using the deformed preset mesh template as the preset three-dimensional role model. The multiple initial three-dimensional role models can be collected from the Internet or made by professional technicians. Due to different sources, the number of vertices of these initial three-dimensional role models may be different, and the number of vertices of these initial three-dimensional role models is generally of a large order of magnitude. In order to reduce them to the same number of vertices, they can be meshed. Regularization operation, specifically, a preset mesh template with a preset number of vertices is set, and the preset mesh template is deformed by a three-dimensional modeling tool (such as Maya software) to fit the collected initial three-dimensional role model, and then the deformed preset mesh template is used as the preset three-dimensional role model mentioned above, that is, as a training sample. In addition, in practical applications, in order to make the results of neural network prediction more accurate, the initial three-dimensional character models of the same type of characters can be collected according to the image of the target mesh model to be processed. For example, if the image of the target mesh model to be processed is that of a young male, the collected initial three-dimensional character model can be designated as the three-dimensional model of the young male character. In this way, when the neural network is trained subsequently, the neural network can be made more targeted, thereby effectively improving its accuracy.

[0052] It should be noted that the number of vertices of the preset three-dimensional character model can be in the order of thousands. In an optional embodiment, the number of vertices here can be 5000. Through experimental summary, it is found that such an order of magnitude can not only maintain good accuracy, but also effectively reduce the amount of calculation. Of course, in other embodiments, the number of vertices of the preset three-dimensional character model can also be set according to the needs of the specific scene. In addition, when using the joint graph neural network and the skin graph neural network to generate the joint points and skin weights of the target mesh model respectively, the same mesh regularization operation can be performed on the target mesh model to control its triangle number within the preset order of magnitude and make its vertex distribution more uniform.

[0053] The mesh model is a graph structure data (Graph), and the graph structure data of the target mesh model includes: multiple vertices, each vertex has corresponding coordinates, and each N vertices form an N-gon, and each N-gon shares edges with other N-gons. Therefore, this embodiment uses a graph neural network (Graph Neural Networks, GNN) to generate the joint points and skin weights of the target mesh model. Graph neural network is a neural network that directly acts on the graph structure. Its basic idea can be considered to aggregate the information of each node and its surrounding nodes in the graph structure through a neural network. Compared with traditional neural networks, graph neural networks can achieve better results in tasks such as property prediction, classification, and regression. In this embodiment, the multiple preset three-dimensional role models mentioned above are used as training samples to train two graph neural networks, among which the joint graph neural network uses the joint points determined in these three-dimensional role models as supervision to learn the classification of each vertex; the skin graph neural network uses the skin weights determined in these three-dimensional role models as supervision to learn the weights of each vertex affected by each joint point. In this way, in the application stage, these two graph neural networks can use the learned knowledge to derive the corresponding joint positions and skin weights according to the input mesh model.

[0054] In some examples, the joint graph neural network and the skinned graph neural network in this step can be graph attention networks (Graph Attention Networks, GAT). Graph attention networks use an attention mechanism that can amplify the influence of the most important parts of the data. Graph attention networks can be effectively applied to inductive learning problems (first learn a certain pattern from the training samples, and then use it to predict the test samples) and transductive learning problems (first observe specific training samples, and then make predictions for specific test samples). Of course, in other embodiments, the joint graph neural network or the skinned graph neural network can also use other graph neural networks, such as graph convolution networks (Graph Convolution Networks, GCN), gated graph neural networks (GGNN), etc., and this specification does not limit this.

[0055] The specific training process of the joint graph neural network and the skin graph neural network in this embodiment is consistent with the training process of the graph neural network in the related art, so this specification will not go into details.

[0056] In the application stage, the target mesh model is input into the trained joint graph neural network, and each vertex gets a corresponding feature vector, which is a representation of the corresponding vertex and is used to measure the correlation between the corresponding vertex and other vertices. After that, the joint graph neural network can use a fully connected network to predict these feature vectors and output the predicted joint position of each vertex, that is, the position of each vertex to the joint closest to it. After obtaining the predicted joint position of each vertex, a clustering algorithm can be used to cluster the predicted joint positions of all vertices, and the center of each class is used as the final joint position. The clustering algorithm here can be a K-means clustering algorithm, and the specified K value can be determined according to the standard number of joint points of the specific model. For example, if a certain type of character model has 24 standard joint points, a K-means clustering algorithm with a K value of 24 can be used for clustering. In this way, the number of joint points can be constrained to meet the standard number of joint points. It should be noted that, in other embodiments, the fully connected network mentioned above can also be an independent neural network. That is to say, only the joint graph neural network is used to output the feature vector of each vertex, and then these feature vectors are used as the input of the fully connected network. The fully connected network is used to obtain the predicted joint position of each vertex. It is concluded through experiments that such a structure can further improve the accuracy of the results.

[0057] Input the target mesh model into the trained skin graph neural network to obtain the skin weights of each joint point. The skin weights of each joint point mentioned here refer to the weights of the influence of each joint point on each vertex of the target mesh model. In some examples, the skin weights of each joint point output by the skin graph neural network can be a matrix representing the skin weights, which is composed of the weights of each vertex affected by each joint point. For example, the skin graph neural network outputs an N*K matrix, where N is the number of vertices and K is the number of joint points. Each vertex and each joint point of the target mesh model has a corresponding sorting number. The weight of the i-th vertex affected by the j-th joint point is the value of the i-th row and j-th column in the matrix. The skin weight is a value in the range of [0,1]. If the x-th vertex is the y-th joint point, the weight of the x-th vertex affected by the y-th joint point is 1.

[0058] In step 103, the joint points and the skin weights are bound to the target mesh model, so that the bound target mesh model is used for motion control.

[0059] After obtaining the joint points and skin weights output by the graph neural network, the joint points and skin weights are bound to the target mesh model, that is, a binding relationship between the target mesh model and each joint point is formed, and a binding weight between each joint point and the vertex of the target mesh model is formed. In this way, the target mesh model can move freely.

[0060] In an optional embodiment, the motion control of the bound target mesh model can be performed in the following manner: obtaining target motion parameters, controlling the joints of the target mesh model to move based on the target motion parameters, and controlling the other vertices of the target mesh model to move based on the joints and skin weights; the other vertices refer to the vertices in the target mesh model other than the joints. The target motion parameters here can be parameters for controlling the motion of each joint, such as motion time, displacement, speed and / or direction, etc. According to the target motion parameters and the initial position of each joint, the translation and / or rotation of each joint in the corresponding direction can be calculated, that is, each joint is controlled to move. After the joint moves, the coordinate information of other vertices can be calculated according to the binding information, thereby driving the mesh model to deform. Optionally, the coordinate information of other vertices can be calculated using a linear skinning algorithm. For example, for a certain vertex, the position of the vertex under the influence of the joint is calculated according to the transformation matrix of the current position of the joint relative to the static position and the weight of this vertex relative to the joint.

[0061] In the embodiment of this specification, joint graph neural network and skin graph neural network are trained based on multiple preset three-dimensional character models, wherein the joint graph neural network learns the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns the weight of each vertex affected by each joint point based on the skin weight determined in the three-dimensional character model. Then, in the application stage, the joint points and skin weights of the target mesh model to be processed are generated respectively through these two graph neural networks, so that the target mesh model can be used for motion control. In this way, the threshold for creating three-dimensional character animation is lowered, and the cost and time of developing three-dimensional character animation are shortened.

[0062] The solution of this embodiment can be applied to the live broadcast client, and the target motion parameters mentioned above can be obtained based on the manual settings of the anchor user, or by parsing the captured anchor movements. For example, after the anchor client uses the solution of this embodiment to automatically create joints and skins for the three-dimensional character model of the target image, it can further identify and parse the anchor's movements through the motion capture system to obtain the target motion parameters, drive the three-dimensional character model to move based on the target motion parameters, and render the picture of the three-dimensional character model in real time and display it in the live broadcast picture. In this way, the live broadcast content is enriched. Moreover, the target image can be a robot, an animal, etc., which increases the fun of the live broadcast and is conducive to improving the audience retention rate of the live broadcast room.

[0063] In order to further explain the processing scheme of the grid model of this specification, a specific embodiment is introduced below:

[0064] In this embodiment, the processing method of the mesh model is applied to a live broadcast client, the user of which is a host, who wants to add a three-dimensional virtual character to the live broadcast to enrich the content of the live broadcast. However, the host currently only has a static mesh model of a three-dimensional character. In the related art, in order to enable the mesh model to form a complete three-dimensional character animation, professional technicians need to create the corresponding character skeleton and skin, but this takes a long time to produce and is costly. This embodiment provides the following solution to automatically create the skeleton and skin for the mesh model:

[0065] S201, collecting a plurality of initial three-dimensional character models with skins from the Internet, wherein the number of vertices of these initial three-dimensional character models is mostly different;

[0066] S202, using a preset mesh template with a preset number of vertices, deforming the preset mesh template through Maya software to make it fit the collected initial 3D character model, and using the deformed preset mesh template as the preset 3D character model. In this embodiment, the preset number of vertices is 5000, and each vertex has a corresponding sorting number;

[0067] S203, using the preset three-dimensional character model as a training sample, using the joint points and the skin weights as supervision, respectively, to train a joint graph neural network and a skin graph neural network;

[0068] S204, obtaining the mesh model specified by the anchor user, and performing the same mesh regularization operation on it through the preset mesh template used in S202, so that the number of triangles is reduced to the same order of magnitude, and the vertex distribution is more uniform; Figure 2 As shown, Figure 2 is a schematic diagram of a grid model shown in this embodiment;

[0069] S205, input the mesh model into the trained joint graph neural network, obtain a corresponding feature vector for each vertex, and then input the feature vector into a fully connected network, output each vertex to the joint position closest to it, that is, predict the joint position, use the K-means clustering algorithm to cluster the predicted joint position of each vertex, and use the center of each class as the final joint position; Figure 3 As shown, Figure 3 is a schematic diagram of generating joint positions for a mesh model shown in this embodiment, wherein the K value is 24, and each joint point has a corresponding sorting number;

[0070] S206, input the mesh model into the trained skinning graph neural network to obtain a matrix representing the skinning weight, which is a 5000*24 matrix, and the weight of the i-th vertex affected by the j-th joint point is the value of the i-th row and j-th column in the matrix; Figure 4 As shown, Figure 4 This embodiment shows a schematic diagram of generating a skin for a mesh model, where one skin area corresponds to one joint.

[0071] For the sake of convenience, the mesh model with created joints and skin weights is recorded as the target 3D model. After the anchor starts broadcasting, the live broadcast client can parse the captured anchor's movements to obtain motion parameters, and drive the joints of the target 3D model to move based on the motion parameters. The vertices other than the joints in the target 3D model move according to the calculation results of the linear skinning algorithm. The current parameters of the linear skinning algorithm are obtained based on the input of the skin weights output by the skinning graph neural network. Figure 5 As shown, Figure 5 It is a schematic diagram of the target three-dimensional model generating movement shown in this embodiment.

[0072] In addition, in some examples, the solution of this embodiment can also be applied to a live broadcast server, where the live broadcast server collects initial three-dimensional character models corresponding to the character images for training according to different population types (e.g., classified according to gender, age, height, weight, etc.) to obtain the joint graph neural network and the skin graph neural network corresponding to different population types, and after the anchor starts broadcasting, identifies the population type to which the mesh model specified by the anchor belongs, selects an appropriate graph neural network to create a skeleton and skin for it, thereby obtaining a target three-dimensional model that can be used for motion control, or the live broadcast server pre-processes and stores target mesh models corresponding to different population types. In this way, when the anchor starts broadcasting, the live broadcast server receives a request from a live broadcast client, and can identify the population type to which the anchor belongs according to the anchor's current live broadcast screen, historical live broadcast screen, or anchor identification, etc., determine the target mesh model (with bound joint points and skin weights) that matches the population type to which the anchor belongs, and then send the target mesh model to the live broadcast client, or directly analyze the anchor's current live broadcast screen to identify the anchor's actions to drive the target mesh model to generate corresponding actions.

[0073] The live broadcast client or live broadcast server renders the target 3D model in real time and displays the rendering result in the live broadcast screen. In this way, the fun of the live broadcast is increased and the audience retention rate of the live broadcast room is effectively improved. The solution of this embodiment lowers the threshold for creating 3D character animation, shortens the cost and time of developing 3D character animation, and can meet the real-time requirements of live broadcast.

[0074] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a processing device for a grid model and a terminal to which it is applied.

[0075] The embodiments of the processing device of the grid model in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the file processing in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 6 The figure is a hardware structure diagram of the computer device where the processing device of the grid model in the embodiment of this specification is located, except Figure 6 In addition to the processor 610, memory 630, network interface 620, and non-volatile memory 640 shown, the server or electronic device where the device 631 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0076] Accordingly, an embodiment of the present specification further provides a computer storage medium, in which a program is stored. When the program is executed by a processor, the method in any of the above embodiments is implemented.

[0077] The embodiments of this specification may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0078] like Figure 7 As shown, Figure 7 is a block diagram of a grid model processing device according to an exemplary embodiment of the present specification, the device comprising:

[0079] An acquisition module 71 is used to acquire a target grid model to be processed;

[0080] A generating module 72, configured to generate joint points of the target mesh model through a joint graph neural network, and then generate skin weights of each joint point of the target mesh model through a skin graph neural network;

[0081] A binding module 73, used to bind the joint points and the skin weights to the target mesh model, so that the bound target mesh model is used for motion control;

[0082] Among them, the joint graph neural network and the skin graph neural network are trained based on multiple preset three-dimensional character models, the three-dimensional character model has multiple vertices, the joint graph neural network learns the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns the weight of each vertex affected by each joint point based on the skin weights determined in the three-dimensional character model.

[0083] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0084] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0085] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0087] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0088] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for processing a mesh model, characterized in that: include: Get the target mesh model to be processed; Generate joint points of the target mesh model through a joint graph neural network, and then generate skin weights of each joint point of the target mesh model through a skin graph neural network; Binding the joint points and the skin weights to the target mesh model so that the bound target mesh model is used for motion control; The joint graph neural network and the skin graph neural network are obtained by training based on a plurality of preset three-dimensional character models, the three-dimensional character model having a plurality of vertices, the joint graph neural network learns to obtain the classification of each vertex based on the joint points determined in the three-dimensional character model, and the skin graph neural network learns to obtain the weight of each vertex affected by each joint point based on the skin weight determined in the three-dimensional character model; The joint points of the target mesh model are obtained based on the following method: Inputting the target mesh model into the joint graph neural network to obtain a feature vector corresponding to each vertex, wherein the feature vector is used to characterize the correlation between the vertex and other vertices; Inputting the feature vector into a fully connected network to obtain a predicted joint position of each vertex; The predicted joint positions of all vertices are clustered, and the center point of each cluster is used as the joint point of the target mesh model.

2. The method according to claim 1, characterized in that The multiple preset three-dimensional character models have the same number of vertices.

3. The method according to claim 2, characterized in that The plurality of preset three-dimensional role models are obtained by processing as follows: Acquire a plurality of initial three-dimensional character models, wherein the plurality of initial three-dimensional character models do not have the same number of vertices; For each initial three-dimensional character model, the preset grid template is deformed to fit the initial three-dimensional character model, and the deformed preset grid template is used as the preset three-dimensional character model.

4. The method according to claim 1, characterized in that: Each vertex and each joint point of the target mesh model has a corresponding sorting number; In the target mesh model, the skin weight of any vertex corresponding to any joint point is determined from the weight matrix output by the skin graph neural network according to the sorting number of the vertex and the sorting number of the joint point.

5. The method according to claim 1, characterized in that The motion of the bound target mesh model can be controlled by: Obtain target motion parameters, control the joints of the target mesh model to move based on the target motion parameters, and control other vertices of the target mesh model to move based on the joints and skin weights; the other vertices refer to vertices in the target mesh model other than the joints.

6. The method according to claim 5, characterized in that The method is applied to a live broadcast client, and the target motion parameter is obtained based on manual settings of the anchor user, or is obtained by analyzing captured anchor movements.

7. The method according to claim 1, characterized in that The method is applied to a live broadcast server; The live broadcast server groups multiple preset three-dimensional character models according to crowd types, pre-trains multiple joint graph neural networks and skin graph neural networks, and after the anchor starts broadcasting, identifies the crowd type to which the target mesh model specified by the anchor belongs, and determines the joint graph neural network and skin graph neural network corresponding to the crowd type, so that the anchor obtains the target three-dimensional model for motion control; Alternatively, the live broadcast server pre-processes and stores target grid models corresponding to different crowd types, so that after the anchor starts broadcasting, it can identify the crowd type to which the anchor belongs, determine the target grid model that matches the crowd type to which the anchor belongs, and send the target grid model to the anchor's live broadcast client.

8. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images