Image processing method and apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose an image processing method and apparatus, a device, and a storage medium. The method comprises: acquiring vertex features of vertexes in a mesh model of an object and topological information of the mesh model; acquiring vertex relationship indication information, wherein the vertex relationship indication information is used for indicating vertexes having a constraint relationship in the mesh model, wherein the constraint relationship refers to a constraint relationship between two different vertexes; and inputting the vertex relationship indication information, the vertex features and the topological information of the mesh model into a skin weight prediction model, wherein the skin weight prediction model is used for predicting a skin weight of the mesh model on the basis of the vertex relationship indication information, the vertex features and the topological information of the mesh model. Thus, in a process of predicting the skin weight of a mesh model, the skin weights of vertexes in the mesh model are constrained by means of vertex relationship indication information, thereby improving the prediction accuracy of the skin matrix of the mesh model.

Description

Image processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 20, 2023, with application number 202310730294.2 and application name “A method, device, equipment and storage medium for image processing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to an image processing method, an image processing apparatus, a computer device, and a computer-readable storage medium. Background Art

[0003] With advancements in scientific research, the animation and gaming industries are developing rapidly. One of the core technologies in these industries is the mesh model of an object; for example, clothing models, character models, and scene models. Because an object's mesh model contains many vertices, its rotational and translational transformations are typically determined by the mesh's corresponding skin matrix. Research has found that the skin matrix corresponding to a mesh model is typically predicted using a skin weight prediction model, resulting in low prediction accuracy.

[0004] Summary of the Invention

[0005] Embodiments of the present application provide an image processing method, apparatus, device, and computer-readable storage medium that can improve the prediction accuracy of the skin matrix of a mesh model.

[0006] In one aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] Obtaining vertex features of each vertex in the mesh model of the object and topological information of the mesh model, wherein the topological information of the mesh model includes the positional relationship between the joint points of the object and each vertex in the mesh model, and the connection relationship between the vertices in the mesh model;

[0008] Obtaining vertex relationship indication information, where the vertex relationship indication information is used to indicate vertices in a mesh model that have a constraint relationship, where a constraint relationship refers to a constraint relationship between two different vertices;

[0009] Inputting the vertex relationship indication information, the vertex features and the topological information of the mesh model into a skin weight prediction model, the skin weight prediction model being used to: predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features and the topological information of the mesh model;

[0010] The skin weight of the mesh model is used to indicate the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation process of the mesh model.

[0011] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0012] an acquisition unit, configured to acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model, wherein the topological information of the mesh model includes positional relationships between joint points of the object and each vertex in the mesh model, and connection relationships between the vertices in the mesh model;

[0013] and for obtaining vertex relationship indication information, where the vertex relationship indication information is used to indicate vertices having a constraint relationship in the mesh model, where the constraint relationship refers to a constraint relationship between two different vertices;

[0014] a processing unit, configured to input the vertex relationship indication information, the vertex features, and the topology information of the mesh model into a skin weight prediction model, wherein the skin weight prediction model is configured to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features, and the topology information of the mesh model;

[0015] The skin weight of the mesh model is used to indicate the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation process of the mesh model.

[0016] Accordingly, the present application provides a computer device, comprising:

[0017] a memory, wherein a computer program is stored in the memory;

[0018] The processor is used to load a computer program to implement the above-mentioned image processing method.

[0019] Accordingly, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned image processing method.

[0020] Accordingly, the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, so that the computer device performs the above-mentioned image processing method.

[0021] In an embodiment of the present application, vertex features of each vertex in a mesh model of an object and topological information of the mesh model are obtained. By obtaining the topological information, the positional relationship between the joint points of the object and each vertex in the mesh model, as well as the connection relationship between each vertex in the mesh model can be obtained; vertex relationship indication information is obtained, and the vertex relationship indication information can be used to indicate vertices in the mesh model that have constraint relationships, and the constraint relationship refers to the constraint relationship between two different vertices; the vertex relationship indication information, the vertex features and the topological information of the mesh model are input into a skin weight prediction model, and the skin weight prediction model is used to: predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features and the topological information of the mesh model, and predict the skin weight by combining the vertex relationship indication information, the vertex features and the topological information. In the process of predicting the skin weight of the mesh model, the skin weight of each vertex in the mesh model can be constrained by the vertex relationship indication information, which can improve the prediction accuracy of the skin matrix of the mesh model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1a is a diagram of an image processing scene provided by an embodiment of the present application;

[0023] FIG1b is a schematic diagram of image processing provided by an embodiment of the present application;

[0024] FIG2 is a flow chart of an image processing method provided in an embodiment of the present application;

[0025] FIG3 is a schematic diagram of a neighborhood vertex provided in an embodiment of the present application;

[0026] FIG4 is a schematic diagram of the architecture of a skin weight prediction model provided in an embodiment of the present application;

[0027] FIG5 is a flowchart of another image processing method provided in an embodiment of the present application;

[0028] FIG6 is a schematic diagram of a key vertex provided in an embodiment of the present application;

[0029] FIG7 is a schematic diagram of establishing edges connecting key vertices according to an embodiment of the present application;

[0030] FIG8 is a schematic diagram of feature conversion of an adjacent submodule provided in an embodiment of the present application;

[0031] FIG9 is a schematic structural diagram of a residual submodule provided in an embodiment of the present application;

[0032] FIG10 is a comparison diagram of effects provided by an embodiment of the present application;

[0033] FIG11 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0034] FIG12 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] This application involves technologies related to artificial intelligence and modeling. The following is a brief introduction to the relevant technologies involved:

[0036] Artificial Intelligence (AI): The so-called AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The embodiment of the present application mainly involves predicting the skin weights of vertex features and topological information of the mesh model based on vertex relationship indication information through a skin weight prediction model to obtain the skin weights of the mesh model.

[0037] AI technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, large-scale application processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0038] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching techniques. The embodiment of the present application mainly trains the skin weight prediction model based on a sample data set to further improve the prediction accuracy of the skin weight prediction model.

[0039] Mesh model: In the gaming and modeling industries, an object can be approximated by a mesh (such as a triangle mesh or polygon mesh). The process of using a mesh to present an object is called modeling. The most basic primitive in the three-dimensional world is the triangle. Generally speaking, the three-dimensional models in the constructed three-dimensional world are hollow and represented by closed surfaces (mesh bodies). From a storage perspective, since a plane can be determined by three points, a three-dimensional model can be represented by closed faces; therefore, a mesh model can be composed of individual vertices, with neither "faces" nor "bodies", and no additional storage information is required, so as to achieve the purpose of maximizing compression. It can be seen that a three-dimensional mesh model appears to be composed of several triangles (polygons), and what is stored during storage are the vertices that make up the three-dimensional mesh model. A complete mesh model (mesh) can be described by a series of basic geometric information. It should be noted that the mesh model of the object involved in this application can be a three-dimensional mesh model or a two-dimensional mesh model, and this application does not impose any restrictions on this.

[0040] Skinning: A 3D animation term (also used in 3D games). It's a production technique for 3D animation. Based on a mesh model created in 3D software, joints are added to the mesh (joints can be customized based on actual needs; for example, for an avatar, joints can be the skeleton of the avatar). Since joints and the mesh are independent of each other, skinning is the process of binding the mesh to these joints, allowing the joints to drive the mesh's transformations (e.g., motion). The skinning process ultimately outputs a skinning matrix containing A*B skinning weights (i.e., the dimensions of the skinning matrix are A*B). Skinning weights are numerical values; where A is the number of vertices in the model and B is the number of joints. The skinning weight in the i-th row and j-th column of the skinning matrix indicates the degree to which the j-th joint influences the deformation of the i-th vertex during the object's motion. i is a positive integer less than or equal to A, and j is a positive integer less than or equal to B. Furthermore, for each vertex, the weights across all joints sum to 1 (i.e., the sum of the skinning weights in each row is 1).

[0041] Based on the above-mentioned artificial intelligence and modeling-related technologies, an embodiment of the present application provides an image processing solution to improve the prediction accuracy of the skin matrix of the mesh model. Figure 1a is an image processing scene diagram provided by an embodiment of the present application. As shown in Figure 1a, the image processing scene provided by the present application includes a computer device 101. The image processing solution provided by the present application can be executed by the computer device 101. The computer device 101 can be a terminal device or a server. Among them, the terminal device can include but is not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, referred to as MID), vehicle terminals, smart home appliances, wearable devices, etc., and the embodiment of the present application does not limit this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and basic cloud computing services such as big data and artificial intelligence platforms. The embodiment of the present application does not limit this.

[0042] It should be noted that the number of computer devices in Figure 1a is for example only and does not constitute an actual limitation of this application; for example, Figure 1a may also include computer device 102 (such as for transmitting vertex relationship indication information to computer device 101), and computer device 101 and computer device 102 can be connected via wired or wireless means, and this application does not impose any restrictions on this.

[0043] Figure 1b is a schematic diagram of an image processing method provided by an embodiment of the present application. As shown in Figure 1b, the general principles of the image processing method provided by the present application are as follows:

[0044] (1) The computer device 101 obtains the vertex features of each vertex in the mesh model of the object and the topological information of the mesh model; wherein the topological information of the mesh model includes the positional relationship between the joint points of the object and each vertex in the mesh model, as well as the connection relationship between each vertex in the mesh model. In one embodiment, the vertex feature of the vertex refers to the vertex parameter of the vertex, and the vertex feature of each vertex in the mesh model of the object includes at least one of the following: the position information of the vertex (such as the position of the vertex in the Cartesian coordinate system), the normal vector of the vertex, and the second distance between the vertex and each joint point of the object (such as the Euclidean distance). The topological information of the mesh model includes the vertex graph topological structure of the mesh model of the object and the joint point graph topological structure of the object; taking the vertex graph topological structure of the mesh model of the object as an example, the vertex graph topological structure of the mesh model of the object includes at least one of the following: the vertices in the mesh model of the object, the edges between the vertices in the mesh model of the object, and the adjacency matrix of each vertex in the mesh model of the object. In one embodiment, the vertex features of each vertex in the mesh model of the object and the topological information of the mesh model can be obtained by extracting the mesh model of the object.

[0045] (2) The computer device 101 obtains vertex relationship indication information, which is used to indicate vertices in a mesh model that have a constraint relationship. The constraint relationship refers to a constraint relationship between two different vertices. In one embodiment, the joint points of the object include at least one basic (universal) joint point and P additional joint points corresponding to the mesh model, where P is a positive integer. For different mesh models of the same object (such as different costumes of the same virtual character), the basic joint points of each mesh model are the same, and the additional joint points of each mesh model can be different. The computer device 101 obtains P key areas (key areas can be indicated by the modeler) that correspond one to one to the P additional joint points in the mesh model, each key area includes at least two vertices, and the vertices in each key area are screened to obtain P key vertex sets. Based on the P key vertex sets, vertex relationship indication information is generated. The existence of a constraint relationship between vertex A and vertex B can be understood as: vertex A and vertex B are different key vertices in the same key vertex set (vertex A and vertex B belong to the same key area), and the difference between the skin weight of vertex A and the skin weight of vertex B is less than the difference threshold.

[0046] (3) The computer device 101 skin weight prediction model is used to predict the skin weight of the mesh model based on vertex relationship indication information, vertex features and topological information of the mesh model; wherein the skin weight of the mesh model is used to indicate the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation of the mesh model. In one implementation, the computer device can convert the vertex relationship indication information, vertex features and topological information of the mesh model to obtain model input features of the skin weight prediction model, which can specifically include: basic features; for example, the position of the vertex, the normal vector, the distance between the vertex and the joint point, etc.; adjacency features (hint neighbor matrix); for example, the adjacency matrix of the mesh model; aggregation features (hint attention weights); for example, features obtained by aggregating the features of the vertex. Among them, the adjacency features and the aggregation features are two features with similarity constraints on the inner and outer skin weights.

[0047] In one embodiment, the skin weight prediction model includes a neural network layer and M feature fusion modules, where M is a positive integer; the computer device 101 calls the M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model to obtain the fusion features of each vertex. Specifically, the computer device 101 uses the vertex relationship indication information as constraint information, and constrains the transformation process of the vertex features of each vertex through the constraint information (such as making the difference between the transformation features of vertices with a constraint relationship less than the feature difference threshold), to obtain the transformation features of each vertex, and then performs feature splicing processing on the transformation features of each vertex and the vertex features of the vertex to obtain the fusion features of each vertex. After obtaining the fusion features of each vertex, the computer device 101 maps the fusion features of each vertex and the topological information of the mesh model through the neural network layer to obtain the skin weight of the mesh model.

[0048] In an embodiment of the present application, vertex features of each vertex in a mesh model of an object and topological information of the mesh model are obtained, and vertex relationship indication information is obtained. The vertex relationship indication information is used to indicate vertices in the mesh model that have a constraint relationship, where the constraint relationship refers to a constraint relationship between two different vertices. A skin weight prediction model is used to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features, and the topological information of the mesh model. It can be seen that in the process of predicting the skin weight of the mesh model, constraining the skin weight of each vertex in the mesh model through the vertex relationship indication information can improve the prediction accuracy of the skin matrix of the mesh model.

[0049] Please refer to Figure 2, which is a flowchart of an image processing method provided in an embodiment of the present application. The image processing method can be executed by a computer device, which can specifically be a terminal device or a server. As shown in Figure 2, the image processing method may include the following steps S201-S203:

[0050] S201: Acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model.

[0051] The vertex features of each vertex in the mesh model of the object include at least one of the following: vertex position information (such as the position of the vertex in the Cartesian coordinate system), the normal vector of the vertex, and a second distance (such as Euclidean distance) between the vertex and each joint point of the object.

[0052] In one embodiment, the computer device obtains the position information and normal vector of each vertex in the mesh model of the object and, based on the position information of each vertex, calculates a second distance (e.g., a Euclidean distance) between the vertex and each joint point of the object. The computer device then combines the position information, normal vector, and second distance between the vertex and each joint point of the object to obtain a vertex feature for the vertex.

[0053] The topological information of the mesh model can be used to indicate the positional relationship between the joint points of the object and each vertex in the mesh model, as well as the connection relationship between each vertex in the mesh model. The topological information of the mesh model includes the vertex graph topology structure of the mesh model of the object and the joint point graph topology structure of the object. The vertex graph topology structure of the mesh model of the object includes at least one of the following: the vertices in the mesh model of the object, the edges between the vertices in the mesh model of the object, and the adjacency matrix of each vertex in the mesh model of the object. In one embodiment, a computer device performs topological graph information extraction processing on the mesh model of the object to obtain the topological information of the mesh model.

[0054] S202: Obtain vertex relationship indication information.

[0055] The vertex relationship indication information is used to indicate vertices with a constraint relationship in the mesh model. The constraint relationship refers to a constraint relationship between two different vertices.

[0056] In one embodiment, the joint points of the object include at least one basic (universal) joint point and P additional joint points corresponding to the mesh model, where P is a positive integer. For different mesh models of the same object (such as different costumes of the same virtual character), the basic joint points of each mesh model are the same, and the additional joint points of each mesh model may be different. The computer device obtains P key areas (key areas can be indicated by the modeler) that correspond one to one to the P additional joint points in the mesh model, each key area includes at least two vertices, and the vertices in each key area are screened to obtain P key vertex sets, and vertex relationship indication information is generated based on the P key vertex sets. The existence of a constraint relationship between vertex A and vertex B can be understood as follows: vertex A and vertex B are different key vertices in the same key vertex set (vertex A and vertex B belong to the same key area), and the difference between the skin weight of vertex A and the skin weight of vertex B is less than the difference threshold.

[0057] In one embodiment, a computer device screens and processes the vertices in each key area to obtain a set of P key vertices, the process including: the qth key area is any one of the P key areas, the qth key area corresponds to the qth additional joint point among the P additional joint points, and the target vertex in the qth key area with the closest second distance (Euclidean distance) to the qth additional joint point, and the neighboring vertices of the target vertex are added to the candidate vertex set; the neighboring vertices of the target vertex refer to vertices within S hops from the target vertex (that is, vertices that can reach vertex A through at least S non-repeated edges that cannot form a loop), q is a positive integer less than or equal to P, and S is a positive integer (which can be dynamically adjusted based on actual conditions).

[0058] Figure 3 is a schematic diagram of a neighborhood vertex provided by an embodiment of the present application. As shown in Figure 3, vertex A is connected to vertex B, vertex C and vertex D respectively (there are edges), then the one-hop neighborhood vertices of vertex A include: vertex B, vertex C and vertex D (that is, the vertices that can reach vertex A through at least one edge); similarly, vertex E and vertex F are indirectly connected to vertex A through vertex B, vertex G is indirectly connected to vertex A through vertex C, and vertex H, vertex I, vertex J and vertex K are indirectly connected to vertex A through vertex D, then the two-hop neighborhood vertices of vertex A include: vertex E, vertex F, vertex G, vertex H, vertex I, vertex J and vertex K (that is, the vertices that can reach vertex A through at least two non-repeating edges that cannot form a loop).

[0059] Furthermore, the computer device obtains the labeling weights of each candidate vertex in the candidate vertex set, as well as the labeling weights of the neighborhood vertices of each candidate vertex, and performs mean processing on the differences between the labeling weights of each candidate vertex in the candidate vertex set and the labeling weights of the neighborhood vertices of the candidate vertex to obtain the difference mean corresponding to each candidate vertex; wherein, the labeling weight of each candidate vertex is used to indicate the degree of influence of the qth additional joint point on the deformation of the candidate vertex during the deformation process of the mesh model of the object; the labeling weight of the neighborhood vertices of each candidate vertex is used to indicate the influence weight of the qth additional joint point on the neighborhood vertex during the deformation process of the mesh model of the object.

[0060] For example, assuming that the label weight of candidate vertex A is 0.22, the neighborhood vertices of candidate vertex A include vertex B and vertex C, the label weight of vertex B is 0.19, and the label weight of vertex C is 0.24; then the difference in label weights between candidate vertex A and vertex B is |0.22-0.19|=0.03, the difference in label weights between candidate vertex A and vertex C is |0.22-0.24|=0.02, and the corresponding mean difference of candidate vertex A is (0.03+0.02) / 2=0.025.

[0061] After obtaining the difference mean corresponding to each candidate vertex, the computer device determines the candidate vertex with the largest difference mean as the representative vertex of the candidate vertex set, and adds the first vertex in the qth key area to the qth key vertex set (the key vertices in each key vertex set are all vertices in the mesh model). The difference between the difference mean of the first vertex and the difference mean of the representative vertex is less than the mean threshold, and the qth key area corresponds to the qth key vertex set in the P key vertex sets. For example, suppose that the candidate vertex set includes candidate vertex A, candidate vertex B and candidate vertex C; the difference mean corresponding to candidate vertex A is 0.31, the difference mean corresponding to candidate vertex B is 0.35, and the difference mean corresponding to candidate vertex C is 0.28; then candidate vertex B is the representative vertex of the candidate vertex set. If the qth key area includes, in addition to candidate vertex A, candidate vertex B, and candidate vertex C, vertex E, vertex F, vertex G and vertex H, and the difference mean corresponding to vertex E is 0.32, the difference mean corresponding to vertex F is 0.21, the difference mean corresponding to vertex G is 0.29, the difference mean corresponding to vertex H is 0.3, and the mean threshold is 0.08; then the first vertex added to the qth key vertex set includes candidate vertex A, candidate vertex B, candidate vertex C, vertex E, vertex G and vertex H.

[0062] S203 , inputting the vertex relationship indication information, vertex features and topological information of the mesh model into a skin weight prediction model, the skin weight prediction model being used to predict the skin weight of the mesh model based on the vertex relationship indication information, vertex features and topological information of the mesh model.

[0063] The skin weight of the mesh model includes the skin weights of the joint points on each vertex in the mesh model, and the skin weight is a numerical value. The skin weight of the mesh model is used to indicate the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation of the mesh model. The magnitude of the deformation influence can be quantified by the numerical value of the skin weight. The degree of deformation influence is positively correlated with the numerical value of the skin weight. The larger the numerical value of the skin weight, the greater the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation of the mesh model. The smaller the numerical value of the skin weight, the smaller the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation of the mesh model. For example, the skin weight of joint point 1 on vertex 1 is greater than the skin weight of joint point 1 on vertex 2. Then, during the deformation process of the mesh model, the influence of joint point 1 on the deformation of vertex 1 is greater than the influence of joint point 2 on vertex 2. If the deformation of joint point 1 refers to a change in the position of joint point 1, the position change of vertex 1 following joint point 1 is greater than the position change of vertex 2 following joint point 1.

[0064] In one embodiment, the skin weight prediction model includes a neural network layer and M feature fusion modules, where M is a positive integer; the computer device calls the M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model to obtain the fused features of each vertex. Feature fusion processing refers to transforming the vertex features of each vertex in the mesh model based on the vertex relationship indication information to obtain the transformed features of each vertex in the mesh model, and then splicing the transformed features of each vertex in the mesh model with the vertex features of each vertex in the mesh model. Specifically, the computer device uses the vertex relationship indication information as constraint information, and constrains the transformation process of the vertex features of each vertex through the constraint information (such as making the difference between the transformed features of vertices with a constraint relationship less than a feature difference threshold) to obtain the transformed features of each vertex, and then performs feature splicing processing on the transformed features of each vertex with the vertex features of the vertex to obtain the fused features of each vertex. Then, the neural network layer maps the fused features of each vertex to the topological information of the mesh model to obtain the skin weight of the mesh model.

[0065] FIG4 is a schematic diagram of the architecture of a skin weight prediction model provided by an embodiment of the present application. As shown in FIG4 , the skin weight prediction model includes a neural network layer and M feature fusion modules. The output of the i-th feature fusion module is the input of the i+1-th fusion module, where i is a positive integer less than M; the input of the first feature fusion module includes the vertex features of each vertex in the mesh model of the object and the topological information of the mesh model. After obtaining the fusion features of each vertex in the mesh model of the object through the M feature fusion modules (this process can be constrained based on vertex relationship indication information), the fusion features of each vertex are mapped through a neural network layer (such as a multi-layer perceptron (MLP)) to obtain the skin weight of the mesh model.

[0066] In one embodiment, the vertex relationship indication information includes P key vertex sets (weight hints point sets), and the key vertices (Hint vertices) in each key vertex set are all vertices in the mesh model, and P is a positive integer. Each feature fusion module includes an attention submodule (Hints Attention), a residual submodule (Residual Vertex) and an adjacent submodule (Hints Neighbor). Among them, the adjacent submodule is used to determine the neighborhood features of each vertex based on the vertex features of each vertex; the attention submodule is used to aggregate the vertex features of each vertex into P key vertex sets to obtain P aggregated features; the residual submodule is used to extract the deep features of each vertex based on the vertex features of each vertex. The computer device calls M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model, and the process of obtaining the fusion features of each vertex is as follows:

[0067] On the one hand, the computer device obtains P first adjacency matrices corresponding to P key vertex sets. In one embodiment, the computer device establishes edges between each key vertex in each key vertex set. After establishing edges between each key vertex in each key vertex set, the computer device obtains P first adjacency matrices corresponding to the P key vertex sets based on the connection relationship between each key vertex in the P key vertex sets. It can be understood that after establishing edges between each key vertex in each key vertex set, there are edges between any key vertex in each key vertex set and the remaining key vertices. For example, assuming that the qth key vertex set includes key vertex 1-key vertex 5, there are edges between key vertex 1 and key vertex 2-key vertex 5.

[0068] Furthermore, the computer device performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule (Hints Neighbor) to obtain the neighborhood features of each vertex. Specifically, the computer device determines the k neighbor vertices of the key vertex j through the qth first adjacency matrix, the qth first adjacency matrix is ​​the first adjacency matrix of the qth key vertex set, the key vertex j belongs to the qth key vertex set, and k is a positive integer. After determining the k neighbor vertices of the key vertex j, the difference between the vertex feature of the key vertex j and the vertex feature of each neighbor vertex in the k neighbor vertices is calculated respectively to obtain k difference features, and the maximum value of the k difference features is determined as the neighborhood feature of the key vertex j.

[0069] On the other hand, the computer device calls the attention submodule (Hints Attention) to aggregate the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregated features. The P aggregated features correspond one-to-one to the P key vertex sets, and the qth aggregated feature is used to indicate the skin weight of the vertex in the mesh model, which is similar to the skin weight of each key vertex in the qth key vertex set. The specific implementation method of the computer device calling the attention submodule to aggregate the vertex features of each vertex in the mesh model into the qth key vertex set to obtain the qth aggregated feature is as follows: obtain the qth weight corresponding to each vertex in the mesh model, q is a positive integer less than or equal to P, perform weighted processing on the vertex features of each vertex according to the qth weight corresponding to the vertex, obtain the weighted features of each vertex, and perform aggregation processing on the weighted features of each vertex to obtain the qth aggregated feature.

[0070] On the other hand, the computer device performs deep feature extraction processing on the vertex features of each vertex through a residual submodule (Residual Vertex) to obtain the deep features of each vertex.

[0071] After obtaining the neighborhood features, P aggregate features, and depth features of each vertex, the computer device splices the neighborhood features, P aggregate features, and depth features of each vertex to obtain the fused features of each vertex in the mesh model of the object.

[0072] In another embodiment, the object has N joints, where N is a positive integer. The computer device invokes a skinning weight prediction model to predict N predicted weights for each vertex in the mesh model based on vertex relationship indication information, vertex features, and topological information of the mesh model. The N joints of the object correspond one-to-one to the N predicted weights. For vertex h, the predicted weight output by the skinning weight prediction model can be expressed as: B is the number of joint points. huis the u-th prediction weight of vertex h, which is used to indicate the degree of influence of joint point u on the deformation of vertex h during the deformation of the mesh model; vertex h is any vertex in the mesh model, and the u-th prediction weight corresponds to joint point u, where u is a positive integer less than or equal to N; 0≤w hu ≤1,

[0073] After obtaining the N predicted weights for each vertex, the computer device selects the largest R (e.g., R = 3) predicted weights from the N predicted weights for each vertex as the R key weights for each vertex, and normalizes the R key weights for each vertex to obtain the skin weight of the mesh model, where R is an integer greater than 1 and less than or equal to N. Through this embodiment, the sparsity of the skin weights of the mesh model can be further enhanced, and the accuracy of the weight prediction results can be improved.

[0074] For example, assuming that the number of joints of an object is 4, the computer device calls the skin weight prediction model to predict the skin weights of the vertex features and the topological information of the mesh model based on the vertex relationship indication information, and obtains 4 predicted weights of the vertex h in the mesh model of the object, the predicted weight of joint point 1 is 0.5, the predicted weight of joint point 2 is 0.3, the predicted weight of joint point 3 is 0.15, and the predicted weight of joint point 4 is 0.05; if R=2, the computer device retains the predicted weight of joint point 1 and the predicted weight of joint point 2 (for joint point 3, the predicted weight is 0.05). The prediction weight of joint point 1 and the prediction weight of joint point 4 are set to zero), and the prediction weight of joint point 1 and the prediction weight of joint point 2 are normalized, and the first skin weight of vertex h (used to indicate the degree of influence of joint point 2 on the deformation of vertex h during the deformation of the mesh model) is 0.625; the second skin weight of vertex h (used to indicate the degree of influence of joint point 2 on the deformation of vertex h during the deformation of the mesh model) is 0.375, and the third skin weight of vertex h (used to indicate the degree of influence of joint point 3 on vertex h during the deformation of the mesh model) and the fourth skin weight (used to indicate the degree of influence of joint point 4 on vertex h during the deformation of the mesh model) are both 0.

[0075] In an embodiment of the present application, vertex features of each vertex in a mesh model of an object and topological information of the mesh model are obtained, and vertex relationship indication information is obtained. The vertex relationship indication information is used to indicate vertices in the mesh model that have a constraint relationship, where the constraint relationship refers to a constraint relationship between two different vertices. A skin weight prediction model is used to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features, and the topological information of the mesh model. It can be seen that in the process of predicting the skin weight of the mesh model, constraining the skin weight of each vertex in the mesh model through the vertex relationship indication information can improve the prediction accuracy of the skin matrix of the mesh model. In addition, the adjacency submodule performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex, and the key vertex set can be added to the second adjacency matrix to aggregate the neighborhood vertex features in the mesh model (such as different layers of mesh) to achieve the purpose of constraining the skin weights between vertices; the residual submodule performs deep feature extraction processing on the vertex features of each vertex, and the deep features (deeper features) of each vertex in the mesh model can be effectively extracted to further improve the prediction accuracy of the skin weight prediction model; the attention submodule is called to aggregate the vertex features of each vertex in the mesh model into each key vertex set, and the vertex features of the vertices in the mesh model can be aggregated into P key vertex sets according to adaptive weights, and the vertex features of the key vertices in the P key vertex sets can also be effectively transferred to the remaining vertices (non-key vertices) in the mesh model.

[0076] Please refer to Figure 5, which is a flowchart of another image processing method provided by an embodiment of the present application. The image processing method can be executed by a computer device, which can specifically be a terminal device or a server. As shown in Figure 5, the image processing method may include the following steps S501-S508:

[0077] S501: Acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model.

[0078] The vertex feature of each vertex in the mesh model of the object includes at least one of the following: the position information of the vertex, the normal vector of the vertex, and the second distance (such as the Euclidean distance) between the vertex and each joint point of the object. In one embodiment, the vertex feature of vertex h can be represented by a feature vector, which can be specifically expressed as:

[0079] in, Used to represent the position of vertex h in the Cartesian coordinate system, It can be represented by a three-dimensional vector (p hx ,p hy ,p hz ) to indicate that phx represents the x-axis coordinate of vertex h in the Cartesian coordinate system, p hy represents the y-axis coordinate of vertex h in the Cartesian coordinate system, p hz Represents the z-axis coordinate of vertex h in the Cartesian coordinate system. Used to represent the normal vector of vertex h, It can be represented by a three-dimensional vector (n hx ,n hy ,n hz ) to indicate. It is used to represent the Euclidean distance from vertex h to each joint point of the object. If the number of joint points of the object is B (i.e. There are a total of |B| dimensions), then

[0080] Furthermore, according to the above-mentioned vector representation of vertex features, the dimension of vertex features of N vertices can be expressed as: (N,6+|B|)

[0081] Where N is the total number of vertices in the object's mesh model, and 6+|B| is the dimension of the feature vector of each vertex.

[0082] The topological information of the mesh model includes the vertex graph topology structure of the mesh model of the object and the joint graph topology structure of the object, which can be specifically expressed as: G = (G m ,G j )

[0083] Among them, G m G is the vertex graph topology of the mesh model of the object; m It can be further expressed as G m =(V m ,ε m ,A m ), where V m Represents the vertices in the object's mesh model, Represents the edges in the mesh model of the object, A m It is a vertex adjacency matrix composed of values ​​(0,1) with a size of N×N, where N is the total number of vertices in the mesh model of the object. The connection relationship between each vertex in the network model can be expressed using the vertex adjacency matrix A. m Indicates that A m It can be used to indicate the connectivity between vertex pairs (i.e. two vertices) in the mesh model of an object. If A m (i,j)=1, it means there is a connecting edge between vertex i and vertex j. If A m(i,j)=0, it means that there is no connecting edge between vertex i and vertex j. For any vertex i in the mesh model of the object, its adjacent node set can be expressed as N i To represent; for example, the one-hop adjacent nodes of vertex i include the vertices in the mesh model of the object that have edges with vertex i (directly connected to vertex i).

[0084] S502: Obtain vertex relationship indication information.

[0085] In one embodiment, the object's joints include at least one basic (universal) joint and P additional joints corresponding to the mesh model, where P is a positive integer. For different mesh models of the same object (such as different costumes for the same virtual character), the basic joints of each mesh model are the same, but the additional joints of each mesh model can be different. The following is an example of determining the key vertex set (weight hints point set) corresponding to the qth additional joint point to explain in detail:

[0086] (1) The computer device obtains the key area corresponding to the qth additional joint point (the key area can be indicated by the modeler) and calculates the Euclidean distance between each vertex in the key area corresponding to the qth additional joint point and the qth additional joint point. i ), and the neighboring vertices of the target vertex are added to the candidate vertex set to obtain the candidate vertex set (such as C i The second distance (Euclidean distance) between the target vertex and the qth additional joint point is smaller than the second distance (Euclidean distance) between the vertices other than the target vertex and the qth additional joint point in the qth key region; that is, in the qth key region, the (Euclidean distance) between the target vertex and the qth additional joint point is the shortest.

[0087] (2) The computer device calculates the difference in labeling weights between each candidate vertex in the candidate vertex set and the adjacent vertices of the candidate vertex (i.e., vertices in the neighborhood), and calculates the mean difference corresponding to each candidate vertex based on the difference in labeling weights between each candidate vertex and the adjacent vertices of the candidate vertex. Specifically, the computer device obtains the labeling weights of each candidate vertex in the candidate vertex set, as well as the labeling weights of the neighborhood vertices of each candidate vertex, and performs mean processing on the difference between the labeling weights of each candidate vertex in the candidate vertex set and the labeling weights of the neighborhood vertices of the candidate vertex, to obtain the mean difference corresponding to each candidate vertex; wherein, the labeling weight of each candidate vertex is used to indicate the degree of influence of the qth additional joint point on the deformation of the candidate vertex during the deformation of the mesh model of the object; the labeling weight of the neighborhood vertices of each candidate vertex is used to indicate the influence weight of the qth additional joint point on the neighborhood vertex during the deformation of the mesh model of the object. The mean difference corresponding to candidate vertex j can be expressed as:

[0088] Among them, d j represents the mean difference value corresponding to candidate vertex j, m is the number of adjacent nodes (i.e., vertices in the neighborhood) of candidate vertex j; weights j Represents the label weight of candidate vertex j, weights k Represents the label weight of the adjacent nodes of candidate vertex j.

[0089] (3) After obtaining the mean difference value corresponding to each candidate vertex, the computer device determines the candidate vertex with the largest mean difference value as the representative vertex of the candidate vertex set. For example, assuming that the candidate vertex set includes candidate vertex A, candidate vertex B, and candidate vertex C; the mean difference value corresponding to candidate vertex A is 0.31, the mean difference value corresponding to candidate vertex B is 0.35, and the mean difference value corresponding to candidate vertex C is 0.28; then candidate vertex B is the representative vertex of the candidate vertex set.

[0090] (4) The computer device adds the first vertex in the key region corresponding to the qth additional joint point (i.e., the qth key region) to the qth key vertex set, and the difference between the difference mean of the first vertex and the difference mean of the representative vertex is less than the mean threshold. For example, assuming that candidate vertex B in the candidate vertex set is the representative vertex of the candidate vertex set (the difference mean corresponding to candidate vertex B is 0.35), if the qth key region includes vertex E, vertex F, vertex G, and vertex H, and the difference mean corresponding to vertex E is 0.32, the difference mean corresponding to vertex F is 0.21, the difference mean corresponding to vertex G is 0.29, the difference mean corresponding to vertex H is 0.3, and the mean threshold is 0.08; then the qth key vertex set includes vertex E, vertex G, and vertex H.

[0091] According to the above-mentioned key vertex set determination process, the computer device can obtain the key vertex set corresponding to each additional joint point, and generate vertex relationship indication information based on the key vertex sets corresponding to the P additional joint points.

[0092] FIG6 is a schematic diagram of a key vertex provided in an embodiment of the present application. As shown in FIG6 , the mesh model of the object in the schematic diagram includes four key areas; that is, the mesh model of the object corresponds to four additional joint points. The bold vertices in each key area are the key vertices belonging to the same key vertex set determined according to the above-mentioned embodiment. It should be noted that the shape of the key area is for example only and does not constitute an actual limitation of the present application; for example, the shape of the key area can also be a circle, other polygons, etc.

[0093] It is understandable that the modeler can also directly indicate the set of key vertices corresponding to each additional joint point.

[0094] S503: Obtain P first adjacency matrices corresponding to the P key vertex sets.

[0095] In one embodiment, the computer device establishes edges between each key vertex in each key vertex set, and obtains P first adjacency matrices corresponding to the P key vertex sets based on the mesh model after establishing the edges. In one embodiment, if there is no edge between key vertex i and key vertex j in the qth key vertex set, the computer device establishes an edge between key vertex i and key vertex j; where q is a positive integer less than or equal to P, and key vertex i and key vertex j are any two key vertices in the qth key vertex set.

[0096] It is understandable that after the edges between the key vertices in each key vertex set are established, any key vertex in each key vertex set and the other key vertices in the key vertex set are adjacent vertices to each other.

[0097] FIG7 is a schematic diagram of establishing edges connecting key vertices provided by an embodiment of the present application. As shown in FIG7 , assume that the key vertex set includes key vertex A, key vertex B, key vertex C, and key vertex D; as shown in 701, before establishing the edges, there is an edge between key vertex A and key vertex C, and an edge between key vertex B and key vertex D; as shown in 702, after establishing the edges (the dotted edges are newly added edges), any key vertex among key vertex A, key vertex B, key vertex C, and key vertex D has an edge with the other three key vertices.

[0098] In practical applications, by establishing connecting edges between key vertices, adjacency relationships can be established for vertices with constrained relationships in the mesh model (such as vertices corresponding to inner and outer layers of clothing). In the subsequent skin weight prediction process, the skin weight prediction model's ability to learn the skin weight similarity characteristics of vertices with constrained relationships in the mesh model can be improved.

[0099] S504 : Perform feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule to obtain neighborhood features of each vertex.

[0100] In one embodiment, the computer device performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule to obtain the neighborhood features of each vertex, which includes: determining the k neighboring vertices of the key vertex j through the qth first adjacency matrix, the qth first adjacency matrix is ​​the first adjacency matrix of the qth key vertex set, the key vertex j belongs to the qth key vertex set, and k is a positive integer. Specifically, the topological information of the mesh model includes the second adjacency matrix (i.e., the A in step S501) m ), the second adjacency matrix is ​​obtained based on the connection relationship between each vertex in the mesh model (before establishing the edges between each key vertex in each key vertex set); that is, the second adjacency matrix can be used to indicate the connection relationship between each vertex in the mesh model. The computer device determines the T key vertices that have edges with the key vertex j through the qth first adjacency matrix, where T is a positive integer. If T is greater than or equal to k, the computer device randomly selects k vertices from the T key vertices as the k neighbor vertices of the key vertex j; correspondingly, if T is less than k, the computer device determines the V vertices that have edges with the key vertex j through the second adjacency matrix, and randomly selects kT vertices from the V vertices, and uses the T key vertices and the selected kT vertices as the k neighbor vertices of the key vertex j, where V is a positive integer.

[0101] After determining the k neighboring vertices of the key vertex j, the computer device calculates the difference between the vertex feature of the key vertex j and the vertex feature of each of the k neighboring vertices to obtain k difference features. The k difference features can be specifically represented as: {x j1 -x j ,x j2 -x j …,x jk -x j}

[0102] Among them, x jk -x j It represents the difference between the vertex feature of key vertex j and the vertex feature of the kth neighbor vertex of key vertex j, which can be recorded as h(x j ,xjk ), that is, h(x j ,x jk )=(x jk -x j ). {x j1 ,x j2 …,x jk} are the vertex features of the k neighboring vertices of the key vertex j.

[0103] Furthermore, the computer device determines the maximum value among the k difference features as the neighborhood feature of the key vertex j.

[0104] Figure 8 is a schematic diagram of feature conversion of an adjacency submodule provided in an embodiment of the present application. As shown in Figure 8, the computer device first determines the k neighbor vertices of each vertex through the vertex features of each vertex in the grid model and the adjacency matrix of each vertex (including at least one of the first adjacency matrix and the second adjacency matrix) (such as through the nearest neighbor algorithm (K-Nearest Neighbor, KNN)); wherein, the vertex features of each vertex in the grid model are expressed as (N, C), N is the number of vertices, and C is the vertex features; the adjacency matrix of the grid model is expressed as (N, N), N is the number of vertices; (N, K, C) includes the vertex features of each vertex and the k neighbor vertices of each vertex. Then, a fully connected multilayer perceptron (MLP) is used to perform feature transformation based on the vertex features of each vertex and the vertex features of the k neighbor vertices of the vertex to obtain k difference features of each vertex, and (N, K, H) includes the difference features between the vertex features of each vertex and the vertex features of the k neighbor vertices of the vertex. The maximum difference feature among the k difference features of each vertex is retained, and the neighborhood feature of each vertex is expressed as (N,H).

[0105] The adjacency submodule performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex to obtain the neighborhood features of each vertex. The key vertex set can be added to the second adjacency matrix to aggregate the neighborhood vertex features in the mesh model (such as different layers of mesh) to achieve the purpose of constraining the skinning weights between vertices (that is, making the vertex features of vertices with constraint relationships as similar as possible).

[0106] S505: Call the attention submodule to aggregate the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregated features.

[0107] The P aggregate features correspond one-to-one to the P key vertex sets. The qth aggregate feature is used to indicate the skin weight of the vertex in the mesh model and its similarity with the skin weight of each key vertex in the qth key vertex set. q is a positive integer less than or equal to P.

[0108] In one embodiment, the computer device calls the attention submodule to aggregate the vertex features of each vertex in the mesh model into each key vertex set, and the process of obtaining P aggregated features includes: obtaining the qth weight corresponding to each vertex in the mesh model, where q is a positive integer less than or equal to P. Specifically, the process of the computer device obtaining the qth weight corresponding to each vertex in the mesh model includes: performing average processing on the first distance between the key vertices in the qth key vertex set and the vertex h, and obtaining the distance between the vertex h and the qth key vertex set (each key vertex set is taken as a whole, and the qth key vertex set is represented as G q ) (i.e., the qth distance mean corresponding to vertex h); vertex h is any vertex in the mesh model. In one embodiment, the first distance between key vertex j in the qth key vertex set and vertex h is the shortest distance from key vertex j to vertex h via the edges in the mesh model (which can be calculated using Dijkstra's algorithm). The qth distance mean corresponding to vertex h can be expressed as:

[0109] Among them, hint_dist qh Represents the qth distance mean of vertex h (it can be understood that the distance from the key vertex in the qth key vertex set to vertex h is hint_dist qh ); m is the number of key vertices in the qth key vertex set, dijkstra(v q ,v h ) represents the shortest distance between vertex h and key vertex q in the qth key vertex set on the mesh model.

[0110] Furthermore, the computer device determines the qth weight of vertex h based on the derivative result of the qth distance mean corresponding to vertex h. In one embodiment, the calculation of the qth weight of vertex h based on the derivative result of the qth distance mean corresponding to vertex h can be expressed as:

[0111] Among them, hint_attention_weights qh Indicates the qth weight corresponding to vertex h, hint_dist qh represents the qth distance mean of vertex h; a is a dynamic parameter that can be adjusted based on actual conditions, for example, a=4.

[0112] Furthermore, the computer device performs weighted processing on the vertex features of each vertex using the qth weight value corresponding to the vertex to obtain the weighted features of each vertex, and aggregates the weighted features of each vertex to obtain the qth aggregated feature. Specifically, it can be expressed as:

[0113] Among them, z q is the qth aggregate feature, hint_attention_weights qh represents the qth weight corresponding to vertex h, N is the number of vertices in the mesh model, f h is the vertex feature of vertex h.

[0114] The attention submodule is called to aggregate the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregated features. The vertex features of the vertices in the mesh model can be aggregated into P key vertex sets according to the adaptive weights (hint_attention_weights). At the same time, the vertex features of the key vertices in the P key vertex sets can be effectively transferred to the remaining vertices (non-key vertices) in the mesh model.

[0115] S506 , performing depth feature extraction processing on the vertex features of each vertex through the residual sub-module to obtain the depth features of each vertex.

[0116] In one embodiment, the residual submodule includes a neural network layer and an activation layer. The computer device performs deep feature extraction processing on the vertex features of each vertex through the residual submodule to obtain the deep features of each vertex, including: mapping the vertex features of vertex j through the neural network layer to obtain a feature mapping result of vertex j; activating the feature mapping result of vertex j using the activation layer to obtain an activation feature of vertex j; and superimposing the activation feature of vertex j with the vertex features of vertex j to obtain a deep feature of vertex j.

[0117] Figure 9 is a schematic diagram of the structure of a residual submodule provided in an embodiment of the present application. As shown in Figure 9, the residual submodule includes two multi-layer perceptrons (neural network layers) and two activation layers; the multi-layer perceptrons are used to map the vertex features of the vertices to obtain feature mapping results; the activation layers are used to activate the feature mapping results, and the activation function used by the activation layers can be a ReLU (Linear Rectification Function) activation function.

[0118] By performing deep feature extraction processing on the vertex features of each vertex through the residual sub-module, the deep features (deeper features) of each vertex in the mesh model can be effectively extracted, further improving the prediction accuracy of the skin weight prediction model.

[0119] It should be noted that steps S504 to S506 can be executed in any order; for example, step S506 is executed first, then step S504, and then step S505; they can also be executed simultaneously, and this application does not impose any restrictions on this.

[0120] S507: Concatenate the neighborhood features of each vertex, the P aggregate features, and the depth features of each vertex to obtain fused features of each vertex in the mesh model.

[0121] In one embodiment, the computer device can splice the neighborhood features of each vertex, P aggregation features and the depth features of each vertex through a fusion model (such as a fusion network) to obtain the fusion features of each vertex in the grid model.

[0122] S508 , mapping the fusion features of each vertex and the topological information of the mesh model through a neural network layer to obtain the skin weight of the mesh model.

[0123] In step S508, the fusion features of each vertex and the topological information of the mesh model can be input into the neural network layer as input information of the neural network layer. The neural network layer is used to map the input information to obtain the skin weight of the mesh model. The mapping process is used to perform nonlinear transformation on the input information and introduce nonlinearity.

[0124] Skin weights are used to indicate the degree to which the object's joints influence the deformation of each vertex in the mesh model during the deformation of the mesh model.

[0125] Furthermore, the computer device may also obtain the rotation and translation matrices of the object's joints and, based on the mesh model's skinning weights and the rotation and translation matrices, perform rotation and translation processing on each vertex in the mesh model to obtain a rotation and translation result of the mesh model. The mesh model may include any of the following: a multi-layer mesh model, a single-layer mesh model, or a combined mesh model (which may be obtained by combining multiple single-layer mesh models).

[0126] In one embodiment, the object may be a virtual character in a game, and the mesh model may be the virtual character's clothing. The computer device obtains vertex features of each vertex in the virtual character's clothing, topological information of the clothing, and vertex relationship indication information. A skin weight prediction model is then used to predict the clothing's skin weight based on the vertex relationship indication information, vertex features, and topological information of the clothing. Using the skin weights of different clothing items, the computer device can render images of the virtual character wearing different clothing items. Using the skin weights of target clothing items, the computer device can also render images of the virtual character wearing the target clothing item in different poses.

[0127] In another embodiment, the object may be an animated character in a three-dimensional animation, and the mesh model may be a three-dimensional garment. The computer device obtains vertex features of each vertex in the animated character's three-dimensional garment, topological information of the three-dimensional garment, and vertex relationship indication information. A skin weight prediction model is then used to predict the skin weight of the three-dimensional garment based on the vertex relationship indication information, the vertex features, and the topological information of the three-dimensional garment. Using the skin weights of different three-dimensional garments, the computer device can render a three-dimensional animation of the animated character wearing different three-dimensional garments. Using the skin weights of a target three-dimensional garment, the computer device can render a three-dimensional animation of the animated character wearing the target three-dimensional garment in different poses.

[0128] In another embodiment, the object may refer to a person (e.g., a real person) in a video or image, and the mesh model may be a virtual garment (e.g., a three-dimensional model of clothing to be tried on). The computer device obtains vertex features of each vertex in the virtual garment that the person is trying on and topological information of the virtual garment, and obtains vertex relationship indication information. Then, a skin weight prediction model is used to predict the skin weight of the virtual garment based on the vertex relationship indication information, vertex features, and topological information of the virtual garment. Through the skin weights of different virtual garments, the computer device can synthesize a fitting effect view of the person trying on the virtual garment; through the skin weight of the target virtual garment, the computer device can render fitting effect views of the person wearing the target virtual garment in different postures.

[0129] Practice has proved that the image processing method provided in this application can constrain the prediction process of skin weights (such as constraining the similarity relationship of skin weights between vertices of different layers of clothing), improve the prediction accuracy of skin weights, and thus meet the requirements of complex mesh models (such as multi-layer structure clothing, combined clothing, etc.) for similarity and deformation coordination of skin weights.

[0130] Figure 10 is a comparison diagram of the effects provided by an embodiment of the present application. As shown in Figure 10, based on the skin weights of the mesh model obtained by the image processing method provided by the present application and the rotation and translation matrices of the object's joint points, the results obtained by performing rotation and translation processing on the mesh model can effectively solve the "mold penetration" problem that occurs after the mesh model is rotated and translated, compared to the results obtained by the comparison scheme, and improve the quality of the mesh model.

[0131] In an embodiment of the present application, vertex features of each vertex in a mesh model of an object and topological information of the mesh model are obtained, and vertex relationship indication information is obtained. The vertex relationship indication information is used to indicate vertices in the mesh model that have constraint relationships. A skin weight prediction model is used to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features, and the topological information of the mesh model. It can be seen that in the process of predicting the skin weight of the mesh model, constraining the skin weight of each vertex in the mesh model through the vertex relationship indication information can improve the prediction accuracy of the mesh model's skin matrix, thereby improving the quality of the mesh model obtained based on the mesh model's skin matrix. In addition, the adjacency sub-module performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex to obtain the neighborhood features of each vertex. The key vertex set can be added to the second adjacency matrix to aggregate the neighborhood vertex features in the mesh model (such as different layers of mesh) to achieve the purpose of constraining the skin weights between vertices; the residual sub-module performs deep feature extraction processing on the vertex features of each vertex, which can effectively extract the deep features (deeper features) of each vertex in the mesh model, further improving the prediction accuracy of the skin weight prediction model; the attention sub-module is called to aggregate the vertex features of each vertex in the mesh model into each key vertex set, which can aggregate the vertex features of the vertices in the mesh model into P key vertex sets according to adaptive weights, and at the same time, the vertex features of the key vertices in the P key vertex sets can be effectively transferred to the remaining vertices (non-key vertices) in the mesh model.

[0132] Please refer to Figure 11, which is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device shown in Figure 11 can be installed in a computer device, which can specifically be a terminal device or a server. The image processing device shown in Figure 11 can be used to perform some or all of the functions of the method embodiments described in Figures 2 and 5 above. Referring to Figure 11, the image processing device includes:

[0133] An acquisition unit 1101 is configured to acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model. The topological information of the mesh model includes positional relationships between joint points of the object and each vertex in the mesh model, as well as connection relationships between the vertices in the mesh model.

[0134] and for obtaining vertex relationship indication information, where the vertex relationship indication information is used to indicate vertices having a constraint relationship in the mesh model, where the constraint relationship refers to a constraint relationship between two different vertices;

[0135] Processing unit 1102 is configured to input the vertex relationship indication information, vertex features, and topology information of the mesh model into a skin weight prediction model, wherein the skin weight prediction model is configured to predict the skin weight of the mesh model based on the vertex relationship indication information, vertex features, and topology information of the mesh model;

[0136] The skin weight of the mesh model is used to indicate the degree of influence of the object's joint points on the deformation of each vertex in the mesh model during the deformation process of the mesh model.

[0137] In one embodiment, the skin weight prediction model includes a neural network layer and M feature fusion modules, where M is a positive integer; the processing unit 1102 is configured to predict the skin weight of the mesh model based on vertex relationship indication information, vertex features, and topological information of the mesh model, specifically for:

[0138] Calling M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model to obtain the fusion features of each vertex;

[0139] The fusion features of each vertex and the topological information of the mesh model are mapped through the neural network layer to obtain the skin weight of the mesh model.

[0140] In one embodiment, the vertex relationship indication information includes a set of P key vertices, where P is a positive integer; each feature fusion module includes an attention submodule, a residual submodule, and an adjacency submodule; the processing unit 1102 calls the M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model, and the process of obtaining the fused features of each vertex includes:

[0141] Obtain P first adjacency matrices corresponding to P key vertex sets, and perform feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule to obtain the neighborhood features of each vertex; the P first adjacency matrices are obtained after establishing the edges between each key vertex in each key vertex set;

[0142] Call the attention submodule to aggregate the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregate features; the P aggregate features correspond one-to-one to the P key vertex sets; the qth aggregate feature is any one of the P aggregate features, the qth aggregate feature corresponds to the qth key vertex set in the P key vertex sets, and the qth aggregate feature is used to indicate the skin weight of the vertex in the mesh model and the similarity with the skin weight of each key vertex in the qth key vertex set, where q is a positive integer less than or equal to P;

[0143] Perform deep feature extraction on the vertex features of each vertex through the residual submodule to obtain the deep features of each vertex;

[0144] The neighborhood features of each vertex, P aggregate features and the depth features of each vertex are spliced ​​to obtain the fusion features of each vertex in the mesh model.

[0145] In one embodiment, the processing unit 1102 performs feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule to obtain the neighborhood features of each vertex, including:

[0146] Determine the k neighbor vertices of key vertex j through the qth first adjacency matrix, where the qth first adjacency matrix is ​​the first adjacency matrix of the qth key vertex set, key vertex j belongs to the qth key vertex set, and k is a positive integer;

[0147] Calculate the difference between the vertex feature of the key vertex j and the vertex feature of each of the k neighbor vertices to obtain k difference features;

[0148] The maximum value among the k difference features is determined as the neighborhood feature of key vertex j.

[0149] In one embodiment, the topological information of the mesh model includes a second adjacency matrix, which is obtained based on the connection relationship between each vertex in the mesh model; the processing unit 1102 is configured to determine k neighbor vertices of the key vertex j using the qth first adjacency matrix, specifically for:

[0150] Through the qth first adjacency matrix, determine the T key vertices that have edges connected to the key vertex j, where T is a positive integer;

[0151] If T is greater than or equal to k, then k vertices are randomly selected from the T key vertices as the k neighbor vertices of the key vertex j;

[0152] If T is less than k, the second adjacency matrix is ​​used to determine the V vertices that have edges connected to the key vertex j, and kT vertices are randomly selected from the V vertices. The T key vertices and the selected kT vertices are used as the k neighbor vertices of the key vertex j, where V is a positive integer.

[0153] In one embodiment, the processing unit 1102 is configured to call the attention submodule to aggregate the vertex features of each vertex in the mesh model into each key vertex set. The process of obtaining P aggregated features includes:

[0154] Get the qth weight corresponding to each vertex in the mesh model, where q is a positive integer less than or equal to P;

[0155] The vertex features of each vertex are weighted by the qth weight value corresponding to the vertex to obtain the weighted features of each vertex;

[0156] Aggregate the weighted features of each vertex to obtain the qth aggregate feature.

[0157] In one embodiment, the process of the processing unit 1102 obtaining the qth weight corresponding to each vertex in the mesh model includes:

[0158] Performing mean processing on the first distances between the key vertices in the qth key vertex set and the vertex h to obtain the qth distance mean corresponding to the vertex h; the vertex h is any vertex in the mesh model;

[0159] Based on the derivative result of the qth distance mean corresponding to vertex h, determine the qth weight of vertex h.

[0160] In one embodiment, the residual submodule includes a neural network layer and an activation layer; the processing unit 1102 performs deep feature extraction processing on the vertex features of each vertex through the residual submodule, and the process of obtaining the deep features of each vertex includes:

[0161] Mapping the vertex features of vertex h through the neural network layer to obtain the feature mapping result of vertex h;

[0162] The activation layer is used to activate the feature mapping result of vertex h to obtain the activation feature of vertex h;

[0163] The activation feature of vertex h and the vertex feature of vertex h are superimposed to obtain the depth feature of vertex h.

[0164] In one embodiment, the joint points of the object include at least one basic joint point and P additional joint points corresponding to the mesh model, where P is a positive integer; the processing unit 1102 is configured to obtain vertex relationship indication information, specifically to:

[0165] Obtain P key regions in the mesh model corresponding to the P additional joint points, each key region including at least two vertices;

[0166] Screen the vertices in each key area to obtain P key vertex sets, and there are constraint relationships between the key vertices in each key vertex set;

[0167] Based on P key vertex sets, vertex relationship indication information is generated.

[0168] In one embodiment, the process of the processing unit 1102 screening the vertices in each key area to obtain P key vertex sets includes:

[0169] Add the target vertex in the qth key region and the neighboring vertices of the target vertex to the candidate vertex set, and the second distance between the target vertex and the qth additional joint point is less than the second distance between other vertices in the qth key region except the target vertex and the qth additional joint point; the qth key region is any one of the P key regions, and the qth key region corresponds to the qth additional joint point among the P additional joint points, where q is a positive integer less than or equal to P;

[0170] Get the label weight of each candidate vertex in the candidate vertex set, as well as the label weight of the neighboring vertices of each candidate vertex;

[0171] Perform mean processing on the differences between the label weight of each candidate vertex in the candidate vertex set and the label weights of the neighboring vertices of the candidate vertex to obtain the mean difference corresponding to each candidate vertex;

[0172] The candidate vertex with the largest mean difference is determined as the representative vertex of the candidate vertex set;

[0173] The first vertex in the qth key region is added to the qth key vertex set, and the difference between the difference mean of the first vertex and the difference mean of the representative vertex is less than the mean threshold.

[0174] In one embodiment, the processing unit 1102 is configured to obtain vertex features of each vertex in the mesh model of the object, specifically to:

[0175] Get the position information and normal vector of each vertex in the mesh model;

[0176] Calculate the second distance between each vertex and each joint point of the object according to the position information of each vertex;

[0177] The position information of each vertex, the normal vector, and the second distance between the vertex and each joint point of the object are combined to obtain the vertex feature of the vertex.

[0178] In one embodiment, the number of joint points of the object is N, where N is a positive integer; the processing unit 1102 is configured to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features, and the topology information of the mesh model, specifically to:

[0179] Based on vertex relationship indication information, vertex features, and topological information of the mesh model, N prediction weights are predicted for each vertex in the mesh model. The N joint points of the object correspond one-to-one to the N prediction weights. The u-th prediction weight of vertex h is used to indicate the degree of influence of joint point u on the deformation of vertex h during the deformation of the mesh model. Vertex h is any vertex in the mesh model, and the u-th prediction weight corresponds to joint point u, where u is a positive integer less than or equal to N.

[0180] Filter out the largest R prediction weights from the N prediction weights of each vertex as the R key weights of each vertex, where R is an integer greater than 1 and less than or equal to N;

[0181] The R key weights of each vertex are normalized to obtain the skin weight of the mesh model.

[0182] In one embodiment, the processing unit 1102 is further configured to:

[0183] Get the rotation and translation matrix of the object's joint points;

[0184] Based on the skin weight and rotation and translation matrix of the mesh model, each vertex in the mesh model is rotated and translated to obtain the rotation and translation result of the mesh model;

[0185] The grid model includes any of the following: a multi-layer grid model, a single-layer grid model, and a combined grid model.

[0186] According to one embodiment of the present application, some of the steps involved in the image processing method shown in Figures 2 and 5 can be performed by the various units in the image processing device shown in Figure 11. For example, steps S201 and S202 shown in Figure 2 can be performed by the acquisition unit 1101 shown in Figure 11, and step S203 can be performed by the processing unit 1102 shown in Figure 11; steps S501-S503 shown in Figure 5 can be performed by the acquisition unit 1101 shown in Figure 11, and steps S504-S508 can be performed by the processing unit 1102 shown in Figure 11. The various units in the image processing device shown in Figure 11 can be individually or completely merged into one or several other units to form a composition, or some of the units can be further divided into multiple smaller functional units to form a composition, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the image processing device may also include other units. In actual applications, these functions may also be implemented with the assistance of other units, and may be implemented by collaboration of multiple units.

[0187] According to another embodiment of the present application, an image processing device as shown in FIG11 can be constructed, and the image processing method of the embodiment of the present application can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding methods shown in FIG2 and FIG5 on a general-purpose computing device such as a computer device, which includes processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device via the computer-readable recording medium and executed therein.

[0188] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the image processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0189] Please refer to Figure 12, which is a structural diagram of a computer device provided in an embodiment of the present application. The computer device can be a terminal device or a server. As shown in Figure 12, the computer device includes at least a processor 1201, a communication interface 1202 and a memory 1203. Among them, the processor 1201, the communication interface 1202 and the memory 1203 can be connected via a bus or other means. Among them, the processor 1201 (or central processing unit (CPU)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions issued by the object to the computer device and control the computer device to perform power on and off operations; for example, the CPU can transmit various types of interactive data between the internal structures of the computer device, and so on. The communication interface 1202 can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 1201; the communication interface 1202 can also be used for the transmission and interaction of data within the computer device. Memory 1203 (Memory) is a memory device within the computer device, used to store programs and data. It is understood that memory 1203 herein may include both the computer device's built-in memory and, of course, the computer device's supported extended memory. Memory 1203 provides storage space for the computer device's operating system, which may include, but is not limited to, Android, iOS, Windows Phone, and the like, though this application does not limit this.

[0190] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the processing system of the computer device. In addition, a computer program suitable for being loaded and executed by the processor 1201 is also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0191] In one embodiment, the processor 1201 executes the image processing method of the above method embodiment by running the computer program in the memory 1203 .

[0192] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the computer device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0193] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the image processing method of the above method embodiment.

[0194] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the above-described image processing method. The computer-readable storage medium may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

Claims

1. An image processing method, characterized in that: The method is performed by a computer device, and the method comprises: Acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model, wherein the topological information of the mesh model includes positional relationships between joint points of the object and each vertex in the mesh model, and connection relationships between the vertices in the mesh model; Acquire vertex relationship indication information, where the vertex relationship indication information is used to indicate vertices in the mesh model that have a constraint relationship, where the constraint relationship refers to a constraint relationship between two different vertices; Inputting the vertex relationship indication information, the vertex features and the topological information of the mesh model into a skin weight prediction model, wherein the skin weight prediction model is used to: predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features and the topological information of the mesh model; The skin weight of the mesh model is used to indicate the degree of influence of the joint points of the object on the deformation of each vertex in the mesh model during the deformation of the mesh model.

2. The method according to claim 1, characterized in that The skin weight prediction model includes a neural network layer and M feature fusion modules, where M is a positive integer; the skin weight of the mesh model is predicted based on the vertex relationship indication information, the vertex features and the topological information of the mesh model, including: Calling the M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex feature of each vertex in the mesh model to obtain fusion features of each vertex; The fusion features of each vertex and the topological information of the mesh model are mapped through the neural network layer to obtain the skin weight of the mesh model.

3. The method according to any one of claims 1 to 2, characterized in that: The vertex relationship indication information includes P key vertex sets, where P is a positive integer; each feature fusion module includes an attention submodule, a residual submodule and an adjacency submodule; calling the M feature fusion modules to perform feature fusion processing on the vertex relationship indication information and the vertex features of each vertex in the mesh model to obtain fusion features of each vertex, including: Acquire P first adjacency matrices corresponding to the P key vertex sets, and perform feature conversion processing on the P first adjacency matrices and vertex features of each vertex through the adjacency submodule to obtain neighborhood features of each vertex; the P first adjacency matrices are obtained after establishing edges between each key vertex in each key vertex set; Calling the attention submodule to aggregate the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregate features; the P aggregate features correspond to the P key vertex sets one by one; the qth aggregate feature is any one of the P aggregate features, the qth aggregate feature corresponds to the qth key vertex set in the P key vertex sets, the qth aggregate feature is used to indicate the similarity between the skin weight of the vertex in the mesh model and the skin weight of each key vertex in the qth key vertex set, and q is a positive integer less than or equal to P; Performing deep feature extraction processing on vertex features of each vertex through the residual submodule to obtain deep features of each vertex; The neighborhood features of each vertex, the P aggregation features and the depth features of each vertex are spliced ​​to obtain the fusion features of each vertex in the grid model.

4. The method according to any one of claims 1 to 3, characterized in that: The step of performing feature conversion processing on the P first adjacency matrices and the vertex features of each vertex through the adjacency submodule to obtain the neighborhood features of each vertex includes: Determine k neighbor vertices of key vertex j through the qth first adjacency matrix, wherein the qth first adjacency matrix is ​​the first adjacency matrix of the qth key vertex set, the key vertex j belongs to the qth key vertex set, and k is a positive integer; Calculate the difference between the vertex feature of the key vertex j and the vertex feature of each neighbor vertex in the k neighbor vertices respectively to obtain k difference features; The maximum value among the k difference features is determined as the neighborhood feature of the key vertex j.

5. The method according to any one of claims 1 to 4, characterized in that: The topological information of the mesh model includes a second adjacency matrix, where the second adjacency matrix is ​​obtained based on the connection relationship between each vertex in the mesh model; The step of determining k neighbor vertices of key vertex j through the qth first adjacency matrix includes: Determine T key vertices having edges connected to key vertex j through the qth first adjacency matrix, where T is a positive integer; If T is greater than or equal to k, k vertices are randomly selected from the T key vertices as k neighbor vertices of the key vertex j; If T is less than k, the second adjacency matrix is ​​used to determine V vertices that are connected to the key vertex j, and kT vertices are randomly selected from the V vertices. The T key vertices and the selected kT vertices are used as k neighbor vertices of the key vertex j, where V is a positive integer.

6. The method according to any one of claims 1 to 5, characterized in that: The calling of the attention submodule aggregates the vertex features of each vertex in the mesh model into each key vertex set to obtain P aggregate features, including: Obtaining the qth weight corresponding to each vertex in the mesh model; The vertex features of each vertex are weighted by the qth weight corresponding to the vertex to obtain the weighted features of each vertex; The weighted features of each vertex are aggregated to obtain the qth aggregate feature.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining of the qth weight corresponding to each vertex in the mesh model comprises: Performing mean processing on the first distance between the key vertices in the qth key vertex set and the vertex h to obtain the qth distance mean corresponding to the vertex h; the vertex h is any vertex in the mesh model; Based on the derivative result of the qth distance mean corresponding to the vertex h, the qth weight of the vertex h is determined.

8. The method according to any one of claims 1 to 7, characterized in that: The residual submodule includes a neural network layer and an activation layer; the residual submodule performs deep feature extraction processing on the vertex features of each vertex to obtain the deep features of each vertex, including: Mapping the vertex features of vertex h through the neural network layer to obtain a feature mapping result of the vertex h; Using the activation layer to perform activation processing on the feature mapping result of the vertex h to obtain the activation feature of the vertex h; The activation feature of the vertex h and the vertex feature of the vertex h are superimposed to obtain the depth feature of the vertex h.

9. The method according to any one of claims 1 to 8, characterized in that The joint points of the object include at least one basic joint point and P additional joint points corresponding to the mesh model, where P is a positive integer; and obtaining vertex relationship indication information includes: Acquire P key areas in the mesh model corresponding to the P additional joint points, each key area including at least two vertices; The vertices in each key area are screened to obtain P key vertex sets, and there are constraint relationships between the key vertices in each key vertex set; The vertex relationship indication information is generated based on the P key vertex sets.

10. The method according to any one of claims 1 to 9, characterized in that: The vertices in each key area are screened to obtain P key vertex sets, including: The target vertex in the qth key area and the neighboring vertices of the target vertex are added to the candidate vertex set; the target vertex The second distance between the point and the qth additional joint point is less than the second distance between other vertices in the qth key area except the target vertex and the qth additional joint point; the qth key area is any one of the P key areas, the qth key area corresponds to the qth additional joint point among the P additional joint points, and q is a positive integer less than or equal to P; Obtaining the label weight of each candidate vertex in the candidate vertex set and the label weight of the neighboring vertices of each candidate vertex; Performing mean processing on the difference between the label weight of each candidate vertex in the candidate vertex set and the label weight of the neighboring vertices of the candidate vertex to obtain the mean of the difference corresponding to each candidate vertex; Determine the candidate vertex with the largest difference mean as the representative vertex of the candidate vertex set; A first vertex in the qth key area is added to the qth key vertex set, wherein the difference between the difference mean of the first vertex and the difference mean of the representative vertex is less than a mean threshold.

11. The method according to any one of claims 1 to 10, characterized in that: The step of acquiring vertex features of each vertex in the mesh model of the object includes: Get the position information and normal vector of each vertex in the mesh model; Calculate the second distance between each vertex and each joint point of the object according to the position information of each vertex; The position information of each vertex, the normal vector, and the second distance between the vertex and each joint point of the object are combined to obtain the vertex feature of the vertex.

12. The method according to any one of claims 1 to 11, characterized in that: The number of joint points of the object is N, where N is a positive integer; and predicting the skin weight of the mesh model based on the vertex relationship indication information, the vertex features and the topological information of the mesh model includes: Based on the vertex relationship indication information, the vertex features and the topological information of the mesh model, N prediction weights of each vertex in the mesh model are predicted, and the N joint points of the object correspond to the N prediction weights one by one; the u-th prediction weight of vertex h is used to indicate the degree of influence of joint point u on the deformation of vertex h during the deformation process of the mesh model; the vertex h is any vertex in the mesh model, the u-th prediction weight corresponds to the joint point u, and u is a positive integer less than or equal to N; Filter out the largest R prediction weights from the N prediction weights of each vertex as the R key weights of each vertex, where R is an integer greater than 1 and less than or equal to N; The R key weights of each vertex are normalized to obtain the skin weight of the mesh model.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: Obtain the rotation and translation matrix of the joint points of the object; Based on the skin weight of the mesh model and the rotation and translation matrix, rotation and translation processing is performed on each vertex in the mesh model to obtain a rotation and translation result of the mesh model; The grid model includes any one of the following: a multi-layer grid model, a single-layer grid model, and a combined grid model.

14. An image processing device, characterized in that: The image processing device comprises: An acquisition unit, used to acquire vertex features of each vertex in a mesh model of an object and topological information of the mesh model, wherein the topological information of the mesh model includes positional relationships between joint points of the object and each vertex in the mesh model, and connection relationships between the vertices in the mesh model; and for obtaining vertex relationship indication information, wherein the vertex relationship indication information is used to indicate vertices having a constraint relationship in the mesh model, wherein the constraint relationship refers to a constraint relationship between two different vertices; A processing unit, configured to input the vertex relationship indication information, the vertex features and the topological information of the mesh model into a skin weight prediction unit. The skin weight prediction model is used to predict the skin weight of the mesh model based on the vertex relationship indication information, the vertex features and the topological information of the mesh model; The skin weight of the mesh model is used to indicate the degree of influence of the joint points of the object on each vertex in the mesh model during the deformation process of the mesh model.

15. A computer device, characterized in that: include: a memory, wherein a computer program is stored in the memory; A processor, used to load the computer program to implement the image processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 13.

17. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 13 is implemented.

Images