Model processing method, apparatus and device, and storage medium and computer program product

Abstract

Disclosed in the present application are a model processing method, apparatus and device, and a storage medium and a computer program product. The method comprises: in response to an acquired expression movement instruction, acquiring a source head model; according to a model feature of a target head model to be subjected to expression transfer, performing deformation matching processing on the source head model, and according to the source head model subjected to deformation matching, and a deformation relationship between a first model expression and a second model expression in the source head model, determining a deformation parameter, wherein the first model expression in the source head model is determined according to neutral feature data of the source head model, and the second model expression is determined according to expression feature data indicated by the expression movement instruction; and performing expression transfer processing on the target head model according to the deformation parameter, so as to obtain a target head model having the second model expression. The second model expression can be automatically transferred between models, that is, the second model expression is automatically generated for the target head model.

Description

Model processing method, device, equipment, storage medium, and computer program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 30, 2023, with application number 202310803347.9 and application name “Model processing method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a model processing method, apparatus, device, storage medium, and computer program product. Background Art

[0003] 3D (3-dimensional) animation is currently widely used in various industries, such as CG (Computer Graphics) movies, 3D cartoons, virtual live broadcasts, virtual 3D operators, etc. Among them, the content production of 3D expression animation is an important bottleneck in the development of the industry; the existing technology usually generates blend shapes (BlendShape, BS) by manually pinching the face, and further constructs model expressions based on the generated blend shapes; but different 3D head models usually have different facial features, such as different facial contours, different facial feature shapes, and different facial feature distributions. Therefore, when it is necessary to construct model expressions for different 3D head models, since they need to be generated based on manual face pinching, the degree of automation is low and the human resources consumption is too high.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a model processing method, apparatus, device, storage medium, and computer program product, which can automatically realize the migration of second model expressions between models and automatically generate second model expressions for the target head model.

[0006] In one aspect, an embodiment of the present application provides a model processing method, comprising:

[0007] In response to the acquired expression action instruction, a source head model is acquired; the source head model is configured with neutral feature data and a plurality of expression feature data relative to the neutral feature data;

[0008] According to the model features of the target head model, the source head model is deformed and matched to obtain the deformed and matched source head model;

[0009] Determining deformation parameters based on the deformed and matched source head model and a deformation relationship from a first model expression to a second model expression in the source head model; wherein the first model expression in the source head model is determined based on neutral feature data of the source head model, and the second model expression in the source head model is determined based on expression feature data indicated by an expression action instruction in the plurality of expression feature data;

[0010] On the target head model, expression transfer processing is performed according to the deformation parameters to obtain a target head model with the expression of the second model.

[0011] In one aspect, an embodiment of the present application provides a model processing device, comprising:

[0012] An acquisition unit, configured to acquire a source head model in response to the acquired expression action instruction; the source head model is configured with neutral feature data and a plurality of expression feature data relative to the neutral feature data;

[0013] a processing unit, configured to perform deformation matching processing on the source head model according to the model features of the target head model, to obtain a deformed and matched source head model;

[0014] The processing unit is further configured to determine deformation parameters based on the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model; wherein the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction in the plurality of expression feature data;

[0015] The processing unit is further used to perform expression migration processing on the target head model according to the deformation parameters to obtain the target head model with the expression of the second model.

[0016] In one aspect, an embodiment of the present application provides a model processing device, the model processing device including an input interface and an output interface, and further including:

[0017] a processor adapted to implement one or more instructions; and

[0018] Computer storage medium, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned model processing method.

[0019] On the one hand, an embodiment of the present application provides a computer storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, they are used to execute the above-mentioned model processing method.

[0020] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer storage medium; the processor of the model processing device reads the computer program from the computer storage medium, and the processor executes the computer program, so that the model processing device performs the above-mentioned model processing method.

[0021] In an embodiment of the present application, after obtaining an expression action instruction, the source head model can be deformed and matched to the target head model, so that the deformed and matched source head model approaches the target head model; then, based on the deformed and matched source head model and the deformation relationship from the first model expression in the source head model to the second model expression indicated by the expression action instruction, deformation parameters can be determined, and expression migration processing can be performed on the target head model based on the deformation parameters to obtain a target head model with the second model expression. The migration of the second model expression from the source head model to the target head model can be automatically achieved. In this way, expression transformation on some target head models (such as a newly designed head model) can be achieved based on the expression transformation of some source head models (such as a standard model). The second model expression can be automatically generated for the target head model, realizing the reuse of the second model expression in the source head model in different head models, further improving the efficiency of generating model expressions for different head models and reducing human resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a schematic diagram of a mixed shape provided in an embodiment of the present application;

[0023] FIG2 is a flow chart of a model processing method provided in an embodiment of the present application;

[0024] FIG3 is a flow chart of another model processing method provided in an embodiment of the present application;

[0025] FIG4 is a schematic diagram of reference facial key points in a two-dimensional facial image provided by an embodiment of the present application;

[0026] FIG5 is a schematic diagram of position coordinates of a source head model and a target triangle in the source head model after deformation matching provided by an embodiment of the present application;

[0027] FIG6 is a schematic diagram of performing expression migration to a target head model according to an embodiment of the present application;

[0028] FIG7 is a flow chart of another model processing method provided in an embodiment of the present application;

[0029] FIG8 is a schematic diagram of a characteristic region provided in an embodiment of the present application;

[0030] FIG9 is a schematic diagram of eyeball area positioning and rotation provided by an embodiment of the present application;

[0031] FIG10 is a schematic diagram of performing expression migration processing on a target head model according to an embodiment of the present application;

[0032] FIG11 is a schematic diagram of determining an eyelid area according to an embodiment of the present application;

[0033] FIG12 is a schematic diagram of a lower eyelid boundary and an upper eyelid boundary provided in an embodiment of the present application;

[0034] FIG13 is a schematic diagram of adjusting the eyelid area according to an embodiment of the present application;

[0035] FIG14 is a schematic diagram of a texture coordinate relationship provided in an embodiment of the present application;

[0036] FIG15 is a schematic diagram of an expression migration to an interactive model provided by an embodiment of the present application;

[0037] FIG16 is a schematic diagram of an effect of expression migration provided by an embodiment of the present application;

[0038] FIG17 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0039] FIG18 is a schematic structural diagram of a model processing device provided in an embodiment of the present application;

[0040] FIG19 is a schematic structural diagram of a model processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] Computer vision is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in object recognition, measurement, and other machine vision tasks. Further image processing is performed to transform the images into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Large model technology has revolutionized the development of computer vision. Pre-trained models in the field of vision, such as the Swin Transformer, ViT, V-MOE, and MAE, can be fine-tuned to quickly and widely adapt to specific downstream tasks. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional (3D) technology, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping, and more.

[0043] Based on the computer vision technology mentioned above, an embodiment of the present application provides a model processing solution that can obtain a source head model in response to an acquired expression action instruction, and perform deformation matching processing on the source head model based on the model features of a target head model, which is the model to be subjected to head migration processing, to obtain a deformed and matched source head model. The model features mainly refer to the physical features of objects such as eyes, ears, mouth, and nose contained in the head, such as the shape and size features of objects such as eyes, ears, mouth, and nose. Then, deformation parameters can be determined based on the deformed and matched source head model and the deformation relationship from the first model expression in the source head model to the second model expression. Expression migration processing is performed on the target head model based on the deformation parameters to obtain a target head model with the second model expression. The source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data. The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction among the multiple expression feature data.

[0044] The above-mentioned model processing scheme can be executed by a model processing device, which can be a terminal device or a server; the terminal device here can include but is not limited to: computers, smart phones, tablet computers, laptops, smart home appliances, vehicle terminals, smart wearable devices, etc.; the server here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Further optionally, the above-mentioned model processing scheme can also be executed individually or collaboratively by any electronic device with computing power. The embodiments of this application are not limited thereto, and the model processing device will be used as an example for explanation in the subsequent embodiments of this application.

[0045] The various models mentioned in the above model processing scheme can be three-dimensional models, and the three-dimensional model can be a set of multiple triangles, where the vertices of the triangles are the vertices of the three-dimensional model. Among them, the source head model can be a head model configured with neutral feature data and multiple expression feature data relative to the neutral feature data. Here, the neutral feature data and the expression feature data refer to a blend shape (BS). Any expression feature data can be obtained by deforming the neutral feature data. Based on the neutral feature data, or one or more of the multiple expression feature data, various model expressions can be obtained by combining. Any model expression can also be considered a blend shape, that is, a combination of multiple blend shapes is also considered a blend shape. The model expression determined only based on the neutral feature data can be called a neutral model expression. The first model expression in the source head model mentioned in the above model processing scheme is the neutral model expression determined based on the neutral feature data of the source head model. For example, an expression feature data indicating "open mouth" can be combined with an expression feature data indicating "closed eyes" to obtain a model expression indicating "open mouth and closed eyes". Refer to Figure 1, which is a schematic diagram of a blending shape provided in an embodiment of the present application; wherein, the blending shape indicated by mark 101 is a "neutral" blending shape, which corresponds to neutral feature data, and the corresponding model expression is a neutral model expression; the blending shape indicated by mark 102 is an "open mouth" blending shape, which corresponds to expression feature data indicating "open mouth", and the corresponding model expression is a model expression indicating "open mouth"; the blending shape indicated by mark 103 is a "bite mouth" blending shape, which corresponds to expression feature data indicating "bite mouth", and the corresponding model expression is a model expression indicating "bite mouth"; the blending shape indicated by mark 104 is a "pursed mouth" blending shape, which corresponds to expression feature data indicating "pursed mouth", and the corresponding model expression is a model expression indicating "pursed mouth".

[0046] The collection and processing of relevant data (such as source head models, target head models, etc.) in this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the subject of personal information should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the subject of personal information. In addition, when the technologies related to face (or other biometric features) are applied to specific products or technologies in the relevant embodiments of this application, the relevant data collection, use and processing processes should comply with the requirements of laws and regulations. Before collecting facial information, the information processing rules should be notified and the separate consent of the corresponding subjects should be obtained. Facial information should be processed in strict accordance with the requirements of laws and regulations and the rules for processing personal information, and technical measures should be taken to ensure the security of relevant data.

[0047] Based on the above model processing scheme, an embodiment of the present application provides a model processing method. Referring to FIG. 2 , a flow chart of a model processing method provided in an embodiment of the present application is shown. The model processing method shown in FIG. 2 can be executed by the model processing device mentioned above. The model processing method shown in FIG. 2 may include the following steps:

[0048] S201, in response to the acquired facial expression action instruction, acquiring a source head model.

[0049] Among them, the source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data. Furthermore, the expression action instruction can be used to guide the model processing device to generate the desired model expression for the source head model. The instruction format of the expression action instruction can be set according to specific needs, for example, it can be a text format, a voice format, etc., which is not limited in the embodiments of the present application; the model processing device can generate the model expression expected by the expression action instruction for the source head model by performing instruction analysis and processing on the expression action instruction; in a feasible embodiment, if the expression action instruction is in text format, the model processing device can generate the model expression indicated by the corresponding expression keyword for the source head model when it is detected that the expression action instruction includes an expression keyword. For example, if it is detected that the expression action instruction includes the expression keyword "open mouth", a model expression indicating "open mouth" can be generated for the source head model; in another feasible embodiment, if the expression action instruction is in text format, the model processing device can perform semantic analysis and processing on the expression action instruction and generate a semantically matching model expression for the source head model. For example, when it is analyzed that the expression action instruction includes the semantics of "startle", a model expression indicating "open mouth" that matches the semantics can be generated for the source head model. Neutral feature data may be data on the positions of vertices of each face mesh in a neutral state. Based on the neutral vertex positions of the face mesh, a neutral face may be rendered. The neutral face may be, for example, an expressionless face. Similarly, expression feature data may be data on the positions of vertices of each face mesh under a specific expression. Based on the expression feature data, a face under a specific expression may be rendered. For example, the expression feature data of a smile may be data on the positions of vertices of each face mesh under the corresponding smile expression, and a smiling face may be rendered based on the vertex positions of each face mesh under the corresponding smile expression. Of course, in addition to the positions of the corresponding face mesh vertices, the neutral feature data and the expression feature data may also include tabular data, in which a face composed of multiple vertices is stored. For example, the tabular data may store data on the vertices that constitute a triangular face.

[0050] S202: Perform a deformation matching process on the source head model based on the model features of the target head model to obtain a deformed and matched source head model. The target head model is the model for which expression transfer is to be performed. The source head model and the target head model have different model features. Model features may be, for example, facial features of the head model, including facial features, face shape, chin, cheekbones, etc. In other words, model features are used to distinguish different head models.

[0051] In one embodiment, the target head model may be a head model different from the source head model. For example, the target head model may have a different face shape, different facial features, or different facial features distribution from the source head model. The model processing device performs deformation matching processing on the source head model based on the model features of the target head model in order to deform the source head model to match the target head model, that is, to adjust the source head model to make it closer to the target head model. For example, one or more features of the source head model, such as face shape, facial features, or facial features distribution, may be made closer to corresponding features of the target head model.

[0052] S203 , determining deformation parameters according to the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model.

[0053] The first model expression in the source head model is determined based on the neutral feature data of the source head model, that is, the first model expression in the source head model is a neutral model expression; the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction in the multiple expression feature data. In other words, the second model expression in the source head model is the model expression generated by the model processing device based on the expression action instruction. In the process of the model processing device performing instruction analysis and processing on the expression action instruction to generate the model expression expected by the expression action instruction for the source head model, the expression action instruction can be analyzed and processed to determine the expression feature data indicated by the expression action instruction, and then the expected model expression (i.e., the second model expression) can be generated based on the determined expression feature data. The second model expression is equivalent to the expression determined from the existing multiple expression feature data according to the expression action instruction. In a feasible embodiment, if the expression action instruction is in text format, the model processing device can determine the expression feature data indicated by the expression action instruction based on the expression keyword when detecting that the expression action instruction includes an expression keyword, and further generate the corresponding model expression based on the determined expression feature data. It can be known that the way in which the above-mentioned model processing device generates the model expression expected by the expression action instruction for the source head model by performing instruction analysis and processing on the expression action instruction is only an exemplary way. For example, when the expression action instruction includes identification information of the expression feature data, the model processing device can directly determine the expression feature data indicated by the expression action instruction through the identification information included in the expression action instruction, and then generate the corresponding model expression.

[0054] The deformation relationship between the source head model and the deformed and matched source head model includes the transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the deformed and matched source head model. The transformation matrix between the vertices is determined based on the affine transformation parameters and the translation parameters. The calculation of this deformation relationship can be found in the relevant description of the subsequent embodiments.

[0055] S204 , performing expression migration processing on the target head model according to the deformation parameters to obtain a target head model having a second model expression.

[0056] In one embodiment, performing expression transfer processing based on the deformation parameters may include: adjusting the positions of feature points in a target head model having a first model expression based on the deformation parameters to obtain a target head model that implements a second model expression. When the model processing device performs expression transfer processing on the target head model based on the deformation parameters, the expression transfer processing is performed on the target head model having the first model expression based on the deformation parameters, that is, the expression transfer processing is performed on the target head model having a neutral model expression based on the deformation parameters. The first model expression in the target head model is adapted to the target head model, that is, it is determined based on the neutral feature data of the target head model, not based on the neutral feature data of the source head model. When performing expression transfer processing on the target head model based on the deformation parameters, the model processing device may adjust the positions of vertices in the target head model having the first model expression based on the deformation parameters to implement the second model expression based on the target head model. For example, if the second model expression in the source head model is a model expression indicating "open mouth", the positions of vertices in the target head model having the first model expression may be adjusted based on the deformation parameters to implement the model expression indicating "open mouth" based on the target head model.

[0057] In an embodiment of the present application, after obtaining an expression action instruction, starting from a source head model having neutral feature data and expression feature data without expression, the expression change of the target head model can be achieved by migrating the expression on the source head model. To achieve the purpose of expression migration, the source head model is first deformed and matched to the target head model, so that the deformed and matched source head model approaches the target head model, so that the source head model having neutral feature data and expression feature data and the target head model are similar in shape; then, based on the deformed and matched source head model and the deformation relationship from the first model expression in the source head model to the second model expression indicated by the expression action instruction, a deformation parameter is determined. Based on the deformation parameter, the target head model with the current expression can be transformed into a target head model with the second model expression. The migration of the second model expression from the source head model to the target head model can be automatically achieved, that is, the second model expression can be automatically generated for the target head model, and the second model expression in the source head model can be reused in different head models, which can further improve the efficiency of generating model expressions for different head models and reduce human resource consumption.

[0058] Based on the relevant embodiments of the above-mentioned model processing method, the embodiment of the present application provides another model processing method. Referring to Figure 3, there is shown a flow chart of another model processing method provided in the embodiment of the present application. The model processing method shown in Figure 3 can be executed by the model processing device mentioned above. The model processing method shown in Figure 3 may include the following steps:

[0059] S301, in response to the acquired facial expression action instruction, acquiring a source head model.

[0060] Among them, the source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data; the relevant process of step S301 is similar to the relevant process of the above-mentioned step S201, and will not be repeated here.

[0061] S302 : Perform deformation matching processing on the source head model according to the model features of the target head model to obtain a deformed and matched source head model.

[0062] In one embodiment, the model processing device performs deformation matching processing on the source head model based on the model features of the target head model in order to deform the source head model to match it to the target head model, that is, to adjust the source head model to make it closer to the target head model. Adjusting the source head model to make it closer to the target head model can be understood as: using the spatial positions of each key point of the same name on the source head model and the target head model to roughly align the source head model through rigid translation, rotation and scaling, so that the initial forms of the two models are similar, that is, the two models are roughly close, for example, at least in general, the face of the source head model and the face of the target head model are similar in size and shape, such as both are round faces and the facial boundary positions are close. Then, the planar distance of the model vertices, the Euclidean distance of the key points of the same name in the two models, and the degree of mesh deformation are used as metrics to adjust the facial details in the source head model. By adjusting the key points of the source head model, the coordinate positions of the key points of the adjusted source head model on the adjusted source head model and the differences between the coordinate positions of the corresponding key points (key points of the same name) in the target head model on the target head model (i.e., the Euclidean distance) meet the similarity conditions (for example, the Euclidean distance is less than the threshold, which is considered to meet the similarity conditions). At the same time, it is ensured that the vertex mesh of the source head model is deformed as little as possible, and the planar distance between the vertices of the deformed source head model and the target head model is as small as possible, so as to obtain the optimal solution under the three constraints. When the optimal solution is reached, the adjusted source head model is considered to be close to the target head model.

[0063] In one embodiment, based on the positions of the vertices and key points on the source head model and the vertices and key points on the target head model, a candidate source head model after the position of each vertex and key point is modified can be calculated while the positions of the vertices and key points on the target head model remain unchanged. According to the planar distance of the vertices of the candidate source head model, the Euclidean distance of the key points with the same name in the candidate source head model and the target head model, and the degree of mesh deformation of the candidate source head model, calculations are performed according to the preset corresponding weight values. The candidate source head model with the minimum value after the weight calculation is obtained, that is, it is determined as the target source head model. The target source head model is a source head model that is close to the target head model. The source head model can be transformed according to the positions of each vertex and key point in the target source head model to obtain a deformed and matched source head model, thereby completing the deformation matching of the source head model. That is, the position values ​​of the vertices and key points in the source head model are modified, and the modified values ​​are included in the weight calculation to obtain the position values ​​of each vertex and key point at the minimum value, which are the values ​​of each vertex and key point of the target source head model. The source head model is adjusted according to the values ​​of each vertex and key point of the target source head model to complete the deformation matching of the source head model, and a source head model close to the target head model is obtained. Among them, the plane distance of the vertex of the candidate source head model refers to the distance from the vertex on the candidate source head model to the corresponding plane of the target head model. The plane distance corresponding to the vertex on the source head model is calculated because the vertex in the source head model and the nearest vertex on the target head model are most likely not specific points on the target head model, but a point on a plane composed of points; the key points with the same name in the candidate source head model and the target head model can, for example, refer to the key points with the same number on the candidate source head model and the target head model, such as the key points with the same number as the corner of the eye key point, the key points with the same number as the tip of the nose key point, etc.; the degree of mesh deformation of the candidate head model can be measured based on the difference between the rotation part of the transformation matrix of the triangle face in the candidate source head model before and after deformation and the unit matrix.

[0064] In one embodiment, the facial key points may be points that can be used to indicate facial features, such as points indicating facial features such as the tip of the nose, eyeballs, facial contours, eyebrows, and eyelids; the model processing device may determine the facial key points in the target head model, which may include: performing face detection processing on the target head model to obtain detection key points corresponding to the target head model, using reference face key point distribution information to perform key point labeling processing on the detection key points corresponding to the target head model, and using the labeled detection key points as the facial key points of the target head model. Optionally, the face detection processing on the target head model may be performed by a face detection model based on CNN (Convolution Neural Network). Specifically, the face key points may be detected by the face detection model. It is known that other neural network models that can detect facial key points may also be used in this process, and the embodiments of the present application are not limited thereto. Further optionally, the reference facial key point distribution information can be set according to specific needs, which can be the distribution information of the reference facial key points in the two-dimensional facial image. The reference facial key points in the two-dimensional facial image can be set according to specific needs. For example, the number and distribution position of the reference facial key points in the two-dimensional facial image can be set. Generally speaking, the number of reference facial key points in the two-dimensional facial image can be set to 82, 76, etc. For example, see Figure 4, which is a schematic diagram of the reference facial key points in a two-dimensional facial image provided in an embodiment of the present application. It can include reference facial key points indicating facial features such as the tip of the nose, eyeballs, facial contours, eyebrows, and eyelids. Furthermore, the target head model for face detection processing can be obtained by rendering. If there is a color texture or point coordinate, color rendering is used with natural lighting.

[0065] In one embodiment, the model processing device performs deformation matching processing on the source head model according to the model features of the target head model to obtain the deformed and matched source head model, which may include: determining the matching relationship between each feature point in the source head model and each feature point in the target head model according to the position of each feature point in the source head model and the position of each feature point in the target head model; wherein the feature points may include: one or two of vertices and facial key points in the corresponding model; according to the matching relationship, taking the position of the feature point in the target head model as the reference position, adjusting the position of the vertices constituting the source head model to obtain the deformed and matched source head model; wherein the distance between the position of the feature point in the deformed and matched source head model and the corresponding reference position determined according to the matching relationship satisfies a proximity condition. That is to say, the model processing device can adjust the positions of the vertices constituting the source head model based on the matching relationship, with the positions of the feature points in the target head model as the reference positions and with reducing the feature point position distance as the deformation target, to obtain the source head model after deformation and matching; wherein, the feature point position distance refers to: the position of the feature point in the source head model and the distance between the reference positions corresponding to the feature points in the target head model that have a matching relationship; the positions of the vertices constituting the source head model are adjusted so that the distance between the positions of the feature points in the source head model after deformation and matching and the corresponding reference positions determined according to the matching relationship meet the proximity condition, and the proximity condition can be set according to specific needs.

[0066] Since the source head model and the target head model are two different models, the number of vertices and the positions of each vertex in the source head model may be different from those in the target head model. The vertices in the source head model and the vertices in the target head model may have one or more of a variety of correspondences, such as one-to-one, one-to-many, many-to-one, and many-to-many. For example, when the source head model has more vertices in the eyebrow area than the target head model, there may be a situation where multiple vertices in the eyebrow area of ​​the source head model correspond to one vertex in the eyebrow area of ​​the target head model. Therefore, when the feature point is a vertex, the data processing device can determine the matching relationship between each vertex in the source head model and each vertex in the target head model based on the position of each vertex in the source head model and the position of each vertex in the target head model. Determining the matching relationship between the vertices is to determine the corresponding relationship between the vertices. When the feature points are facial key points, the data processing device can determine the matching relationship between each facial key point in the source head model and each facial key point in the target head model based on the positions of each facial key point in the source head model and the positions of each facial key point in the target head model. Determining the matching relationship between facial key points is to determine the correspondence between facial key points. For example, it is determined that there is a matching relationship between the facial key point indicating the tip of the nose in the source head model and the facial key point indicating the tip of the nose in the target head model. The process of determining the facial key points in the source head model is similar to the process of determining the facial key points in the above-mentioned target head model, and will not be repeated here.

[0067] Furthermore, the model processing device can adjust the positions of the vertices constituting the source head model based on the matching relationship and the positions of the feature points in the target head model as reference positions to obtain a deformed and matched source head model; wherein the distances between the positions of the feature points in the deformed and matched source head model and the corresponding reference positions determined according to the matching relationship meet a proximity condition. For example, the proximity condition can be set as follows: the average value of the distances between the positions of the feature points in the deformed and matched source head model and the corresponding reference positions determined according to the matching relationship is less than a first threshold; for another example, the proximity condition can be set as follows: the sum of the distances between the positions of the feature points in the deformed and matched source head model and the corresponding reference positions determined according to the matching relationship is less than a second threshold, wherein the first threshold and the second threshold can be set according to specific needs.

[0068] Further optionally, when the feature points are selected from vertices and facial key points at the same time, the model processing device can first adjust the positions of the vertices constituting the source head model based on the matching relationship between the facial key points to obtain an intermediate source head model, and then further adjust the positions of the vertices in the intermediate source head model based on the matching relationship between the vertices to obtain the source head model after deformation and matching. Wherein, based on the matching relationship between the facial key points, when adjusting the positions of the vertices constituting the source head model, the distance between the facial key points can be used as a metric, that is, the position of the feature points (i.e., facial key points) in the target head model is used as a reference position, and the distance between the feature points when based on the facial key points is reduced as the deformation target. This process can be achieved by rigidly translating, rotating, and scaling the source head model. This process is expected to make the initial states of the intermediate source head model and the target head model consistent, such as the head orientation, size, etc.; based on the matching relationship between the vertices, when continuing to adjust the positions of the vertices in the intermediate source head model, the distance between the vertices can be used. The distance between them is used as a metric, that is, the position of the feature points (i.e., vertices) in the target head model is used as a reference position, and the distance between the feature points when based on vertices is reduced as the deformation target. Further optionally, the distance between key points and the distance between vertices can be used together for measurement, that is, the position of the feature points (i.e., vertices and facial key points) in the target head model is used as a reference position, and the distance between the feature points when based on vertices and the distance between the feature points when based on facial key points are reduced as the deformation target. Further optionally, the distance between vertices can be the distance between vertices with a matching relationship (i.e., the feature point position distance when based on vertices), or the point-to-surface distance of vertices can be selected.

[0069] S303, determining the deformation relationship between the source head model and the deformed head model according to the position of each feature point in the deformed and matched source head model and the position of the corresponding feature point in the source head model; the feature point includes one or more of a vertex and a facial key point.

[0070] In one embodiment, the model processing device determines the deformation relationship between the source head model and the source head model after deformation matching based on the positions of each feature point in the source head model after deformation matching and the positions of the corresponding feature points in the source head model. This is to determine the vertex deformation relationship between each vertex in the source head model and the corresponding vertex in the source head model after deformation matching, that is, the transformation matrix (also called deformation field) from the vertex in the source head model to the corresponding vertex in the source head model after deformation matching. The target triangle in the source head model and the vertices of the target triangle are used as an example for explanation. The target triangle can be any triangle in the source head model. For example, see Figure 5, which is a schematic diagram of the position coordinates of a source head model and a target triangle in the source head model after deformation matching provided in an embodiment of the present application; the position coordinates of the three vertices of the target triangle in the source head model are respectively expressed as: the position coordinates V1 of the first vertex, the position coordinates V2 of the second vertex, and the position coordinates V3 of the third vertex, and the normal vector of the target triangle is represented by V4; the position coordinates of the three vertices of the target triangle in the source head model after deformation matching are respectively expressed as: the position coordinates V′1 of the first vertex, the position coordinates V′2 of the second vertex, and the position coordinates V′3 of the third vertex, and the normal vector of the target triangle is represented by V′4. First, the model processing device can determine the normal vector of the target triangle according to the positions of the vertices of the target triangle in the source head model, and further determine the local coordinate system of the target triangle according to the positions and normal vectors of the vertices of the target triangle in the source head model.

[0071] The normal vector of the target triangle in the source head model can be expressed by the following formula 1.1:

[0072] The local coordinate system of the target triangle in the source head model can be expressed as follows:

[0073] V=[V2-V1 V3-V1 V4-V1] (1.2)

[0074] Furthermore, the model processing device can determine the normal vector of the target triangle based on the positions of each vertex of the target triangle in the source head model after deformation matching, and further determine the local coordinate system of the target triangle based on the positions and normal vector of each vertex of the target triangle in the source head model after deformation matching.

[0075] The normal vector of the target triangle in the source head model after deformation matching can be expressed by the following formula 2.1:

[0076] The local coordinate system of the target triangle in the source head model after deformation matching can be expressed by the following formula 2.2:

[0077] V′=[V′2-V′1 V′3-V′1 V′4-V′1] (2.2)

[0078] By defining a spatial affine transformation parameter Q and a translation parameter t and applying them to the three vertices of the target triangle in the source head model, the position coordinates of the three vertices of the target triangle in the source head model after deformation matching are obtained. The corresponding relationship between the position coordinates of the vertices of the target triangle in the source head model and the corresponding vertices of the target triangle in the source head model after deformation matching can be shown by the following formula 3:

[0079] Q*V i +t=V′ i (3)

[0080] Here, i represents the independent variable of the vertex of the target triangle, i∈[1, 3].

[0081] Furthermore, the model processing device can determine affine transformation parameters based on the local coordinate system of the target triangle in the source head model and the local coordinate system of the target triangle in the source head model after deformation matching; and determine translation parameters based on the positions of each vertex of the target triangle in the source head model and the positions of each vertex of the target triangle in the source head model after deformation matching. Among them, the affine transformation parameter Q can be expressed by the following formula 4, and the translation parameter t can be expressed by the following formula 5:

[0082] Q=V′V -1 (4)

[0083] t=[V′1-V1 V′2-V2 V′3-V3] (5)

[0084] Furthermore, the model processing device can determine the vertex deformation relationship between each vertex of the target triangle in the source head model and the corresponding vertex in the source head model after deformation matching based on the determined affine transformation parameters and the determined translation parameters, that is, the transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the source head model after deformation matching. When the target triangle is the i-th triangle among the multiple triangles of the source head model, the transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the source head model after deformation matching can be shown by the following formula 6, where the transformation matrix includes the vertex deformation relationship of the three vertices in the i-th triangle:

[0085] T″ i =[Q i ,t i ] (6)

[0086] Among them, Q i Indicates the affine transformation parameters corresponding to the ith triangle determined based on the local coordinate system of the ith triangle in the source head model and the local coordinate system of the ith triangle in the source head model after deformation matching, t i Indicates the translation parameter corresponding to the ith triangle determined based on the positions of the vertices of the ith triangle in the source head model and the positions of the vertices of the ith triangle in the source head model after deformation matching, T″ i Represents the transformation matrix from the vertex of the i-th triangle in the source head model to the corresponding vertex in the deformed and matched source head model, that is, the transformation matrix from the i-th triangle in the source head model to the i-th triangle in the deformed and matched source head model.

[0087] S304 , determining deformation parameters according to the deformation relationship between the source head model and the deformed and matched source head model, and the deformation relationship from the first model expression to the second model expression in the source head model.

[0088] Among them, the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction in the multiple expression feature data. That is to say, according to the position of each vertex included in the neutral feature data, a first model expression can be rendered. The first model expression can be, for example, a face without common expressions. The source head model can be provided with a plurality of expression feature data with common expressions, such as smiling expression feature data and grinning expression feature data. Based on the expression action instruction, expression feature data of a common expression can be determined therefrom. These expression feature data can be used to render the corresponding expression. For example, if the expression action instruction indicates a WeChat instruction, then the second model expression is a smiling expression.

[0089] Furthermore, the deformation relationship from the first model expression to the second model expression in the source head model can be determined based on the positions of each feature point in the source head model with the second model expression and the positions of the corresponding feature points in the source head model with the first model expression; specifically, it is to determine the vertex deformation relationship between each vertex in the source head model with the first model expression and the corresponding vertex in the source head model with the second model expression, that is, the transformation matrix (also called deformation field) from the vertex in the source head model with the first model expression to the corresponding vertex in the source head model with the second model expression; this process is similar to the process of determining the deformation relationship between the above-mentioned source head model and the source head model after deformation matching, and will not be repeated here.

[0090] In one embodiment, the deformation parameters may include vertex deformation parameters of vertices in the target head model; taking the vertex deformation parameters of a target vertex in the target head model as an example, the target vertex may be any vertex in the target head model. At this time, the model processing device determines the deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model. It may include: determining a first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching based on the deformation relationship between the source head model and the source head model after deformation matching; determining a second vertex deformation relationship between each reference vertex in the source head model with the first model expression and the corresponding vertex in the source head model with the second model expression based on the deformation relationship from the first model expression in the source head model; and determining the vertex deformation parameters of the target vertex corresponding to each reference vertex in the target head model based on the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex. If the target vertex is the i-th vertex in the target head model, i∈[1, the number of vertices in the target head model], then the vertex deformation parameter of the target vertex in the target head model can be expressed by the following formulas 7.1 and 7.2:

[0091] T i =∑ j∈N(i) w i,j T′ j T″ j (7.1)

[0092] ∑ i∈N(i) w i,j =1 (7.2)

[0093] Wherein, N(i) represents a reference triangle index set consisting of the indices of the reference triangles corresponding to the target vertex (i.e., the i-th vertex) in the target head model in the source head model, wherein the reference vertices corresponding to the target vertex (i.e., the i-th vertex) in the source head model are the vertices of the reference triangles. Further optionally, the reference triangle can be the nearest triangle in the source head model corresponding to the i-th vertex in the target head model.

[0094] T″ j Represents the transformation matrix from the vertex of the j-th triangle in the source head model to the corresponding vertex in the deformed matching source head model, that is, the transformation matrix from the j-th triangle in the source head model to the j-th triangle in the deformed matching source head model, T′ jrepresents the transformation matrix from the vertex of the j-th triangle in the source head model with the first model expression to the corresponding vertex in the source head model with the second model expression, that is, the transformation matrix from the j-th triangle in the source head model with the first model expression to the j-th triangle in the source head model with the second model expression; w i,j It represents the proximity weight of the i-th vertex in the target head model corresponding to the j-th triangle face in the source head model, which can be set according to specific needs.

[0095] S305 , performing expression transfer processing on the target head model according to the deformation parameters to obtain a target head model with a second model expression.

[0096] In one feasible embodiment, the model processing device performs expression migration processing on the target head model based on the deformation parameters to obtain the target head model with the second model expression, which may include: adjusting the positions of corresponding vertices in the target head model with the first model expression based on the vertex deformation parameters of each vertex in the target head model to obtain the target head model with the second model expression. Taking the target vertex in the target head model (with the first model expression) as the i-th vertex as an example, the process of adjusting the position of the target vertex based on the vertex deformation parameters of the target vertex in the target head model can be shown by the following formula 8:

[0097] in, represents the position of the target vertex (i.e., the i-th vertex) in the target head model, Indicates the adjusted position of the target vertex.

[0098] Refer to Figure 6, which is a schematic diagram of expression migration to a target head model provided in an embodiment of the present application; after obtaining the source head model and the target head model, the model processing device can perform deformation matching processing on the source head model (having the first model expression, i.e., having a neutral model expression) according to the model features of the target head model (having the first model expression, i.e., having a neutral model expression) to obtain the deformed and matched source head model, wherein the deformed and matched source head model and the target head model are close; further, the model processing device can determine the deformation relationship between the source head model and the deformed and matched source head model, as well as the deformation relationship from the first model expression in the source head model to the second model expression, and determine the deformation parameters based on the two determined deformation relationships, and perform expression migration processing on the target head model based on the deformation parameters to obtain the target head model with the second model expression, so as to realize the second model expression of the target head model, that is, to realize the migration of the second model expression from the source head model to the target head model. Furthermore, in the source head model, the index of each reference triangle corresponding to the target vertex in the target head model can be represented by the proximity mapping corresponding to the target vertex. Based on this, the model processing device can obtain the proximity mapping table corresponding to the corresponding vertex in the process of determining the vertex deformation parameters of the vertex in the target head model, and determine the vertex deformation parameters of the corresponding vertex based on the proximity mapping table and the two determined deformation relationships, and adjust the corresponding vertex based on the vertex deformation parameters.

[0099] In the embodiment of the present application, the migration of the second model expression from the source head model to the target head model can be automatically realized, that is, the second model expression can be automatically generated for the target head model, and the second model expression in the source head model can be reused in different head models, which can further improve the efficiency of generating model expressions for different head models and reduce human resource consumption. Furthermore, in the process of determining the deformation relationship, the deformation relationship between the source head model and the source head model after deformation matching is determined according to the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model, and then the deformation parameters can be determined according to the deformation relationship between the source head model and the source head model after deformation matching, as well as the deformation relationship from the first model expression in the source head model to the second model expression, and the deformation parameters can be determined by using the feature points on the model as the analysis object to determine the deformation relationship and deformation parameters, which can improve the accuracy and efficiency of the model deformation; compared with determining the complete expression deformation parameters according to the deformation relationship from the first model expression in the source head model to the second model expression, due to the source head model Having different facial features from the target head model may make the determined deformation parameters incompatible with the target head model, resulting in poor effect of migrating the second model expression to the target head model. In this solution, on the one hand, the source head model after deformation matching is brought close to the target head model, so that the deformation relationship corresponding to the expression change in the source head model can be used to control the deformation control of the target head model. On the other hand, the expression transformation on the target head model is realized based on the deformation relationship of feature points rather than the deformation relationship of pixel points, which can improve the adaptation efficiency of the determined deformation parameters and the target head model, and further improve the effect of expression migration based on deformation parameters, that is, improve the effect of migrating the second model expression to the target head model.

[0100] Based on the relevant embodiments of the above-mentioned model processing method, the embodiment of the present application provides another model processing method. Referring to Figure 7, there is a flow chart of another model processing method provided in the embodiment of the present application. The model processing method shown in Figure 7 can be executed by the model processing device mentioned above. The model processing method shown in Figure 7 may include the following steps:

[0101] S701, in response to the acquired facial expression action instruction, acquire a source head model.

[0102] Among them, the source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data.

[0103] S702 , performing deformation matching processing on the source head model according to the model features of the target head model to obtain a deformed and matched source head model.

[0104] S703 , determining deformation parameters according to the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model.

[0105] The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instructions in the multiple expression feature data. The deformation parameters include vertex deformation parameters of vertices in the facial feature region of the target head model. The relevant processes of steps S701 to S703 are similar to the relevant processes of steps S201 to S203 and steps S301 to S304 described above, and are not further described here.

[0106] S704: Determine a plurality of feature regions constituting the target head model in the target head model.

[0107] In one embodiment, the model processing device determines multiple feature areas constituting the target head model in the target head model, which may include: determining the connected area (i.e., connected area) of the target head model in the target head model, and determining the overlapping boundaries between each connected area; performing region merging processing on each connected area according to the length of each determined overlapping boundary, and using the multiple connected areas obtained by region merging as multiple feature areas. Optionally, when the model processing device performs region merging processing on each connected area according to the length of each determined overlapping boundary, the corresponding connected area may be region merged when the length of the determined overlapping boundary is greater than a length threshold, or the corresponding connected area may be region merged when the ratio of the length of the determined overlapping boundary to the boundary length of the corresponding connected area is greater than an overlapping ratio threshold, wherein the length threshold and the overlapping ratio threshold can be set according to specific needs. Referring to Figure 8, a schematic diagram of a feature area provided in an embodiment of the present application is provided, and an exemplary feature area can be shown as marked 801.

[0108] S705 , adjusting positions of corresponding vertices in the target head model having the first model expression according to vertex deformation parameters of respective vertices in the facial feature region of the target head model, to obtain an adjusted target head model.

[0109] The facial feature region is the feature region with the largest surface area among the multiple feature regions. The feature region with the largest surface area among the multiple feature regions is determined as the facial feature region because the connected region formed by the face and head has the largest actual surface area, not the largest number of vertices. The process in which the model processing device adjusts the positions of corresponding vertices in the target head model having the first model expression based on the vertex deformation parameters of each vertex in the facial feature region in the target head model is similar to the process in step S305 in which the positions of corresponding vertices in the target head model having the first model expression are adjusted based on the vertex deformation parameters of each vertex in the target head model. Such a process is not further described here. This process is also known as the face pinching process, and the positions of the vertices in the facial feature region in the target head model can be changed.

[0110] S706 , based on the adjusted target head model, and based on the adjustment rule of the target feature region in the multiple feature regions, performing expression adjustment processing on the target feature region to obtain a target head model with a second model expression.

[0111] In one embodiment, the target feature area can be any other feature area among the multiple feature areas except the facial feature area, for example, it can be a feature area indicating eyebrows, a feature area indicating eyelashes, a feature area indicating tongue, etc.; the model processing device performs expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas, which may include: determining the linkage feature area corresponding to the target feature area from the multiple feature areas; based on the indication of the adjustment rule of the target feature area, following the adjustment of the linkage feature area on the target feature area; that is, based on the adjustment rule of the target feature area, the target feature area can be subjected to expression adjustment processing following the adjustment of the linkage feature area.

[0112] The linkage feature region corresponding to the target feature region may be a feature region among multiple feature regions that can be used to guide the target feature region in performing expression adjustment processing. Generally, a facial feature region can be determined as the linkage feature region corresponding to the target feature region. Furthermore, based on the indication of the adjustment rule for the target feature region, the model processing device performs adjustment processing on the target feature region in accordance with the adjustment of the linkage feature region. This may include: if the overlapping area between the target feature region and the linkage feature region is less than an area threshold, adjusting the positions of vertices in the target feature region according to a rigid adjustment method, following the adjustment of the vertex positions in the linkage feature region; if the overlapping area between the target feature region and the linkage feature region is greater than or equal to the area threshold, adjusting the positions of vertices in the target feature region according to a Laplace method, following the adjustment of the vertex positions in the linkage feature region. The area threshold can be set according to specific needs, and the rigid adjustment method includes rigid rotation and translation. For example, if the target feature region is a feature region indicating eyebrows, the linkage feature region corresponding to the eyebrow feature region is a facial feature region, and the overlapping area between the eyebrow feature region and the facial feature region is less than the area threshold, adjusting the positions of vertices in the eyebrow feature region according to the rigid adjustment method, following the adjustment of the vertex positions in the facial feature region. After determining the target feature area where the expression transformation needs to be completed, the present application will also determine the feature area that is linked to the target feature area. In this way, when the expression transformation is adjusted in the target area, the linked feature area will also move accordingly, making the expression transformation more natural.

[0113] In a feasible embodiment, when the positions of the vertices in the target feature area are adjusted according to the rigid adjustment method following the adjustment of the positions of the vertices in the linkage feature area, the rigid adjustment parameters can be determined according to the positions of the vertices in the linkage feature area before adjustment and the positions of the vertices in the linkage feature area after adjustment, and the positions of the vertices in the target feature area are rigidly adjusted based on the rigid adjustment parameters. When the linkage feature area is a facial feature area, the positions of the vertices in the linkage feature area before adjustment are the positions of the vertices in the facial feature area in the target head model, and the positions of the vertices in the linkage feature area after adjustment are the positions of the vertices in the facial feature area in the target head model after adjustment. When determining the rigid adjustment parameters based on the positions of the vertices before and after the linkage feature area is adjusted, the center points of the first point cloud matrix and the second point cloud matrix can be determined based on the first point cloud matrix composed of the positions of the vertices before and after the linkage feature area is adjusted; the decentered first point cloud matrix is ​​determined based on the first point cloud matrix and the corresponding center point, and the decentered second point cloud matrix is ​​determined based on the second point cloud matrix and the corresponding center point, and then the decentralized first point cloud matrix and the decentralized second point cloud matrix can be subjected to singular value decomposition (SVD) to obtain the rotation matrix and translation parameters, wherein the decomposed rotation matrix and translation parameters constitute the rigid adjustment parameters.

[0114] Optionally, taking the linkage feature area as the facial feature area as an example, the facial feature area before adjustment can be considered as the mesh component (i.e., mesh component) corresponding to the neutral model expression, and the facial feature area after adjustment can be considered as the mesh component corresponding to a certain non-neutral model expression. The first point cloud matrix formed by the positions of the vertices of the linkage feature area before adjustment can be shown by the following formula 9.1, and the second point cloud matrix formed by the positions of the vertices of the linkage feature area after adjustment can be shown by the following formula 9.2; wherein, if the vertices in the linkage feature area are counted from 0, then [x0 y0 z0] represents the position coordinates of the vertices whose count is 0 before the linkage feature area is adjusted (i.e., the coordinates of the x-axis, y-axis, and z-axis), [x n y n z n] represents the position coordinates of the vertex counted as n before the linkage feature area is adjusted (i.e., the coordinates of the x-axis, y-axis, and z-axis), where since the vertices in the linkage feature area are counted from 0, n is the value after the number of vertices in the linkage feature area is subtracted by 1, that is, n+1 represents the number of vertices in the linkage feature area; [x′0 y′0 z′0] represents the position coordinates of the vertex counted as 0 after the linkage feature area is adjusted (i.e., the coordinates of the x-axis, y-axis, and z-axis), that is, [x′ n y′ n z′ n ] represents the position coordinates of the vertex with a count of n after the linkage feature area is adjusted (i.e., the coordinates of the x-axis, y-axis, and z-axis):

[0115] The center point of the first point cloud matrix can be expressed by the following formula 9.3, and the center point of the second point cloud matrix can be expressed by the following formula 9.4:

[0116] The centralized first point cloud matrix can be expressed by the following formula 9.5, and the decentralized second point cloud matrix can be expressed by the following formula 9.6:

[0117] The corresponding relationship when performing singular value decomposition on the decentralized first point cloud matrix and the decentralized second point cloud matrix can be expressed as follows:

[0118] Formula 9.7 shows that the decomposed rotation matrix (R) can be expressed by the following formula 9.8, and the translation parameter (t) can be expressed by the following formula 9.9:

[0119] R=U T V (9.8)

[0120] Among them, U represents a left unitary matrix, S represents a non-negative real diagonal matrix, and V represents a right unitary matrix.

[0121] In one embodiment, the linkage feature region corresponding to the target feature region may be a feature region among multiple feature regions that can be used to guide the target feature region to perform expression adjustment processing. Generally speaking, a facial feature region may be determined as the linkage feature region corresponding to the target feature region. Furthermore, when the target feature region belongs to the eyeball appendage region, the linkage feature region corresponding to the target feature region may also be the eyeball region determined from multiple feature regions (i.e., the multiple feature regions may include the eyeball region). Therefore, the model processing device needs to determine the eyeball region and the eyeball appendage region from the multiple feature regions, wherein the eyeball appendage region is a feature region indicating the eyeball appendages, and the eyeball appendages may include eyeball contents, eyelids, eyelashes, eyebrows, and other structural components related to the eyeball.

[0122] The method in which the model processing device determines the eyeball area from multiple feature areas includes: determining the center point and sphere similarity of each feature area; determining the eyeball area from multiple feature areas based on the distance between the center point of each feature area and the eyeball key point in the facial key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the sphere similarity of each feature area; among which, the sphere similarity (Circularity) is also called sphericity, which can measure the degree to which an object belongs to a sphere, and the sphere similarity of a feature area is used to indicate the degree to which the corresponding feature area belongs to a sphere.

[0123] The model processing device determines the center point and sphere similarity of each feature area, which may include: for the feature area currently traversed in each feature area, determining the center point of the feature area currently traversed according to the positions of each vertex in the feature area currently traversed; determining the radius of each vertex in the feature area currently traversed from the corresponding center point, and the average radius corresponding to each vertex in the feature area currently traversed according to the positions of each vertex in the feature area currently traversed and the position of the center point; determining the sphere similarity of the feature area currently traversed according to the difference between the radius of each vertex in the feature area currently traversed from the corresponding center point and the average radius; further, the average value of the absolute value of the difference between the radius of each vertex in the feature area currently traversed from the corresponding center point and the average radius can be used as the sphere similarity of the feature area currently traversed.

[0124] Among them, when the model processing device determines the center point of the currently traversed feature area based on the positions of each vertex in the currently traversed feature area, it can average the positions of each vertex in the currently traversed feature area and use the position obtained by the average processing as the position of the center point of the currently traversed feature area; the position of the center point of the currently traversed feature area can be expressed by the following formulas 10.1, 10.2 and 10.3, where N represents the number of vertices in the currently traversed feature area, i is the independent variable, and if the vertices in the currently traversed feature area are counted starting from 0, then x i Indicates the x-axis coordinate of the vertex with count i in the currently traversed feature area, i Indicates the y-axis coordinate of the vertex with count i in the currently traversed feature area, z i Indicates the z-axis coordinate of the vertex with count i in the currently traversed feature area. as well as Respectively represent the x-axis, y-axis, and z-axis coordinates of the center point of the currently traversed feature area:

[0125] Wherein, when the model processing device determines the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius corresponding to each vertex in the currently traversed feature area based on the position of each vertex in the currently traversed feature area and the position of the center point, the radius of any vertex in the currently traversed feature area from the corresponding center point can be the distance between any vertex and the corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature area is obtained by averaging the radius of each vertex in the currently traversed feature area from the corresponding center point; wherein, the radius of the vertex in the currently traversed feature area from the corresponding center point can be expressed by the following formula 10.4, and the average radius corresponding to each vertex in the currently traversed feature area can be expressed by the following formula 10.5, R i Indicates the radius of the vertex with count i in the currently traversed feature area from the corresponding center point. Indicates the average radius corresponding to each vertex in the currently traversed feature area:

[0126] The model processing device can determine the sphere similarity of the currently traversed feature area based on the difference between the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius. For example, the average of the absolute values ​​of the difference between the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius can be used as the sphere similarity of the currently traversed feature area. The sphere similarity of the currently traversed feature area can be expressed by the following formula 10.6, where Circularity represents the sphere similarity:

[0127] It can be seen from formula 10.6 that the spherical similarity of a feature area is negatively correlated with the degree to which the corresponding feature area belongs to a sphere; based on this, the model processing device determines the eyeball area from multiple feature areas according to the distance between the center point of each feature area and the eyeball key point in the facial key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the spherical similarity of each feature area. It can include: taking the feature area with a spherical similarity less than the similarity threshold in each feature area as a candidate eyeball area; determining the distance between the center point of each candidate eyeball area and the eyeball key point in the facial key points of the adjusted target head model; and determining the candidate eyeball area indicated by the minimum distance in each candidate eyeball area as the eyeball area. Among them, the similarity threshold can be set according to specific needs, and the eyeball key point is a facial key point that can indicate the eyeball; since the eyeball can include the left eyeball and the right eyeball, the eyeball key point can include the left eyeball key point and the right eyeball key point. Based on this, when determining the distance between the center point of each candidate eyeball area and the eyeball key point of the adjusted target head model, the distance between the left eyeball key point and the right eyeball key point can be determined respectively, and then the candidate indicated by the minimum distance from the left eyeball key point in each candidate eyeball area is selected. The eyeball area is determined to be the left eyeball area, and the candidate eyeball area indicated when there is a minimum distance with the right eyeball key point is determined to be the right eyeball area. That is to say, the left eyeball area and the right eyeball area can be determined based on the similarity between the center point of the candidate eyeball area and the left eyeball key point and the right eyeball key point; since the eyeball in the three-dimensional model is usually a multi-layer concentric spherical shell, used to represent the iris, pupil, sclera, cornea and other components that constitute the eyeball, the determined left eyeball area and the right eyeball area may include multiple concentric spherical shells (also called multi-layer eyeball components). The eyeball area is determined based on the above method because: when the sphere similarity is lower than the similarity threshold, or the center point of the feature area is too far away from the eyeball space center with the highest confidence, it is not considered to be the eyeball area. Based on the features of the eyes, the eye area and eyeball position in the model can be determined more accurately through the calculation method mentioned above, so that when some expressions involving the eyeballs change, the movement control of the feature points can be better performed, and the expressions involving the eyes can be more naturally displayed in the target head model.

[0128] Since the linkage feature area corresponding to the eyeball appendage area may be the eyeball area, that is, the eyeball appendage area may be adjusted along with the adjustment of the eyeball area, therefore, after the model processing device determines the eyeball area from multiple feature areas, it can also perform expression adjustment processing on the eyeball area; generally speaking, the expression adjustment processing of the eyeball area mainly involves the rotation of the eyeball, and the positions of the corresponding vertices in the adjusted target head model can be adjusted according to the vertex deformation parameters of each vertex in the eyeball area in the target head model, that is, based on the blendshape rotation. Referring to FIG9 , a schematic diagram of eyeball region positioning and rotation provided for the implementation of the present application is provided; the model processing device can determine multiple feature regions constituting the target head model in the target head model, and determine the eyeball region from the multiple feature regions; in the process of determining the eyeball region from the multiple feature regions, the eyeball region can be determined based on the center points of the determined feature regions and the similarity of the spheres (i.e., the multi-layer eyeball components constituting the eyeball region are determined). The center point of the eyeball region, i.e., the center point of the multi-layer eyeball component, is determined based on the positions of the vertices in the multi-layer eyeball component. Since the multi-layer eyeball component is a concentric spherical shell, the center point of the eyeball region can also be determined based on the positions of the vertices in any eyeball component. Furthermore, the model processing device can perform expression adjustment processing on the eyeball region, such as based on blendshape rotation.

[0129] In one embodiment, after determining the eyeball region and the eyeball appendage region from multiple feature regions, a linkage feature region corresponding to a target feature region (i.e., the eyeball appendage region) can be determined from the multiple feature regions. Specifically, if the target feature region belongs to the eyeball appendage region, the positional relationship between the target feature region and the eyeball region among the multiple feature regions is determined; based on the positional relationship between the target feature region and the eyeball region, a linkage feature region is determined from the eyeball region and the facial feature regions; wherein, if the distance between the target feature region and the eyeball region is less than a distance threshold, the eyeball region is determined as the linkage feature region; otherwise, the facial feature region is determined as the linkage feature region. The eyeball appendage region is a feature region indicating eyeball appendages, which may include eyeball contents, eyelids, eyelashes, eyebrows, and other eye-related structural components. When the model processing device determines the linkage feature area from the eyeball area and the facial feature area according to the positional relationship between the target feature area and the eyeball area, it can be determined based on the distance between the target feature area and the eyeball area. If the distance between the target feature area and the eyeball area is less than the distance threshold, the eyeball area is determined as the linkage feature area; otherwise, the facial feature area is determined as the linkage feature area. The distance threshold can be set according to specific needs. For example, the eyeball area can be determined as the linkage feature area corresponding to the eyeball appendage area indicating the eyeball contents, and the facial feature area can be determined as the linkage feature area corresponding to the eyeball appendage area indicating the eyelids, the eyeball appendage area indicating the eyelashes, and the eyeball appendage area indicating the eyebrows.

[0130] Further optionally, when the model processing device determines the linkage feature area from the eyeball area and the facial feature area based on the positional relationship between the target feature area and the eyeball area, it can also determine the type of each eyeball appendage area based on the positional relationship between the target feature area (i.e., the eyeball appendage area) and the eyeball area, and determine the linkage feature area from the eyeball area and the facial feature area based on the type of each eyeball appendage area; for example, the eyeball appendage area that overlaps with the eyeball area among the eyeball appendage areas can be determined as the eyeball appendage area indicating the eyeball contents, and each eyeball appendage area can be determined as the eyeball appendage area indicating the eyeball contents. Among the eyeball appendage areas, the eyeball appendage area located above the eyeball area and closer to the facial key point indicating the eyebrow among the facial key points is determined as the eyeball appendage area indicating the eyebrow; when further determining the linkage feature area from the eyeball area and the facial feature area according to the type of each eyeball appendage area, the eyeball area can be determined as the linkage feature area corresponding to the eyeball appendage area indicating the eyeball contents, and the facial feature area can be determined as the linkage feature area corresponding to the eyeball appendage area indicating the eyelid, the eyeball appendage area indicating the eyelash, and the eyeball appendage area indicating the eyebrow.

[0131] In one embodiment, the model processing device may also determine an oral region from multiple feature regions, and further determine an oral appendage region; wherein the oral appendage region is a feature region indicating the oral appendage area, and oral appendages may include upper and lower jaws, teeth, tongue, and other structural components related to the oral cavity; after determining the oral appendage region, the model processing device may determine the feature region indicating the upper and lower jaws as the linked feature region corresponding to the oral appendage region, so that the oral appendage region can be adjusted in accordance with the adjustment of the feature region indicating the upper and lower jaws, for example, it can be rigidly adjusted in accordance with the adjustment of the feature region indicating the upper and lower jaws. It is worth noting that, within the oral appendage region, the oral appendage region indicating the tongue can also be adjusted accordingly based on the vertex deformation parameters of the vertices in the oral appendage region indicating the tongue in the target head model.

[0132] Referring to FIG10 , there is shown a schematic diagram of an embodiment of the present application for performing expression migration processing on a target head model. Based on the connected areas of the target head model, multiple feature regions constituting the target head model are determined in the target head model. A facial feature region with the largest surface area is determined from the multiple feature regions, and the positions of corresponding vertices in the target head model having the first model expression are adjusted based on vertex deformation parameters of each vertex in the facial feature region of the target head model to obtain an adjusted target head model. An eyeball region and an eyeball appendage region are determined from the multiple feature regions. Based on the adjusted target head model, expression adjustment processing is performed on the eyeball region and the eyeball appendage region. For example, the eyeball appendage region indicating the eyelashes is adjusted in accordance with the adjustment of the facial feature region. An oral appendage region is determined from the multiple feature regions. Based on the expression adjustment processing performed on the eyeball region and the eyeball appendage region, expression adjustment processing is performed on the oral appendage region. For example, the oral appendage region indicating the upper and lower jaws and the oral appendage region indicating the teeth are adjusted in accordance with the adjustment of the feature region indicating the upper and lower chins.

[0133] In one embodiment, if the target feature area belongs to the eyelid area, the first model expression in the source head model includes: an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes: an expression indicating that the source head model is in a closed-eye state; then the model processing device performs expression adjustment processing on the target feature area based on the adjustment rules of the target feature area in multiple feature areas, which may include: determining the lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid area; according to the size parameters and positions of the eyeball area in multiple feature areas, and based on the Laplace equation, performing collision deformation processing on the eyelid area to obtain a deformed eyelid area; this process makes the upper eyelid boundary and the lower eyelid boundary in the deformed eyelid area coincide as much as possible. Among them, since the eyes include the left eye and the right eye, the adjustment of the eyelid area in the above process can be the adjustment of the eyelid area corresponding to the left eye, the adjustment of the eyelid area corresponding to the right eye, or the adjustment of the eyelid area corresponding to both eyes. For example, when the second model expression in the source head model indicates that the source head model is in the left eye closed state, the corresponding adjustment is the eyelid area corresponding to the left eye. When the second model expression in the source head model indicates that the source head model is in the right eye closed state, the corresponding adjustment is the eyelid area corresponding to the right eye. When the second model expression in the source head model indicates that the source head model is in the right eye closed state, the corresponding adjustment is the eyelid area corresponding to the right eye.

[0134] In an optional embodiment, the eyelid area can be determined from multiple feature areas, or it can be determined based on the distribution position of the facial key points in the target head model. In a specific implementation, the facial key points indicating the lower part of the eyebrows and the upper part of the eyes in the target head model can be connected by the shortest distance, and the connected area can be determined as the eyelid area. See Figure 11, which is a schematic diagram of determining the eyelid area provided in an embodiment of the present application; wherein, the facial key points indicating the lower part of the eyebrows can be shown as marked 1101, the facial key points indicating the upper part of the eyes can be shown as marked 1102, and the determined eyelid area can be shown as marked 1103. See Figure 12, which is a schematic diagram of the lower eyelid boundary and the upper eyelid boundary provided in an embodiment of the present application, wherein the upper eyelid boundary can be shown as marked 1201, and the lower eyelid boundary can be shown as marked 1202.

[0135] The model processing device performs collision deformation processing on the eyelid region based on the size parameters and position of the eyeball region in multiple feature regions and based on the Laplace equation. In the process of obtaining the deformed eyelid region, the size parameters and position of the eyeball region are used as new constraints in the Laplace equation. By solving the Laplace equation, the collision between the eyelid vertices and the eyeball can be resolved while maintaining the consistency of the mesh surface normal vector and the original vertex position as much as possible, thereby reducing the possibility of the eyeball penetrating the model during the collision deformation processing of the eyelid region. In addition, when performing collision deformation processing on the eyelid region, the boundary points except for the upper eyelid boundary area can be maintained stationary, and the remaining vertices can be deformed to make the upper eyelid boundary coincide with the lower eyelid boundary. This process, by adding constraints to the Laplace equation, can reduce the possibility of the eyeball penetrating the model during the collision deformation processing of the eyelid region, improve the migration effect of the second model expression indicating the closed eyes state to the target head model, and even make the target head model achieve a better "closed eyes" effect.

[0136] Referring to Figure 13, a schematic diagram of adjusting the eyelid area provided in an embodiment of the present application is provided; wherein, the reference facial key points in the two-dimensional facial image can be shown as marked 1301, and the model processing device can determine the facial key points in the target head model based on the reference facial key points in the two-dimensional facial image, and their distribution is shown as marked 1302; based on the shortest distance connection (also called the shortest path connection), the eyelid area can be determined from the target head model, wherein the determined eyelid area can be shown as marked 1303, and further, after determining the lower eyelid boundary and the upper eyelid boundary, the eyelid area can be subjected to collision deformation processing based on the Laplace equation to obtain the deformed eyelid area, and the upper eyelid boundary in the deformed eyelid area coincides with the lower eyelid boundary, and the corresponding adjustment effect can be shown as marked 1304.

[0137] In one embodiment, the target head model can be obtained by simplifying the head model area of ​​the interaction model, and the number of vertices in the target head model is less than the number of vertices in the head model area of ​​the interaction model; then, after obtaining the target head model with the second model expression, the model processing device can also: determine the grid area corresponding to the texture feature point in the target head model according to the position of the texture feature point in the head model area of ​​the interaction model, and determine the position mapping relationship between the texture feature point in the head model area of ​​the interaction model and the corresponding grid area in the target head model; determine the position of the texture feature point in the head model area of ​​the interaction model after position deformation according to the position mapping relationship and the positions of each vertex included in the corresponding grid area; and deform the texture feature points in the head model area of ​​the interaction model according to the position of the texture feature point in the head model area of ​​the interaction model after position deformation to obtain the interaction model with the second model expression. Among them, the interaction model can be a model used to interact with an object, such as a model obtained by pinching the user's face, a model in a film or television work, a model in a game scene, etc. Optionally, the interaction model can be a human body model, a head model, etc.; the texture feature point can be a texture coordinate point corresponding to the vertex, and further optionally, the position of the texture feature point can be the position indicated by the texture coordinate of the texture coordinate point, and the position of each vertex included in the grid area can be the position indicated by the texture coordinate of the corresponding vertex; the position mapping relationship of the texture feature point in the head model area of ​​the interaction model to the corresponding grid area in the target head model can be the centroid coordinates of the texture feature point in the corresponding grid area, and the grid area is a triangle.

[0138] In a feasible embodiment, when the model processing device simplifies the head model area of ​​the interactive model, it can perform texture expansion and decomposition of the head model area of ​​the interactive model, and then simplify the model by reducing vertices; the number of vertices required to be retained for model simplification can be set according to specific needs, for example, it can be set to retain 10,000 vertices, and the boundary points of the head model area of ​​the interactive model need to be retained when simplifying the model. Referring to Figure 14, a schematic diagram of a texture coordinate relationship provided in an embodiment of the present application; wherein, the texture feature point in the head model area of ​​the interactive model can be shown as marked 1401, the texture coordinates of the texture feature point can be represented as P', and its corresponding grid area can be shown as marked 1402, the texture coordinates of the three vertices can be represented as P1, P2, and P3 respectively, if the centroid coordinates of the texture feature point in the corresponding grid area are represented as [w1 w2 w3], then the position of the texture feature point after position deformation can be represented by the following formula 11:

[0139] P′=w1P1+w2P2+w3P3 (11)

[0140] Since the target head model is obtained by simplifying the head model area of ​​the interaction model, the head model of the interaction model can be considered as a high-poly model and the target head model as a low-poly model. This process is also the process of high-low poly expression migration, that is, the deformation of the low-poly target head model is migrated to the high-poly head model area of ​​the interaction model, so that the second model expression can be realized in the head model area of ​​the interaction model. Based on the above method, the deformation is migrated from the low poly model to the high poly model without losing accuracy. The resulting mixed shape (the mixed shape corresponding to the second model expression) does not need to be smoothed twice, and a higher-precision mixed shape result can be obtained on the high-poly model, that is, the second model expression effect realized on the high-poly model is good.

[0141] Referring to FIG15 , there is shown a schematic diagram of an embodiment of the present application for performing expression migration to an interactive model; the model processing device can obtain a source head model in response to an acquired expression action instruction, and perform model simplification on the head model region of the interactive model to obtain a target head model; the model processing device determines the key points of the face in the target head model; the model processing device performs deformation matching processing on the source head model based on the model features of the target head model to obtain a deformed and matched source head model, wherein the model features of the target head model may include: the positions of the key points of the face in the target head model. Furthermore, the model processing device can determine deformation parameters based on the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model, and perform expression migration processing on the target head model based on the deformation parameters to obtain a target head model having the second model expression. In the process of performing expression migration processing on the target head model based on the deformation parameters to obtain the target head model having the second model expression, the positions of the corresponding vertices in the target head model having the first model expression can be adjusted based on the vertex deformation parameters of each vertex in the facial feature region of the target head model to obtain an adjusted target head model; based on the adjusted target head model, the expression adjustment processing is performed on the target feature region based on the adjustment rules of the target feature region in the multiple feature regions to obtain the target head model having the second model expression. This process can be referred to as a post-migration processing process. Furthermore, the model processing device can determine the grid region corresponding to the texture feature point in the head model region of the interaction model based on the position of the texture feature point, and determine the position mapping relationship between the texture feature point in the head model region of the interaction model and the corresponding grid region in the target head model; determine the position of the texture feature point in the head model region of the interaction model after position deformation based on the position mapping relationship and the positions of each vertex included in the corresponding grid region; and perform deformation processing on the texture feature point in the head model region of the interaction model based on the position of the texture feature point in the head model region of the interaction model to obtain the interaction model having the second model expression.

[0142] Refer to Figure 16, which is a schematic diagram of the effect of expression migration provided by an embodiment of the present application; as shown in 1601 and 1602, the model processing method proposed based on this solution can enable the target head model to achieve a good "closed eyes" effect; in addition, since this method can perform deformation migration from a low-poly model to a high-poly model without losing accuracy, the resulting mixed shape (the mixed shape corresponding to the second model expression) does not need to be smoothed twice, and a higher-precision mixed shape result can be obtained on the high-poly model, that is, the second model expression effect achieved on the high-poly model is good. An exemplary effect achieved by implementing a certain second model expression on a high-poly model can be shown as marked 1603.

[0143] In one embodiment, the above-mentioned model processing method can be applied to object interaction scenarios, for example, it can be used to generate a model with certain model expressions to realize object interaction; for example, when the expression action instruction includes a target text for indicating the generation of a second model expression, the model processing device can perform speech synthesis processing based on the target text to obtain a target speech corresponding to the target text; and after obtaining a target head model with the second model expression, output a target head model with the second model expression and a target speech, so as to perform interaction processing based on the target head model. Among them, the speech synthesis processing can be performed by a TTS (Text To Speech) module, the target text can be input by the object, the target head model can be set by the object, for example, it can be the head model of a game character set by the object in the game scene, and the target head model and the target speech can be output to the terminal device of the object to realize interaction with the object. Refer to Figure 17, which is a schematic diagram of an application scenario provided in an embodiment of the present application; the model processing device can perform speech synthesis processing based on the target text to obtain a target speech corresponding to the target text when receiving an expression action instruction including a target text for instructing to generate a second model expression; obtain a source head model, and generate the second model expression expected by the expression action instruction for the source head model by performing instruction analysis processing on the expression action instruction; migrate the second model expression to the target head model to obtain a target head model with the second model expression; output the target head model with the second model expression and the target speech to perform interactive processing based on the target head model. In an embodiment of the present application, the expression display of the source head model and the migration of the expression to the target head model can be instructed by voice, which can more conveniently realize the expression control of the designed target head model, meeting the automation and intelligent requirements for the expression control of the head model.

[0144] In an embodiment of the present application, in the process of performing expression transfer processing based on deformation parameters to obtain a target head model having a second model expression, the positions of corresponding vertices in the target head model having the first model expression can be adjusted based on the vertex deformation parameters of each vertex in the facial feature region of the target head model to obtain an adjusted target head model. Further, based on the adjusted target head model, expression adjustment processing is performed on the target feature region based on adjustment rules for target feature regions in multiple feature regions to obtain a target head model having the second model expression. Deformation transfer of the eye region, eyeball appendage region, oral appendage region, and eyelid region in the target head model can be processed to improve the expression transfer effect of the corresponding regions. Furthermore, a low-poly model target head model can be obtained by simplifying a high-poly model head model, and expression transfer can be performed on the low-poly model target head model. After obtaining the target head model having the second model expression, the corresponding deformation can be transferred from the low-poly model to the high-poly model to complete the transfer of the second model expression in the high-poly model head model, thereby accelerating the expression transfer process of the high-poly model.

[0145] Based on the above-mentioned embodiments related to the model processing method, the present invention provides a model processing device. Referring to FIG18 , which is a schematic diagram of the structure of a model processing device provided in the present invention, the model processing device may include an acquisition unit 1801 and a processing unit 1802. The model processing device shown in FIG18 may be used to perform the following operations:

[0146] The acquisition unit 1801 is used to acquire a source head model in response to the acquired expression action instruction; the source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data; the processing unit 1802 is used to perform deformation matching processing on the source head model according to the model features of the target head model to obtain the deformed and matched source head model; the processing unit 1802 is also used to determine the deformation parameters based on the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model; wherein the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction in the multiple expression feature data; the processing unit 1802 is also used to perform expression migration processing on the target head model according to the deformation parameters to obtain a target head model with the second model expression.

[0147] In one embodiment, the processing unit 1802 performs deformation matching processing on the source head model according to the model features of the target head model, and when obtaining the deformed and matched source head model, specifically performs the following operations:

[0148] According to the positions of each feature point in the source head model and the positions of each feature point in the target head model, the matching relationship between each feature point in the source head model and each feature point in the target head model is determined; the feature points include: one or two of vertices in the corresponding model and facial key points; according to the matching relationship, the positions of the feature points in the target head model are used as reference positions, and the positions of the vertices constituting the source head model are adjusted to obtain the deformed and matched source head model; wherein the distance between the position of the feature point in the deformed and matched source head model and the corresponding reference position determined according to the matching relationship meets the proximity condition.

[0149] In one embodiment, when the processing unit 1802 determines the deformation parameters based on the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model, it specifically performs the following operations:

[0150] The deformation relationship between the source head model and the deformed source head model is determined based on the position of each feature point in the deformed source head model and the position of the corresponding feature point in the source head model; the feature points include: one or two of the vertices and facial key points in the corresponding model; the deformation parameters are determined based on the deformation relationship between the source head model and the deformed source head model, as well as the deformation relationship from the first model expression to the second model expression in the source head model.

[0151] In one embodiment, when the processing unit 1802 determines the deformation parameters based on the deformation relationship between the source head model and the deformed and matched source head model, and the deformation relationship from the first model expression to the second model expression in the source head model, it specifically performs the following operations:

[0152] According to the deformation relationship between the source head model and the source head model after deformation matching, the first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching is determined; according to the deformation relationship from the first model expression to the second model expression in the source head model, the second vertex deformation relationship between each reference vertex in the source head model with the first model expression and the corresponding vertex in the source head model with the second model expression is determined; according to the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex, the vertex deformation parameters of the target vertex corresponding to each reference vertex in the target head model are determined.

[0153] In one embodiment, when the processing unit 1802 performs expression transfer processing on the target head model according to the deformation parameters to obtain the target head model having the second model expression, the processing unit 1802 specifically performs the following operations:

[0154] A plurality of feature regions constituting the target head model are determined in the target head model; positions of corresponding vertices in a facial feature region in the target head model having a first model expression are adjusted based on vertex deformation parameters of corresponding vertices in the facial feature region in the target head model to obtain an adjusted target head model; wherein the facial feature region is a feature region having the largest surface area among the plurality of feature regions; and based on the adjusted target head model, expression adjustment processing is performed on the target feature region based on an adjustment rule for the target feature region in the plurality of feature regions to obtain a target head model having a second model expression.

[0155] In one embodiment, the processing unit 1802 performs the following operations when performing expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas:

[0156] A linkage feature region corresponding to a target feature region is determined from a plurality of feature regions; and based on an instruction of an adjustment rule of the target feature region, a follow-up adjustment process is performed on the target feature region according to the adjustment of the linkage feature region.

[0157] In one embodiment, the processing unit 1802 performs the following operations when performing follow-up adjustment processing on the target feature region according to the adjustment of the linkage feature region based on the indication of the adjustment rule of the target feature region:

[0158] If the overlapping area of ​​the target feature region and the linkage feature region is less than the area threshold, the positions of the vertices in the target feature region are adjusted according to the rigid adjustment method, following the adjustment of the vertex positions in the linkage feature region; if the overlapping area of ​​the target feature region and the linkage feature region is greater than or equal to the area threshold, the positions of the vertices in the target feature region are adjusted according to the Laplace method, following the adjustment of the vertex positions in the linkage feature region.

[0159] In one embodiment, when the processing unit 1802 determines the linked feature region corresponding to the target feature region from the multiple feature regions, it specifically performs the following operations:

[0160] If the target feature area belongs to the eyeball appendage area, the positional relationship between the target feature area and the eyeball area among the multiple feature areas is determined; based on the positional relationship between the target feature area and the eyeball area, a linkage feature area is determined from the eyeball area and the facial feature area; wherein, if the distance between the target feature area and the eyeball area is less than a distance threshold, the eyeball area is determined as the linkage feature area, otherwise, the facial feature area is determined as the linkage feature area.

[0161] In one embodiment, the multiple feature areas include an eyeball area; when the processing unit 1802 determines the eyeball area from the multiple feature areas, it specifically performs the following operations:

[0162] Determine the center point and sphere similarity of each feature area; the sphere similarity of a feature area is used to indicate the degree to which the corresponding feature area belongs to a sphere; determine the eyeball area from multiple feature areas based on the distance between the center point of each feature area and the eyeball key point in the facial key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the sphere similarity of each feature area.

[0163] In one embodiment, the sphere similarity of a feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere;

[0164] When determining the eye region from the plurality of feature regions, the processing unit 1802 specifically performs the following operations based on the distance between the center point of each feature region and the eye key point in the adjusted target head model's facial key points, and the degree to which each feature region belongs to a sphere as indicated by the sphere similarity:

[0165] Among the feature areas, the feature areas with a ball similarity less than a similarity threshold are selected as candidate eye areas; the distance between the center point of each candidate eye area and the eye key point in the facial key point of the adjusted target head model is determined; and the candidate eye area indicated by the minimum distance among the candidate eye areas is determined as the eye area.

[0166] In one embodiment, when the processing unit 1802 determines the center point of each feature region and the sphere similarity, it specifically performs the following operations:

[0167] For the currently traversed feature area in each feature area, the center point of the currently traversed feature area is determined according to the positions of each vertex in the currently traversed feature area; according to the positions of each vertex in the currently traversed feature area and the position of the center point, the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius corresponding to each vertex in the currently traversed feature area are determined; according to the difference between the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius, the sphere similarity of the currently traversed feature area is determined.

[0168] In one embodiment, the target feature area belongs to the eyelid area; the first model expression in the source head model includes: an expression indicating that the source head model is in an open-eyes state; the second model expression in the source head model includes: an expression indicating that the source head model is in a closed-eyes state;

[0169] When the processing unit 1802 performs expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas, the processing unit 1802 specifically performs the following operations:

[0170] The lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid area are determined; and the eyelid area is subjected to collision deformation processing based on the Laplace equation according to the size parameters and positions of the eyeball area in the multiple feature areas to obtain the deformed eyelid area.

[0171] In one embodiment, when the processing unit 1802 determines the multiple feature regions constituting the target head model in the target head model, it specifically performs the following operations:

[0172] The connected areas of the target head model are determined in the target head model, and the overlapping boundaries between the connected areas are determined; the connected areas are merged according to the lengths of the determined overlapping boundaries, and the multiple connected areas obtained by the region merging are used as multiple feature areas.

[0173] In one embodiment, the target head model is obtained by simplifying the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model. The processing unit 1802 is further configured to:

[0174] According to the positions of the texture feature points in the head model area of ​​the interaction model, the grid areas corresponding to the texture feature points are determined in the target head model, and the position mapping relationship between the texture feature points in the head model area of ​​the interaction model and the corresponding grid areas in the target head model is determined; according to the position mapping relationship and the positions of the respective vertices included in the corresponding grid areas, the positions of the texture feature points in the head model area of ​​the interaction model after position deformation are determined; according to the positions of the texture feature points in the head model area of ​​the interaction model after position deformation, the texture feature points in the head model area of ​​the interaction model are deformed to obtain an interaction model having an expression of the second model.

[0175] In one embodiment, the expression action instruction includes a target text for instructing to generate the second model expression; the processing unit 1802 is further configured to:

[0176] Speech synthesis processing is performed based on the target text to obtain a target speech corresponding to the target text; after obtaining a target head model with a second model expression, the target head model with the second model expression and the target speech are output to perform interactive processing based on the target head model.

[0177] According to one embodiment of the present application, the various steps involved in the model processing method shown in Figures 2, 3, and 7 can be performed by various units in the model processing device shown in Figure 18. For example, step S201 shown in Figure 2 can be performed by the acquisition unit 1801 in the model processing device shown in Figure 18, and steps S202 to S204 shown in Figure 2 can be performed by the processing unit 1802 in the model processing device shown in Figure 18. For another example, step S301 shown in Figure 3 can be performed by the acquisition unit 1801 in the model processing device shown in Figure 18, and steps S302 to S305 shown in Figure 3 can be performed by the processing unit 1802 in the model processing device shown in Figure 18. For another example, step S701 shown in Figure 7 can be performed by the acquisition unit 1801 in the model processing device shown in Figure 18, and steps S702 to S706 shown in Figure 7 can be performed by the processing unit 1802 in the model processing device shown in Figure 18.

[0178] According to another embodiment of the present application, the various units in the model processing device shown in Figure 18 can be separately or all merged into one or several other units to constitute, or some of the units can be further split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. For example, the functions implemented by the above-mentioned units can be implemented by one processing unit. In other embodiments of the present application, the model processing device based on logical function division can also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0179] According to another embodiment of the present application, a model processing device as shown in FIG18 can be constructed, and the model processing method of the embodiment of the present application can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding methods shown in FIG2, FIG3 and FIG7 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM). The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0180] In an embodiment of the present application, after obtaining an expression action instruction, the source head model can be deformed and matched to the target head model, so that the deformed and matched source head model approaches the target head model; then, based on the deformed and matched source head model and the deformation relationship from the first model expression in the source head model to the second model expression indicated by the expression action instruction, deformation parameters can be determined, and expression migration processing can be performed on the target head model based on the deformation parameters to obtain a target head model with the second model expression. The migration of the second model expression from the source head model to the target head model can be automated, that is, the second model expression can be automatically generated for the target head model, and the second model expression in the source head model can be reused in different head models, which can further improve the efficiency of generating model expressions for different head models and reduce human resource consumption.

[0181] Based on the relevant embodiments of the above-mentioned model processing method and the embodiment of the model processing device, the present application also provides a model processing device. Referring to Figure 19, a structural diagram of a model processing device provided in an embodiment of the present application is shown. The model processing device shown in Figure 19 may include at least a processor 1901, an input interface 1902, an output interface 1903, and a computer storage medium 1904. The processor 1901, the input interface 1902, the output interface 1903, and the computer storage medium 1904 may be connected via a bus or other means.

[0182] Computer storage medium 1904 can be stored in the memory of the model processing device. Computer storage medium 1904 is used to store a computer program, which includes program instructions. Processor 1901 is used to execute the program instructions stored in computer storage medium 1904. Processor 1901 (or CPU (Central Processing Unit)) is the computing core and control core of the model processing device. It is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the above-mentioned model processing method process or corresponding functions.

[0183] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in a model processing device for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the terminal and, of course, the extended storage medium supported by the terminal. The computer storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1901 are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0184] In one embodiment, the processor 1901 may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps of the method in the above-mentioned model processing method embodiments of FIG. 2 , FIG. 3 , and FIG. 7 . In a specific implementation, the processor 1901 loads one or more instructions in the computer storage medium and executes the following steps:

[0185] In response to the acquired expression action instruction, a source head model is acquired; the source head model is configured with neutral feature data and multiple expression feature data relative to the neutral feature data; according to the model features of the target head model, the source head model is deformed and matched to obtain the deformed and matched source head model; according to the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model, deformation parameters are determined; wherein, the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression action instruction in the multiple expression feature data; on the target head model, expression migration processing is performed according to the deformation parameters to obtain a target head model with the second model expression.

[0186] In one embodiment, the processor 1901 performs deformation matching processing on the source head model according to the model features of the target head model. When obtaining the deformed and matched source head model, the processor 1901 specifically performs the following operations:

[0187] According to the positions of each feature point in the source head model and the positions of each feature point in the target head model, the matching relationship between each feature point in the source head model and each feature point in the target head model is determined; the feature points include: one or two of vertices in the corresponding model and facial key points; according to the matching relationship, the positions of the feature points in the target head model are used as reference positions, and the positions of the vertices constituting the source head model are adjusted to obtain the deformed and matched source head model; wherein the distance between the position of the feature point in the deformed and matched source head model and the corresponding reference position determined according to the matching relationship meets the proximity condition.

[0188] In one embodiment, when the processor 1901 determines the deformation parameters based on the deformed and matched source head model and the deformation relationship from the first model expression to the second model expression in the source head model, it specifically performs the following operations:

[0189] The deformation relationship between the source head model and the deformed source head model is determined based on the position of each feature point in the deformed source head model and the position of the corresponding feature point in the source head model; the feature points include: one or two of the vertices and facial key points in the corresponding model; the deformation parameters are determined based on the deformation relationship between the source head model and the deformed source head model, as well as the deformation relationship from the first model expression to the second model expression in the source head model.

[0190] In one embodiment, when the processor 1901 determines the deformation parameters based on the deformation relationship between the source head model and the deformed and matched source head model, and the deformation relationship from the first model expression to the second model expression in the source head model, the processor 1901 specifically performs the following operations:

[0191] According to the deformation relationship between the source head model and the source head model after deformation matching, the first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching is determined; according to the deformation relationship from the first model expression to the second model expression in the source head model, the second vertex deformation relationship between each reference vertex in the source head model with the first model expression and the corresponding vertex in the source head model with the second model expression is determined; according to the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex, the vertex deformation parameters of the target vertex corresponding to each reference vertex in the target head model are determined.

[0192] In one embodiment, when the processor 1901 performs expression migration processing on the target head model according to the deformation parameters to obtain the target head model having the second model expression, the processor 1901 specifically performs the following operations:

[0193] A plurality of feature regions constituting the target head model are determined in the target head model; positions of corresponding vertices in a facial feature region in the target head model having a first model expression are adjusted based on vertex deformation parameters of corresponding vertices in the facial feature region in the target head model to obtain an adjusted target head model; wherein the facial feature region is a feature region having the largest surface area among the plurality of feature regions; and based on the adjusted target head model, expression adjustment processing is performed on the target feature region based on an adjustment rule for the target feature region in the plurality of feature regions to obtain a target head model having a second model expression.

[0194] In one embodiment, the processor 1901 performs the following operations when performing expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas:

[0195] A linkage feature region corresponding to a target feature region is determined from a plurality of feature regions; and based on an instruction of an adjustment rule of the target feature region, a follow-up adjustment process is performed on the target feature region according to the adjustment of the linkage feature region.

[0196] In one embodiment, the processor 1901 performs the following operations when following the adjustment of the linkage feature region based on the instruction of the adjustment rule of the target feature region:

[0197] If the overlapping area of ​​the target feature region and the linkage feature region is less than the area threshold, the positions of the vertices in the target feature region are adjusted according to the rigid adjustment method, following the adjustment of the vertex positions in the linkage feature region; if the overlapping area of ​​the target feature region and the linkage feature region is greater than or equal to the area threshold, the positions of the vertices in the target feature region are adjusted according to the Laplace method, following the adjustment of the vertex positions in the linkage feature region.

[0198] In one embodiment, when the processor 1901 determines the linked feature region corresponding to the target feature region from the multiple feature regions, the processor 1901 specifically performs the following operations:

[0199] If the target feature area belongs to the eyeball appendage area, the positional relationship between the target feature area and the eyeball area among the multiple feature areas is determined; based on the positional relationship between the target feature area and the eyeball area, a linkage feature area is determined from the eyeball area and the facial feature area; wherein, if the distance between the target feature area and the eyeball area is less than a distance threshold, the eyeball area is determined as the linkage feature area, otherwise, the facial feature area is determined as the linkage feature area.

[0200] In one embodiment, the multiple feature areas include an eye area; when the processor 1901 determines the eye area from the multiple feature areas, it specifically performs the following operations:

[0201] Determine the center point and sphere similarity of each feature area; the sphere similarity of a feature area is used to indicate the degree to which the corresponding feature area belongs to a sphere; determine the eyeball area from multiple feature areas based on the distance between the center point of each feature area and the eyeball key point in the facial key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the sphere similarity of each feature area.

[0202] In one embodiment, the sphere similarity of a feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere. When determining the eye region from multiple feature regions based on the distance between the center point of each feature region and the eye key point among the facial key points of the adjusted target head model, and the degree to which each feature region belongs to a sphere as indicated by the sphere similarity, the processor 1901 specifically performs the following operations:

[0203] Among the feature areas, the feature areas with ball similarity less than the similarity threshold are regarded as candidate eye areas; the distance between the center point of each candidate eye area and the eye key point in the face key point of the adjusted target head model is determined; and the candidate eye area indicated by the minimum distance among the candidate eye areas is determined as the eye area.

[0204] In one embodiment, when the processor 1901 determines the center point of each feature area and the sphere similarity, it specifically performs the following operations:

[0205] For the currently traversed feature area in each feature area, the center point of the currently traversed feature area is determined according to the positions of each vertex in the currently traversed feature area; according to the positions of each vertex in the currently traversed feature area and the position of the center point, the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius corresponding to each vertex in the currently traversed feature area are determined; according to the difference between the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius, the sphere similarity of the currently traversed feature area is determined.

[0206] In one embodiment, the target feature region belongs to the eyelid region; the first model expression in the source head model includes an expression indicating that the source head model is in an open-eyes state, and the second model expression in the source head model includes an expression indicating that the source head model is in a closed-eyes state; when the processor 1901 performs expression adjustment processing on the target feature region based on the adjustment rule of the target feature region among the multiple feature regions, the processor 1901 specifically performs the following operations:

[0207] The lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid area are determined; and the eyelid area is subjected to collision deformation processing based on the Laplace equation according to the size parameters and positions of the eyeball area in the multiple feature areas to obtain the deformed eyelid area.

[0208] In one embodiment, when the processor 1901 determines the multiple feature regions constituting the target head model in the target head model, it specifically performs the following operations:

[0209] The connected areas of the target head model are determined in the target head model, and the overlapping boundaries between the connected areas are determined; the connected areas are merged according to the lengths of the determined overlapping boundaries, and the multiple connected areas obtained by the region merging are used as multiple feature areas.

[0210] In one embodiment, the target head model is obtained by simplifying the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model. The processor 1901 is further configured to:

[0211] According to the positions of the texture feature points in the head model area of ​​the interaction model, the grid areas corresponding to the texture feature points are determined in the target head model, and the position mapping relationship between the texture feature points in the head model area of ​​the interaction model and the corresponding grid areas in the target head model is determined; according to the position mapping relationship and the positions of the respective vertices included in the corresponding grid areas, the positions of the texture feature points in the head model area of ​​the interaction model after position deformation are determined; according to the positions of the texture feature points in the head model area of ​​the interaction model after position deformation, the texture feature points in the head model area of ​​the interaction model are deformed to obtain an interaction model having an expression of the second model.

[0212] In one embodiment, the expression action instruction includes a target text for instructing to generate the second model expression; the processor 1901 is further configured to:

[0213] Speech synthesis processing is performed based on the target text to obtain a target speech corresponding to the target text; after obtaining a target head model with a second model expression, the target head model with the second model expression and the target speech are output to perform interactive processing based on the target head model.

[0214] The present application provides a computer program product, which includes a computer program stored in a computer storage medium. A processor of a model processing device reads the computer program from the computer storage medium and executes the computer program, causing the model processing device to perform the method embodiments shown in Figures 2, 3, and 7. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0215] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A model processing method, characterized in that: include: In response to the acquired expression action instruction, a source head model is acquired; The source head model is configured with neutral feature data and a plurality of expression feature data relative to the neutral feature data; According to the model features of the target head model, the source head model is subjected to deformation matching processing to obtain a source head model after deformation matching; Determine the deformation parameters according to the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model; wherein the first model expression in the source head model is determined according to the neutral feature data of the source head model, and the second model expression in the source head model is determined according to the expression feature data indicated by the expression action instruction in the plurality of expression feature data; On the target head model, expression migration processing is performed according to the deformation parameters to obtain a target head model with the expression of the second model.

2. The method according to claim 1, characterized in that The method of performing deformation matching processing on the source head model according to the model features of the target head model to obtain the source head model after deformation matching comprises: According to the positions of the feature points in the source head model and the positions of the feature points in the target head model, a matching relationship between the feature points in the source head model and the feature points in the target head model is determined; the feature points include: vertices in the corresponding models, and one or two of the key points of the face; According to the matching relationship, taking the positions of the feature points in the target head model as reference positions, adjusting the positions of the vertices constituting the source head model to obtain a deformed and matched source head model; Wherein, the distance between the position of the feature point in the source head model after the deformation matching and the corresponding reference position determined according to the matching relationship satisfies the proximity condition.

3. The method according to claim 1 or 2, characterized in that The step of determining deformation parameters according to the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model comprises: Determine the deformation relationship between the source head model and the source head model after deformation matching according to the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model; the feature point includes: one or two of the vertices and the key points of the face in the corresponding model; The deformation parameters are determined according to the deformation relationship between the source head model and the deformed and matched source head model, and the deformation relationship from the first model expression to the second model expression in the source head model.

4. The method according to any one of claims 1 to 3, characterized in that: The deformation relationship between the source head model and the deformed and matched source head model includes: a transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the deformed and matched source head model; wherein the transformation matrix between vertices is determined based on affine transformation parameters and translation parameters.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining the deformation parameters according to the deformation relationship between the source head model and the deformed and matched source head model, and the deformation relationship from the first model expression to the second model expression in the source head model, comprises: Determine, according to the deformation relationship between the source head model and the source head model after deformation matching, a first vertex deformation relationship between each reference vertex in the source head model and a corresponding vertex in the source head model after deformation matching; Determining, according to the deformation relationship from the first model expression to the second model expression in the source head model, a second vertex deformation relationship between each reference vertex in the source head model having the first model expression and a corresponding vertex in the source head model having the second model expression; According to the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex, vertex deformation parameters of target vertices corresponding to each reference vertex in the target head model are determined.

6. The method according to any one of claims 1 to 5, characterized in that: The step of performing expression migration processing on the target head model according to the deformation parameters to obtain a target head model having an expression of the second model comprises: Determining, in the target head model, a plurality of feature regions constituting the target head model; According to vertex deformation parameters of each vertex of the facial feature region in the target head model, positions of corresponding vertices in the target head model having the first model expression are adjusted to obtain an adjusted target head model; wherein the facial feature region is a feature region having the largest surface area among the multiple feature regions; On the basis of the adjusted target head model, based on the adjustment rule of the target feature region among the multiple feature regions, expression adjustment processing is performed on the target feature region to obtain the target head model with the second model expression.

7. The method according to any one of claims 1 to 6, characterized in that: The performing expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas comprises: Determining a linkage feature region corresponding to the target feature region from the multiple feature regions; Based on the indication of the adjustment rule of the target feature region, the target feature region is subjected to follow-up adjustment processing according to the adjustment of the linkage feature region.

8. The method according to any one of claims 1 to 7, characterized in that: The step of following and adjusting the target feature region according to the adjustment of the linkage feature region based on the indication of the adjustment rule of the target feature region includes: If the overlapping area between the target feature region and the linkage feature region is less than the area threshold, the position of the vertex in the target feature region is adjusted according to the rigid adjustment method, following the adjustment of the vertex position in the linkage feature region; If the overlapping area between the target feature region and the linkage feature region is greater than or equal to the area threshold, the positions of the vertices in the target feature region are adjusted according to the Laplace method, following the adjustment of the positions of the vertices in the linkage feature region.

9. The method according to any one of claims 1 to 8, characterized in that The step of determining the linkage feature region corresponding to the target feature region from the multiple feature regions includes: If the target feature region belongs to the eyeball appendage region, determining the positional relationship between the target feature region and the eyeball region among the multiple feature regions; Determining the linkage feature region from the eyeball region and the facial feature region according to the positional relationship between the target feature region and the eyeball region; If the distance between the target feature region and the eyeball region is less than a distance threshold, the eyeball region is determined as the linkage feature region; otherwise, the facial feature region is determined as the linkage feature region.

10. The method according to any one of claims 1 to 9, characterized in that: The multiple feature areas include an eyeball area; and the method of determining the eyeball area from the multiple feature areas includes: Determine the center point and sphere similarity of each feature region; the sphere similarity of a feature region is used to indicate the degree to which the corresponding feature region belongs to a sphere; The eyeball area is determined from the multiple feature areas based on the distance between the center point of each feature area and the eyeball key point in the face key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the sphere similarity of each feature area.

11. The method according to any one of claims 1 to 10, characterized in that: The sphere similarity of a feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere; The step of determining the eyeball region from the plurality of feature regions according to the distance between the center point of each feature region and the eyeball key point in the face key points of the adjusted target head model, and the degree of belonging to the sphere indicated by the sphere similarity of each feature region, comprises: Among the feature regions, the feature regions whose ball similarity is less than the similarity threshold are used as candidate eye regions; Determine the distance between the center point of each candidate eyeball region and the eyeball key point in the face key points of the adjusted target head model; The candidate eyeball region indicated by the minimum distance among the candidate eyeball regions is determined as the eyeball region.

12. The method according to any one of claims 1 to 11, characterized in that: The determining of the center point of each feature area and the sphere similarity includes: For a currently traversed feature region among the feature regions, determining a center point of the currently traversed feature region according to positions of each vertex in the currently traversed feature region; According to the positions of the vertices in the currently traversed feature area and the position of the center point, determining the radius of each vertex in the currently traversed feature area from the corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature area; The sphere similarity of the currently traversed feature area is determined according to the difference between the radius of each vertex in the currently traversed feature area from the corresponding center point and the average radius.

13. The method according to any one of claims 1 to 12, characterized in that: The target feature area belongs to the eyelid area; The first model expression in the source head model includes: an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes: an expression indicating that the source head model is in a closed-eye state; The performing expression adjustment processing on the target feature area based on the adjustment rule of the target feature area among the multiple feature areas comprises: Determining a lower eyelid boundary in the adjusted target head model and an upper eyelid boundary in the eyelid region; According to the size parameters and positions of the eyeball regions in the multiple feature regions and based on the Laplace equation, the eyelid region is subjected to collision deformation processing to obtain a deformed eyelid region.

14. The method according to any one of claims 1 to 13, characterized in that: The step of determining a plurality of feature areas constituting the target head model in the target head model comprises: Determining the connected areas of the target head model in the target head model, and determining the overlapping boundaries between the connected areas; According to the determined lengths of the respective overlapping boundaries, the respective connected regions are subjected to region merging processing, and a plurality of connected regions obtained by the region merging are used as the plurality of feature regions.

15. The method according to any one of claims 1 to 14, characterized in that: The target head model is obtained by simplifying the head model region of the interaction model, and the number of vertices in the target head model is less than the number of vertices in the head model region of the interaction model; the method further includes: According to the positions of the texture feature points in the head model area of ​​the interaction model, determining the grid area corresponding to the texture feature points in the target head model, and determining the position mapping relationship between the texture feature points in the head model area of ​​the interaction model and the corresponding grid area in the target head model; Determine the position of the texture feature points in the head model area of ​​the interaction model after position deformation according to the position mapping relationship and the positions of the vertices included in the corresponding grid area; According to the positions of the texture feature points in the head model area of ​​the interaction model after position deformation, the texture feature points in the head model area of ​​the interaction model are deformed to obtain an interaction model with a second model expression.

16. The method according to any one of claims 1 to 15, characterized in that: The expression action instruction includes a target text for instructing to generate a second model expression; the method further includes: Perform speech synthesis processing based on the target text to obtain a target speech corresponding to the target text; After obtaining the target head model with the second model expression, the target head model with the second model expression and the target voice are output to perform interactive processing based on the target head model.

17. A model processing device, characterized in that: include: An acquisition unit, used for acquiring a source head model in response to the acquired expression action instruction; The source head model is configured with neutral feature data and a plurality of expression feature data relative to the neutral feature data; A processing unit, configured to perform deformation matching processing on the source head model according to the model features of the target head model, so as to obtain a deformed and matched source head model; The processing unit is further used to determine the deformation parameters according to the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model; wherein the first model expression in the source head model is determined according to the neutral feature data of the source head model, and the second model expression in the source head model is determined according to the expression feature data indicated by the expression action instruction in the plurality of expression feature data; The processing unit is further used to perform expression migration processing on the target head model according to the deformation parameters to obtain a target head model with a second model expression.

18. A model processing device, characterized in that: The model processing device includes an input interface and an output interface, and also includes: a processor adapted to implement one or more instructions; and, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the model processing method according to any one of claims 1 to 16.

19. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, which, when executed by a processor, are used to execute the model processing method according to any one of claims 1 to 16.

20. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the model processing method according to any one of claims 1 to 16 are implemented.

Images