Model processing method, model processing device, model processing device, and computer program

Abstract

This application discloses a model processing method, apparatus, device, storage medium, and computer program product, the method comprising the steps of: acquiring a source head model in response to an acquired facial expression command; performing deformation matching processing on the source head model based on the model features of a target head model to which facial expression transition is to be performed, and determining deformation parameters based on the source head model after deformation matching and the deformation relationship from a first model expression to a second model expression in the source head model, wherein the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression is determined based on the facial expression feature data indicated by the facial expression command; and performing facial expression transition processing on the target head model based on the deformation parameters to acquire a target head model having the second model expression, thereby enabling automatic transition of the second model expression between models and enabling automatic generation of the second model expression for the target head model.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application filed with the Chinese National Intellectual Property Office on 30 June 2023, application number 202310803347.9, titled "Model Processing Method, Apparatus, Device and Storage Medium," the entire contents of which are incorporated into this application by reference.

[0002] This application relates to the field of artificial intelligence, and more particularly to model processing methods, apparatus, devices and storage media and computer program products. [Background technology]

[0003] 3D animation is currently widely applied in various industries, including computer graphics (CG) films, 3D animation, virtual live streaming, and virtual 3D operators. Among these, the creation of 3D facial animation content remains a significant bottleneck in the industry's development. Traditional techniques typically generate blend shapes (BS) by manually pinching the face, and then construct model expressions based on these blend shapes. However, different 3D head models usually have different facial features, such as different facial contours, different shapes of the five senses, and different distributions of the five senses. Therefore, when it is necessary to construct model expressions for different 3D head models, it is necessary to generate them by manually pinching the face, resulting in a low degree of automation and excessive consumption of human resources. [Overview of the project] [Problems that the invention aims to solve]

[0004] Embodiments of this application provide a model processing method, apparatus, device, storage medium, and computer program product that can automatically realize the transition between second model expressions and automatically generate second model expressions for a target head model. [Means for solving the problem]

[0005] According to one embodiment of this application, an embodiment of the present application is a model processing method, A step of acquiring a source head model in response to an acquired facial movement command, wherein the source head model is configured with neutral feature data and multiple facial feature data corresponding to the neutral feature data. The steps include: performing deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching; A step of determining deformation parameters based on the source head model after deformation matching and the deformation relationship from a first model expression to a second model expression in the source head model, wherein the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial feature data indicated by the facial motion command from among a plurality of facial feature data, The process involves performing an expression transition process on the target head model based on deformation parameters to obtain a target head model with a second model expression, and This provides a method that includes this.

[0006] According to one aspect, the embodiment of the present application is a model processing apparatus, An acquisition unit for acquiring a source head model in response to acquired facial motion commands, wherein the source head model includes an acquisition unit on which neutral feature data and multiple facial feature data corresponding to the neutral feature data are arranged. Includes a processing unit for performing deformation matching on a source head model based on the model features of a target head model, and obtaining the source head model after deformation matching. The processing unit further determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model. The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial expression feature data indicated by the facial expression movement command among multiple facial expression feature data. The processing unit further provides a model processing device that performs facial expression transition processing on the target head model based on deformation parameters to obtain a target head model with a second model facial expression.

[0007] According to one embodiment of the present application, an embodiment provides a model processing device including an input interface and an output interface, the model processing device further includes A processor suitable for executing one or more instructions, A computer storage medium containing one or more instructions suitable for a processor to load and execute the model processing method described above includes:

[0008] According to one embodiment of the present application, an embodiment provides a computer storage medium in which computer program instructions are stored, and the computer program instructions are used to execute the above-described model processing method when executed by a processor.

[0009] According to one embodiment of the present application, an embodiment provides a computer program product including a computer program stored in a computer storage medium, the processor of a model processing device reads the computer program from the computer storage medium, and the processor executes the computer program, thereby causing the model processing device to execute the above-described model processing method.

[0010] In the embodiments of this application, after obtaining an expression motion command, the source head model is deformed and matched to the target head model, so that the source head model after deformation matching approaches the target head model. Furthermore, deformation parameters can be determined based on the source head model after deformation matching and the deformation relationship from the first model expression in the source head model to the second model expression indicated by the expression motion command. In the target head model, an expression transition process is performed based on the deformation parameters to obtain a target head model with the second model expression. The transition of the second model expression from the source head model to the target head model can be automatically realized. In this way, expression transformation can be realized on several target head models (e.g., newly designed head models) based on the expression transformation of several source head models (e.g., standard models). A second model expression can be automatically generated for the target head model. The second model expression in the source head model can be reused on different head models. Furthermore, the efficiency of generating model expressions for different head models can be improved, and the consumption of human resources can be reduced. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a blend shape according to an embodiment of the present application. [Figure 2] This is a flowchart of the model processing method according to the embodiment of this application. [Figure 3] This is a flowchart of another model processing method according to an embodiment of this application. [Figure 4] This is a schematic diagram of a reference facial keypoint in a two-dimensional facial image according to an embodiment of this application. [Figure 5] This is a schematic diagram showing the position coordinates of the target triangle in the source head model and the source head model after deformation matching according to the embodiment of this application. [Figure 6] This is a schematic diagram illustrating the transition of facial expressions to a target head model according to the embodiment of this application. [Figure 7]It is a flowchart of another model processing method according to an embodiment of the present application. [Figure 8] It is a schematic diagram of a feature area according to an embodiment of the present application. [Figure 9] It is a schematic diagram of the positioning and rotation of an eyeball area according to an embodiment of the present application. [Figure 10] It is a schematic diagram of an expression transition process to a target head model according to an embodiment of the present application. [Figure 11] It is a schematic diagram of the determination of an eyelid area according to an embodiment of the present application. [Figure 12] It is a schematic diagram of a lower eyelid boundary and an upper eyelid boundary according to an embodiment of the present application. [Figure 13] It is a schematic diagram of adjusting an eyelid area according to an embodiment of the present application. [Figure 14] It is a schematic diagram of a texture coordinate relationship according to an embodiment of the present application. [Figure 15] It is a schematic diagram of an expression transition to an interaction model according to an embodiment of the present application. [Figure 16] It is a schematic diagram of the effect of an expression transition according to an embodiment of the present application. [Figure 17] It is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 18] It is a schematic diagram of the configuration of a model processing device according to an embodiment of the present application. [Figure 19] It is a schematic diagram of the configuration of a model processing device according to an embodiment of the present application. **Embodiments for Carrying Out the Invention**

[0012] Hereinafter, in order to clearly and completely describe the technical solutions in the embodiments of the present application, they will be described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the protection scope of the present application.

[0013] Computer vision is the science of studying how to make things "visible" to machines. More specifically, it refers to using cameras and computers in place of human eyes to perform machine vision such as recognition and measurement of targets, and further processing the images to make them more suitable for observation by the human eye or transmission to devices. As a scientific field, computer vision aims to study related theories and technologies and build artificial intelligence systems that can acquire information from images and multidimensional data. Large-scale model technology has brought about a significant change in the development of computer vision technology, and pre-trained models in the vision field such as swin-transformer, ViT, V-MOE, and MAE can be fine-tuned and rapidly applied to specific downstream tasks. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / motion recognition, 3D technologies, 3D object reconstruction, virtual reality, augmented reality, synchronous positioning, and map building.

[0014] Based on the computer vision technology described above, the embodiments of this application provide a model processing solution that, in response to an acquired facial expression command, acquires a source head model, performs deformation matching on the source head model based on the model features of the target head model which is the model to be processed for head transition, and acquires a source head model after deformation matching. Model features mainly refer to the physical features of objects included in the head, such as eyes, ears, mouth, and nose, such as the shape and size features of objects such as eyes, ears, mouth, and nose. Furthermore, deformation parameters are determined based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, and facial expression transition processing is performed on the target head model based on the deformation parameters to acquire a target head model with the second model expression. The source head model contains neutral feature data and multiple facial expression feature data for the neutral feature data. The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial expression feature data indicated by the facial expression command from among the multiple facial expression feature data.

[0015] The above model processing solution can be executed by a model processing device, which may be a terminal device or a server, where terminal devices include, but are not limited to, computers, smartphones, tablet computers, laptops, smart home appliances, in-vehicle terminals, smart wearable devices, etc. A server may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms. Furthermore, optionally, the above model processing solution may be executed alone or in cooperation with any electronic device having computing power, and is not limited in the embodiments of this application. Subsequent embodiments of this application will describe model processing devices as examples.

[0016] The various models mentioned in the above model processing solution may be 3D models, and a 3D model may be a set of multiple triangles, where the vertices of the triangles are the vertices of the 3D model. The source head model is a head model in which neutral feature data and multiple facial feature data are arranged on the neutral feature data. Here, the neutral feature data and facial feature data are blend shapes (BS), and each facial feature data is deformed based on the neutral feature data. Various model expressions can be obtained by the neutral feature data or a combination of one or more of the multiple facial feature data. Any model expression can be considered a blend shape, that is, a combination of multiple blend shapes can also be considered a blend shape, and a model expression determined solely based on the neutral feature data can be called a neutral model expression. The first model expression in the source head model mentioned in the above model processing solution is a neutral model expression determined based on the neutral feature data of the source head model. For example, by combining one facial feature data point indicating "opening the mouth" and another facial feature data point indicating "closing the eyes," a single model facial expression representing "opening the mouth and closing the eyes" can be obtained.Referring to Figure 1, which is a schematic diagram of blend shapes according to an embodiment of the present application, the blend shape shown at mark 101 is a "neutral" blend shape, which corresponds to neutral feature data, and the corresponding model expression is a neutral model expression; the blend shape shown at mark 102 is an "open mouth" blend shape, which corresponds to expression feature data indicating "open mouth," and the corresponding model expression is an "open mouth" model expression; the blend shape shown at mark 103 is a "bite lip" blend shape, which corresponds to expression feature data indicating "bite lip," and the corresponding model expression is a "bite lip" model expression; and the blend shape shown at mark 104 is a "purse lip" blend shape, which corresponds to expression feature data indicating "purse lip," and the corresponding model expression is a "purse lip" model expression.

[0017] The collection and processing of relevant data in this application (e.g., source head model, target head model, etc.) must strictly comply with the requirements of applicable laws and regulations in the applicable example, obtain informed consent or individual consent from the data subject, and carry out subsequent data use and processing within the scope permitted by law and the data subject. Furthermore, if the facial (or other biometric authentication) related technology included in the relevant embodiments of this application is applied to a specific product or technology, the collection, use, and processing processes of relevant data must comply with the requirements of law, the data processing rules must be notified and individual consent obtained from the subject before collecting facial information, the facial information must be processed in strict compliance with the requirements of law and personal data processing rules, and technical measures must be taken to ensure the security of the relevant data.

[0018] Based on the model processing solution described above, embodiments of this application provide a model processing method. Referring to Figure 2, a flowchart of the model processing method according to embodiments of this application is shown. The model processing method shown in Figure 2 is executed by the model processing device described above. The model processing method shown in Figure 2 may include the following steps.

[0019] S201 acquires the source head model in response to the acquired facial expression command.

[0020] The source head model contains neutral feature data and multiple facial feature data for the neutral feature data. Furthermore, the facial motion command is used to instruct the model processing device to generate a desired model facial expression in the source head model. The command format of the facial motion command can be set according to specific needs and may be, for example, a text format or an audio format, and the embodiments of this application are not limited thereto. The model processing device can generate a model facial expression in the source head model that the facial motion command expects by performing a command analysis process on the facial motion command. In an executable embodiment, if the facial motion command is in text format, the model processing device can generate a model facial expression in the source head model indicated by the corresponding facial keyword if it detects that the facial motion command contains a facial keyword. For example, if it detects that the facial motion command contains the facial keyword "open mouth," it can generate a model facial expression in the source head model indicating "open mouth." In another executable embodiment, if the facial expression command is in text format, the model processing device can perform semantic analysis on the facial expression command and generate a semantically matching model expression for the source head model. For example, if the facial expression command is analyzed to contain the meaning of "surprise," the model processing device can generate a semantically matching model expression for the source head model indicating "open mouth." Neutral feature data may be data of the position of each face grid vertex in the case of neutral, and a neutral face can be rendered based on the position of the neutral face grid vertices, which may be, for example, a face without expression. Similarly, facial expression feature data may be data of the position of each face grid vertex in a particular specific facial expression, and a face of a particular facial expression, for example, a smiling facial expression feature data, can be rendered based on the facial expression feature data, and a smile can be rendered based on the position of each face grid vertex in the corresponding smiling facial expression.Naturally, the neutral feature data and facial feature data may include table data in addition to the positions of the corresponding face grid vertices, and this table data may store a single face consisting of multiple vertices. For example, the table data may store data for the vertices that make up a single triangular face.

[0021] S202 performs deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching. The target head model is the model to be used for facial expression transitions. The model features of the source head model and the target head model are different, and the model features may be, for example, the facial features of the head model, and the facial features may include features such as the five senses, face shape, jaw, cheekbones, etc. Alternatively, the model features may be used to distinguish between different head models.

[0022] In one embodiment, the target head model may be a different head model from the source head model. For example, the shape of the face of the target head model may be different from that of the source head model, the shape of the five sense organs of the target head model may be different from that of the source head model, and the distribution of the five sense organs of the target head model may be different from that of the source head model. The model processing device performs a deformation matching process on the source head model based on the model features of the target head model in order to deform and match the source head model to the target head model, that is, to adjust the source head model to be closer to the target head model. For example, one or more features of the source head model, such as the shape of the face, the shape of the five sense organs, and the distribution of the five sense organs, may be brought closer to the corresponding features of the target head model.

[0023] S203 determines the deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model.

[0024] The first model expression in the source head model is determined based on the neutral feature data of the source head model; that is, the first model expression in the source head model is the neutral model expression. The second model expression in the source head model is determined based on the facial feature data indicated by the facial motion command from among multiple facial feature data. In other words, the second model expression in the source head model is a model expression generated by the model processing device based on the facial motion command. The model processing device performs command analysis processing on the facial motion command to generate the model expression expected by the facial motion command in the source head model. First, it performs command analysis processing on the facial motion command to determine the facial feature data indicated by the facial motion command, and then it can generate the desired model expression (i.e., the second model expression) based on the determined facial feature data. The second model expression is an expression determined from multiple existing facial feature data according to the facial motion command. In an executable embodiment, if the facial expression command is in text format, the model processing device, upon detecting that the facial expression command contains facial expression keywords, determines the facial expression feature data indicated by the facial expression command based on those keywords, and further generates a corresponding model expression based on the determined facial expression feature data. It is understood that the above model processing device generating a model expression expected by the facial expression command for the source head model by performing command analysis processing on the facial expression command is merely one example. For example, if the facial expression command contains identification information for facial expression feature data, the model processing device can directly determine the facial expression feature data indicated by the facial expression command based on the identification information contained in the facial expression command, and further generate a corresponding model expression.

[0025] The deformation relationship between the source head model and the deformation-matched source head model includes a transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the deformation-matched source head model, where the transformation matrix between vertices is determined based on affine transformation parameters and translation parameters. For details on the calculation of this deformation relationship, please refer to the relevant descriptions in subsequent embodiments.

[0026] S204, In the target head model, an expression transition process is performed based on the deformation parameters to obtain a target head model with a second model expression.

[0027] In one embodiment, performing facial expression transition processing based on the deformation parameters may include adjusting the position of feature points in a target head model having a first model facial expression based on the deformation parameters to obtain a target head model that realizes a second model facial expression. When the model processing device performs facial expression transition processing on a target head model based on the deformation parameters, it performs facial expression transition processing on a target head model having a first model facial expression based on the deformation parameters, that is, it performs facial expression transition processing on a target head model having a neutral model facial expression based on the deformation parameters. The first model facial expression in the target head model is one that fits the target head model, that is, it is determined based on the neutral feature data of the target head model, and not based on the neutral feature data of the source head model. When a model processing device performs facial expression transition processing on a target head model based on deformation parameters, it can adjust the positions of vertices in the target head model having a first model expression based on the deformation parameters to realize a second model expression based on the target head model. For example, if the second model expression in the source head model is a model expression indicating "opening the mouth," the device can adjust the positions of vertices in the target head model having the first model expression based on the deformation parameters to realize a model expression indicating "opening the mouth" based on the target head model.

[0028] In the embodiments of this application, after acquiring an expression movement command, the process starts with a source head model equipped with neutral feature data and expression feature data, and transitions to a target head model through expression transformation in the source head model, thereby realizing an expression change for the target head model. To achieve the objective of expression transition, first, the source head model is deformed and matched to the target head model so that the source head model after deformation matching becomes closer to the target head model, and the source head model equipped with neutral feature data and expression feature data becomes closer to the shape of the target head model. Next, deformation parameters are determined based on the deformation relationship between the source head model after deformation matching and the first model expression in the source head model to the second model expression indicated by the expression movement command, and based on these deformation parameters, the target head model with the current expression can be transformed into a target head model with the second model expression. The system can automatically transition the second model expression from the source head model to the target head model, meaning it can automatically generate the second model expression for the target head model, allowing the second model expression from the source head model to be reused in different head models. Furthermore, it can improve the efficiency of generating model expressions for different head models, thereby reducing the consumption of human resources.

[0029] Based on the relevant embodiments of the model processing method described above, embodiments of this application provide another model processing method. Referring to Figure 3, a flowchart of the other model processing method according to embodiments of this application is shown. The model processing method shown in Figure 3 is performed by the model processing device described above. The model processing method shown in Figure 3 may include the following steps.

[0030] S301: In response to the acquired facial expression command, the source head model is acquired.

[0031] The source head model contains neutral feature data and multiple facial expression feature data for the neutral feature data. The process in step S301 is similar to the process in step S201 described above and will not be explained again here.

[0032] S302 performs deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching.

[0033] In one embodiment, the model processing device deforms and matches the source head model to the target head model, that is, adjusts the source head model to be closer to the target head model by performing deformation matching on the source head model based on the model features of the target head model. Adjusting the source head model to be closer to the target head model is understood to involve roughly aligning the source head model using stiffness translation, rotation and scaling, utilizing the spatial positions of the same-named key points in the source head model and the target head model, so that the initial shapes of the two models are similar, that is, the two models are roughly closer, for example, the faces of the source head model and the target head model are both round faces, the facial boundary positions are close together, and so that their size and shape are similar. Next, the facial details in the source head model are adjusted using the planar distance between model vertices, the Euclidean distance between identically named keypoints in the two models, and the degree of grid deformation as indicators. By adjusting the keypoints in the source head model, the difference (i.e., Euclidean distance) between the coordinate position of each keypoint in the adjusted source head model and the coordinate position of each corresponding keypoint (keypoint with the same name) in the target head model satisfies the similarity condition (for example, if the Euclidean distance is smaller than the threshold, the similarity condition is considered to be satisfied). At the same time, the deformation of the vertex grid of the source head model is minimized, the distance between the vertices of the deformed source head model and the plane of the target head model is minimized, and the optimal solution under the three constraints is obtained. When the optimal solution is reached, the adjusted source head model is considered to be close to the target head model.

[0034] In one embodiment, based on the positions of the vertices and keypoints in the source head model and the vertices and keypoints in the target head model, a candidate source head model is calculated in which the positions of each vertex and keypoint are changed without changing the positions of the vertices and keypoints in the target head model. Based on the planar distance of the vertices of the candidate source head model, the Euclidean distance between keypoints with the same name in the candidate source head model and the target head model, and the degree of grid deformation of the candidate source head model, a weight calculation is performed according to the corresponding weight values ​​set in advance. The candidate source head model that yields the minimum weight is determined as the target source head model. This target source head model is a source head model close to the target head model, and a positional transformation is performed on the source head model based on the positions of each vertex and keypoint in the target source head model to obtain the source head model after deformation matching, thereby completing the deformation matching for the source head model. In other words, by correcting the position values ​​of vertices and keypoints in the source head model and using the corrected values ​​in the weight calculation, the position values ​​of each vertex and keypoint when the minimum value is obtained are the values ​​of each vertex and keypoint in the target source head model. By adjusting the position relative to the source head model according to the values ​​of each vertex and keypoint in the target source head model, deformation matching of the source head model can be completed, and a source head model close to the target head model can be obtained. Note that the planar distance of the vertices of the candidate source head model is the distance from the vertex in the candidate source head model to the plane corresponding to the target head model. Calculating the planar distance corresponding to the vertices in the source head model is necessary because the closest vertex in the target head model to a vertex in the source head model is likely to be a single point on a plane composed of points, rather than a specific point in the target head model.Keypoints with the same name in the candidate source head model and the target head model are, for example, keypoints with the same number in both the candidate source head model and the target head model, such as the keypoint number at the outer corner of the eye and the keypoint number at the tip of the nose. The degree of grid deformation in the candidate head model can be measured in the candidate source head model based on the difference between the rotation portion and the identity matrix in the transformation matrix before and after deformation of the triangular faces.

[0035] In one embodiment, face keypoints may be points that can be used to indicate facial features, such as the tip of the nose, eyeballs, facial contours, eyebrows, eyelids, etc., and the model processing device determining face keypoints in a target head model may include the steps of performing face detection processing on the target head model to obtain detected keypoints corresponding to the target head model, performing keypoint labeling processing on the detected keypoints corresponding to the target head model using reference face keypoint distribution information, and making the labeled detected keypoints the face keypoints of the target head model.Optionally, performing face detection processing on a target head model may be performed by a face detection model based on a CNN (Convolutional Neural Network), specifically, face keypoints can be detected by the face detection model, and other neural network models capable of detecting face keypoints can also be used in this process, and the embodiments of this application are not limited thereto. Furthermore, optionally, the reference face keypoint distribution information can be set according to specific needs, and may be distribution information of reference face keypoints in a 2D face image. The reference face keypoints in the 2D face image can be set according to specific needs, for example, the number of reference face keypoints in the 2D face image, their distribution positions, etc. Generally, the number of reference face keypoints in the 2D face image can be set to 82, 76, etc. For example, referring to Figure 4, which is a schematic diagram of reference face keypoints in a 2D face image according to an embodiment of this application, it may include reference face keypoints that can show facial features such as the tip of the nose, eyeballs, facial contours, eyebrows, and eyelids. Furthermore, the target head model that performs face detection processing may be obtained by rendering, and if there is a color texture or point coordinates, color rendering is used, which makes the lighting more natural.

[0036] In one embodiment, the model processing device performs deformation matching on a source head model based on the model features of a target head model to obtain a source head model after deformation matching. This step includes determining a matching relationship between each feature point in the source head model and each feature point in the target head model based on the position of each feature point in the source head model and the position of each feature point in the target head model, wherein the feature point can include one or both of vertices and face keypoints in the corresponding model; and obtaining a source head model after deformation matching by adjusting the positions of the vertices constituting the source head model, using the position of the feature point in the target head model as a reference position, based on the matching relationship, provided that the distance between the position of the feature point in the source head model after deformation matching and the corresponding reference position determined based on the matching relationship satisfies the proximity condition. In other words, the model processing device obtains a source head model after deformation matching by adjusting the positions of the vertices constituting the source head model, using the position of the feature point in the target head model as a reference position, with the deformation objective being to reduce the distance between the feature points. The feature point position distance is the distance between the position of a feature point in the source head model and the reference position corresponding to the feature point in the target head model that is in a matching relationship with that feature point. The positions of the vertices constituting the source head model are adjusted so that the distance between the position of the feature point in the source head model after deformation matching and the corresponding reference position determined based on the matching relationship satisfies the proximity condition, and this proximity condition can be set according to specific needs.

[0037] Since the source head model and the target head model are two different models, the number of vertices in the source head model and the position of each vertex may differ from those in the target head model. Vertices in the source head model and vertices in the target head model may have various correspondences, such as one-to-one, one-to-many, many-to-one, or many-to-many. For example, if the number of vertices in the eyebrow region of the source head model is greater than the number of vertices in the eyebrow region of the target head model, then multiple vertices in the eyebrow region of the source head model may correspond to a single vertex in the eyebrow region of the target head model. Therefore, if feature points are vertices, the model processing device can determine the matching relationship between each vertex in the source head model and each vertex in the target head model based on the position of each vertex in the source head model and the position of each vertex in the target head model, thereby determining the matching relationship between vertices, i.e., the correspondence between vertices. If the feature points are facial keypoints, the model processing device can determine the matching relationship between each facial keypoint in the source head model and each facial keypoint in the target head model based on the position of each facial keypoint in the source head model and the position of each facial keypoint in the target head model, thereby determining the matching relationship between facial keypoints, i.e., the correspondence relationship between facial keypoints. For example, it can determine that there is a matching relationship between the facial keypoint indicating the tip of the nose in the source head model and the facial keypoint indicating the tip of the nose in the target head model. The process for determining the facial keypoints in the source head model is similar to the process for determining the facial keypoints in the target head model described above, so it will not be explained again here.

[0038] Furthermore, the model processing device obtains a source head model after deformation matching by adjusting the positions of the vertices constituting the source head model, using the positions of the feature points in the target head model as reference positions based on the matching relationship. The distance between the positions of the feature points in the source head model after deformation matching and the corresponding reference positions determined based on the matching relationship satisfies the proximity condition. For example, the proximity condition may be set such that the average distance between the position of each feature point in the source head model after deformation matching and the corresponding reference position determined based on the matching relationship is less than a first threshold, or the proximity condition may be set such that the sum of the distances between the positions of each feature point in the source head model after deformation matching and the corresponding reference positions determined based on the matching relationship is less than a second threshold, and the first and second thresholds can be set according to specific needs.

[0039] Furthermore, if feature points simultaneously use vertices and face keypoints (optional), the model processing device first adjusts the positions of vertices constituting the source head model based on the matching relationships between face keypoints to obtain an intermediate source head model, and then further adjusts the positions of vertices in the intermediate source head model based on the matching relationships between vertices to obtain a source head model after deformation matching. Note that when adjusting the positions of vertices constituting the source head model based on the matching relationships between face keypoints, the distance between face keypoints can be used as an indicator. That is, the position of the feature points (i.e., face keypoints) in the target head model is used as the reference position, and the deformation objective is to reduce the distance between the feature point positions based on the face keypoints. This process can be achieved by performing stiffness translation, rotation, and scaling on the source head model. This process is expected to ensure that the initial model states of the intermediate source head model and the target head model match, for example, that the orientation and size of the head match. The position of vertices in the intermediate source head model is then further adjusted based on the matching relationships between vertices. In this case, the distance between vertices can be used as an indicator, that is, with the position of a feature point (i.e., a vertex) in the target head model as the reference position, the deformation objective is to reduce the distance of the feature point position when based on the vertex. Furthermore, optionally, the distance between keypoints and the distance between vertices can be used together as indicators, that is, with the position of a feature point (i.e., vertices and face keypoints) in the target head model as the reference position, the deformation objective is to reduce the distance of the feature point position when based on the vertex and the distance of the feature point position when based on the face keypoint. Furthermore, optionally, the distance between vertices can be the distance between vertices in a matching relationship (i.e., the distance of the feature point position when based on the vertex), or the distance between a vertex point and a face may be used.

[0040] S303, the deformation relationship between the source head model and the source head model after deformation matching is determined based on the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model, where the feature point includes one or more of the vertices and face keypoints.

[0041] In one embodiment, the model processing device determines the deformation relationship between the source head model and the source head model after deformation matching, based on the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model, in order to determine the vertex deformation relationship between each vertex in the source head model and the corresponding vertex in the source head model after deformation matching, i.e., the transformation matrix (also called the deformation field) from the vertex in the source head model to the corresponding vertex in the source head model after deformation matching. To illustrate with an example, the target triangle and its vertices in the source head model, the target triangle may be any of the triangles in the source head model. For example, referring to Figure 5, which is a schematic diagram of the position coordinates of the target triangle in the source head model after deformation matching and the source head model according to the embodiment of this application, the position coordinates of the three vertices of the target triangle in the source head model are represented as the position coordinates V1 of the first vertex, V2 of the second vertex, and V3 of the third vertex, respectively, and the normal vector of the target triangle is represented by V4. In the deformation-matched source head model, the position coordinates of the three vertices of the target triangle are represented as the position coordinates V'1 of the first vertex, V'2 of the second vertex, and V'3 of the third vertex, respectively, and the normal vector of the target triangle is represented by V'4. First, the model processing device can determine the normal vector of the target triangle based on the position of each vertex of the target triangle in the source head model, and further determine the local coordinate system of the target triangle based on the position of each vertex and the normal vector of the target triangle in the source head model.

[0042] Furthermore, the normal vector of the target triangle in the source head model is given by the following equation 1.1.

number

number

[0043] Furthermore, the model processing device can determine the normal vector of a target triangle based on the position of each vertex in the source head model after deformation matching, and can also determine the local coordinate system of the target triangle based on the position of each vertex and the normal vector in the source head model after deformation matching.

[0044] The normal vector of the target triangle in the source head model after deformation matching is given by the following equation 2.1.

number

number

[0045] By defining the spatial affine transformation parameter Q and the translation parameter t, and applying them to the three vertices of the target triangle in the source head model, the position coordinates of the three vertices of the target triangle in the source head model after deformation matching are obtained. The correspondence between the position coordinates of the vertices of the target triangle in the source head model and the corresponding vertices of the target triangle in the source head model after deformation matching is shown by the following equation 3.

number

[0046] Furthermore, the model processing device can determine affine transformation parameters based on the local coordinate system of the target triangle in the source head model and the local coordinate system of the target triangle in the source head model after deformation matching, and determine translation parameters based on the position of each vertex of the target triangle in the source head model and the position of each vertex of the target triangle in the source head model after deformation matching. The affine transformation parameter Q is given by the following equation 4, and the translation parameter t is given by the following equation 5.

number

[0047] Furthermore, the model processing device can determine, based on the determined affine transformation parameters and translation parameters, the vertex deformation relationship between each vertex of the target triangle in the source head model and the corresponding vertex in the source head model after deformation matching, i.e., the transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the source head model after deformation matching. If the target triangle is the i-th triangle among multiple triangles in the source head model, the transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertex in the source head model after deformation matching is given by the following equation 6, and this transformation matrix includes the vertex deformation relationship of the three vertices in the i-th triangle.

number

[0048] Furthermore, Q irepresents the local coordinate system of the i-th triangle in the source head model, and the affine transformation parameters corresponding to the i-th triangle determined based on the local coordinate system of the i-th triangle in the source head model after deformation matching, and t i This represents the translation parameter corresponding to the i-th triangle, determined based on the positions of each vertex of the i-th triangle in the source head model and the positions of each vertex of the i-th triangle in the source head model after deformation matching, and T i ´´ This represents the transformation matrix from the vertex of the i-th triangle in the source head model to the corresponding vertex in the source head model after deformation matching, i.e., the transformation matrix from the i-th triangle in the source head model to the i-th triangle in the source head model after deformation matching.

[0049] S304. Based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, the deformation parameters are determined.

[0050] The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial feature data indicated by the facial action command from among multiple facial feature data. In other words, the first model expression can be rendered based on the position of each vertex included in the neutral feature data, and this first model expression may be, for example, a human face without a normal expression. Multiple facial feature data with normal expressions may be associated with the source head model, for example, facial feature data for a smile and facial feature data for a grin, and based on the facial action command, facial feature data with a normal expression can be determined from there, and by rendering these facial feature data, the corresponding expression can be obtained, for example, if the facial action command indicates a smile, the second model expression will be a smile.

[0051] Furthermore, the deformation relationship from the first model expression to the second model expression in the source head model can be determined based on the position of each feature point in the source head model with the second model expression and the position of the corresponding feature point in the source head model with the first model expression. Specifically, this is the vertex deformation relationship between each vertex in the source head model with the first model expression and the corresponding vertex in the source head model with the second model expression, i.e., the transformation matrix (also called the deformation field) from the vertex in the source head model with the first model expression to the corresponding vertex in the source head model with the second model expression. This process is similar to the process for determining the deformation relationship between the source head model and the source head model after deformation matching, so it will not be explained again here.

[0052] In one embodiment, the deformation parameters may include vertex deformation parameters of vertices in the target head model, and taking the determination of vertex deformation parameters of target vertices in the target head model as an example, the target vertex may be any vertex in the target head model, in which case the model processing device determining the deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from a first model expression to a second model expression in the source head model may include the steps of: determining a first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching, based on the deformation relationship between the source head model and the source head model after deformation matching; determining a second vertex deformation relationship between each reference vertex in the source head model having a first model expression and the corresponding vertex in the source head model having a second model expression, based on the deformation relationship from a first model expression to a second model expression in the source head model; and determining the vertex deformation parameters of the corresponding target vertex in the target head model for each reference vertex based on the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex. If the target vertex is the i-th vertex in the target head model, i ∈ [1, number of vertices in the target head model], then the vertex deformation parameters of the target vertex in the target head model are given by the following equations 7.1 and 7.2.

number

[0053] N(i) represents a set of reference triangle indices consisting of the indices of each reference triangle in the source head model that correspond to the target vertex (i.e., the i-th vertex) in the target head model, where each reference vertex in the source head model that corresponds to the target vertex (i.e., the i-th vertex) is a vertex of each reference triangle, and optionally, a reference triangle may be the closest triangle in the source head model that corresponds to the i-th vertex in the target head model.

[0054] T j '' represents the transformation matrix from the vertex of the j-th triangle in the source head model to the corresponding vertex in the source head model after deformation matching, i.e., the transformation matrix from the j-th triangle in the source head model to the j-th triangle in the source head model after deformation matching, T j ' represents the transformation matrix from the j-th triangle vertex in the source head model with the first model expression to the corresponding vertex in the source head model with the second model expression, i.e., the transformation matrix from the j-th triangle in the source head model with the first model expression to the j-th triangle in the source head model with the second model expression, and w i,j This represents the proximity weight where the i-th vertex in the target head model corresponds to the j-th triangular face in the source head model, and this can be set according to specific needs.

[0055] S305, In the target head model, an expression transition process is performed based on the deformation parameters to obtain a target head model with a second model expression.

[0056] In an executable embodiment, the step in which the model processing device performs expression transition processing on the target head model based on the deformation parameters to obtain a target head model with a second model expression may include the step of adjusting the positions of corresponding vertices in the target head model with the first model expression based on the vertex deformation parameters of each vertex in the target head model to obtain a target head model with a second model expression. Taking the target vertex in the target head model (with the first model expression) as the i-th vertex as an example, the process of adjusting the position of the target vertex based on the vertex deformation parameter of the target vertex in the target head model may be represented by the following mathematical formula 8.

Number

[0057] Referring to Figure 6, which is a schematic diagram of the facial expression transition to the target head model according to the embodiment of this application, the model processing device acquires the source head model and the target head model, and then performs deformation matching on the source head model (which has the first model expression, i.e., the neutral model expression) based on the model features of the target head model (which has the first model expression, i.e., the neutral model expression), thereby acquiring the source head model after deformation matching, and the source head model after deformation matching becomes closer to the target head model. Furthermore, the model processing device determines the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, and determines deformation parameters based on these two determined deformation relationships, and performs facial expression transition processing on the target head model based on the deformation parameters, thereby acquiring the target head model with the second model expression, thereby realizing the transition of the second model expression of the target head model, i.e., the second model expression, from the source head model to the target head model. Furthermore, in the source head model, the index of each reference triangle corresponding to a target vertex in the target head model can be represented by a proximity mapping corresponding to the target vertex. Based on this, the model processing device can obtain a proximity mapping table corresponding to the corresponding vertex in the process of determining the vertex deformation parameters of the vertex in the target head model, determine the vertex deformation parameters of the corresponding vertex based on the proximity mapping table and these two deformation relationships, and adjust the corresponding vertex based on the vertex deformation parameters.

[0058] In the embodiments of this application, the transition of a second model expression from a source head model to a target head model can be automatically realized, that is, a second model expression can be automatically generated for the target head model, the second model expression in the source head model can be reused for different head models, further improving the efficiency of generating model expressions for different head models and reducing the consumption of human resources. Furthermore, in the process of determining the deformation relationship, the deformation relationship between the source head model and the source head model after deformation matching is determined based on the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model. Furthermore, deformation parameters can be determined based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model. By determining the deformation relationship and deformation parameters using feature points in the model as analysis objects, the accuracy and efficiency of model deformation can be improved. The complete expression deformation parameters can be determined based on the deformation relationship from the first model expression to the second model expression in the source head model, and the source head model and target head model can be analyzed. Because each model has different facial features, the determined deformation parameters may not fit the target head model, potentially worsening the transition effect to the second model expression in the target head model. In contrast, this solution, on the one hand, brings the source head model closer to the target head model after deformation matching, and controls the deformation of the target head model using the deformation relationship corresponding to the facial expression change in the source head model. On the other hand, it realizes facial expression transformation in the target head model based on the deformation relationship of feature points rather than the deformation relationship of pixel points, thereby improving the adaptation efficiency between the determined deformation parameters and the target head model, and further improving the effect of performing facial expression transitions based on deformation parameters, i.e., improving the transition effect to the second model expression in the target head model.

[0059] Based on the relevant embodiments of the model processing method described above, embodiments of this application provide another model processing method. Referring to Figure 7, a flowchart of the other model processing method according to embodiments of this application is shown. The model processing method shown in Figure 7 is performed by the model processing device described above. The model processing method shown in Figure 7 may include the following steps.

[0060] S701 acquires the source head model in response to the acquired facial expression command.

[0061] The source head model contains neutral feature data and multiple facial expression feature data corresponding to the neutral feature data.

[0062] S702 performs deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching.

[0063] S703 determines the deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model.

[0064] The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial feature data indicated by the facial motion command from among multiple facial feature data. The deformation parameters include the vertex deformation parameters of the vertices of the facial feature region in the target head model. The related processes from steps S701 to S703 are similar to the related processes from steps S201 to S203 and from steps S301 to S304 described above, so they will not be explained again here.

[0065] S704 determines multiple feature regions that constitute the target head model.

[0066] In one embodiment, the step of a model processing device determining a plurality of feature regions constituting the target head model may include the steps of determining the connected regions (i.e., communication regions) of the target head model and determining the weight boundaries between each connected region, and performing a region merge process on each connected region based on the length of each determined weight boundary, thereby making the plurality of connected regions obtained by the region merge a plurality of feature regions. Optionally, when the model processing device performs a region merge process on each connected region based on the length of each determined weight boundary, it may perform a region merge process on the corresponding connected region if the length of the determined weight boundary is greater than a length threshold, and if the ratio of the length of the determined weight boundary to the boundary length of the corresponding connected region is greater than a weight ratio threshold, it may perform a region merge process on the corresponding connected region, and the length threshold and weight ratio threshold can be set according to specific needs. Referring to Figure 8, which is a schematic diagram of feature regions according to an embodiment of the present application, an example of a feature region is shown as mark 801.

[0067] S705 adjusts the position of the corresponding vertices in the target head model with the first model expression based on the vertex deformation parameters of each vertex in the facial feature region of the target head model, and obtains the adjusted target head model.

[0068] The facial feature region is the feature region with the largest surface area among multiple feature regions. The reason for determining the feature region with the largest surface area among multiple feature regions as the facial feature region is that the connected region consisting of the face and head has the largest actual surface area, but not the largest number of vertices. The related process by which the model processing device adjusts the position of the corresponding vertices in the target head model having the first model expression based on the vertex deformation parameters of each vertex in the facial feature region of the target head model is similar to the related process in step S305 above, which adjusts the position of the corresponding vertices in the target head model having the first model expression based on the vertex deformation parameters of each vertex in the target head model, and therefore will not be explained again here. This process is a pinching process, which can change the position of the vertices of the facial feature region in the target head model.

[0069] S706 uses the adjusted target head model as a base and performs facial expression adjustment processing on the target feature regions based on adjustment rules for the target feature regions in multiple feature regions to obtain a second target head model with a second model facial expression.

[0070] In one embodiment, the target feature region may be any feature region other than the face feature region among a plurality of feature regions, for example, a feature region representing eyebrows, a feature region representing eyelashes, a feature region representing the tongue, etc. When a model processing device performs facial expression adjustment processing on the target feature region based on an adjustment rule for the target feature region among a plurality of feature regions, the steps of determining a linked feature region corresponding to the target feature region from the plurality of feature regions and performing follow-up adjustment processing on the target feature region in accordance with the adjustment of the linked feature region based on the instructions of the adjustment rule for the target feature region, in other words, facial expression adjustment processing can be performed on the target feature region by making it follow the adjustment of the linked feature region based on the adjustment rule for the target feature region.

[0071] Furthermore, the linked feature region corresponding to the target feature region may be any of several feature regions that can be used to guide the facial expression adjustment process on the target feature region, and generally, the face feature region can be determined as the linked feature region corresponding to the target feature region. In addition, the step by which the model processing device performs follow-up adjustment on the target feature region in accordance with the adjustment of the linked feature region, based on the instructions of the adjustment rules for the target feature region, may include: if the overlapping area between the target feature region and the linked feature region is smaller than the area threshold, adjusting the position of the vertices in the target feature region in accordance with the adjustment of the vertex positions in the linked feature region according to the stiffness adjustment method; and if the overlapping area between the target feature region and the linked feature region is greater than or equal to the area threshold, adjusting the position of the vertices in the target feature region in accordance with the adjustment of the vertex positions in the linked feature region according to the Laplace method. The area threshold can be set according to specific needs, and the stiffness adjustment method includes stiffness rotation and translation. For example, if the target feature region is a feature region representing eyebrows, and the linked feature region corresponding to that eyebrow feature region is a face feature region, and the overlapping area between the eyebrow feature region and the face feature region is smaller than the area threshold, then, according to the rigidity adjustment method, the position of the vertices in the eyebrow feature region is adjusted to follow the adjustment of the vertex positions in the face feature region. After determining the target feature region for which facial expression transformation needs to be completed, this application determines the feature region linked to the target feature region accordingly. In this way, when facial expression transformation adjustment is performed in the target region, the linked feature region also moves accordingly, resulting in a more natural facial expression transformation.

[0072] In an executable embodiment, when adjusting the position of vertices in a target feature region according to a stiffness adjustment method, in accordance with the adjustment of vertex positions in a linked feature region, stiffness adjustment parameters are determined based on the position of vertices in the linked feature region before adjustment and the position of vertices in the linked feature region after adjustment. Stiffness adjustment processing is performed on the position of vertices in the target feature region based on the stiffness adjustment parameters. If the linked feature region is a face feature region, the position of vertices in the linked feature region before adjustment is the position of vertices in the face feature region of the target head model, and the position of vertices in the linked feature region after adjustment is the position of vertices in the face feature region of the target head model after adjustment. When determining stiffness adjustment parameters based on the positions of vertices in the linked feature region before and after adjustment, the center points of the first and second point cloud matrices are determined by the first point cloud matrix consisting of the positions of vertices in the linked feature region before adjustment and the second point cloud matrix consisting of the positions of vertices in the linked feature region after adjustment. Based on the first point cloud matrix and the corresponding center points, a decentered first point cloud matrix can be determined, and based on the second point cloud matrix and the corresponding center points, a decentered second point cloud matrix can be determined. Furthermore, by performing singular value decomposition (SVD) on the decentered first and second point cloud matrices, rotation matrices and translation parameters can be obtained, and the rotation matrices and translation parameters obtained through decomposition constitute the stiffness adjustment parameters.

[0073] Optionally, taking the example that the linked feature region is the face feature region, the face feature region before adjustment can be considered as a grid component (i.e., mesh component) corresponding to a neutral model expression, and the face feature region after adjustment can be considered as a grid component corresponding when it has a specific non-neutral model expression. The first point cloud matrix consisting of the positions of the vertices of the linked feature region before adjustment is shown by the following equation 9.1, and the second point cloud matrix consisting of the positions of the vertices of the linked feature region after adjustment is shown by the following equation 9.2, where, when counting the vertices in the linked feature region from 0, [x0y0z0] represents the position coordinates (i.e., x, y, and z axis coordinates) of the vertices that were counted as 0 before adjusting the linked feature region, and [x n y n z n ] represents the position coordinates (i.e., x, y, and z axis coordinates) of the vertices counted as n before adjusting the linked feature region. When counting vertices in the linked feature region from 0, n is the number of vertices in the linked feature region minus 1, i.e., n+1 represents the number of vertices in the linked feature region. [x´0y´0z´0] represents the position coordinates (i.e., x, y, and z axis coordinates) of the vertices counted as 0 after adjusting the linked feature region, i.e., [x´ n y' n z' n ] represents the position coordinates (i.e., x, y, and z axis coordinates) of the vertices counted as n after adjusting the linked feature region.

number

[0074] The center of the first point cloud matrix is ​​given by equation 9.3 below, and the center of the second point cloud matrix is ​​given by equation 9.4 below.

number

number

number

[0075] In one embodiment, the linked feature region corresponding to the target feature region represents a feature region that can be used to guide the target feature region to perform facial expression adjustment processing, and generally, the face feature region can be determined as the linked feature region corresponding to the target feature region. Furthermore, if the target feature region belongs to the eyeball appendage region, the linked feature region corresponding to the target feature region may be the eyeball region determined from multiple feature regions (i.e., the eyeball region may be included in multiple feature regions). Therefore, the model processing device needs to determine the eyeball region and the eyeball appendage region from multiple feature regions. The eyeball appendage region is a feature region that indicates eyeball appendages, and eyeball appendages may include structural components related to the eyeball, such as eyeball contents, eyelids, eyelashes, and eyebrows.

[0076] The method by which a model processing device determines the eyeball region from multiple feature regions includes the steps of determining the center point and spherical similarity of each feature region, and determining the eyeball region from multiple feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity of each feature region. Spherical similarity (also called sphereness) represents the degree to which an object belongs to a sphere, and the spherical similarity of one feature region represents the degree to which the corresponding feature region belongs to a sphere.

[0077] The steps by which a model processing device determines the center point and spherical similarity of each feature region may include: determining the center point of the currently traversed feature region based on the position of each vertex in the currently traversed feature region; determining the radius of each vertex in the currently traversed feature region from its corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature region, based on the position of each vertex and the center point in the currently traversed feature region; determining the spherical similarity of the currently traversed feature region based on the difference between the radius of each vertex in the currently traversed feature region from its corresponding center point and the average radius; and further determining the average of the absolute differences between the radius of each vertex in the currently traversed feature region from its corresponding center point and the average radius as the spherical similarity of the currently traversed feature region.

[0078] Furthermore, when the model processing device determines the center point of the feature region currently being traversed based on the position of each vertex in the feature region currently being traversed, it can perform an averaging process on the position of each vertex in the feature region currently being traversed, and use the position obtained by the averaging process as the position of the center point of the feature region currently being traversed. The position of the center point of the feature region currently being traversed is shown by the following equations 10.1, 10.2, and 10.3, where N represents the number of vertices in the feature region currently being traversed, and i is an independent variable, and when counting the vertices in the feature region currently being traversed from 0, x i represents the x-axis coordinate of the position of the vertex counted as i in the feature region currently being traversed, and y i This represents the y-axis coordinate of the vertex counted as i in the feature region currently being traversed, and z irepresents the z-axis coordinate at the position of vertex i in the feature region currently being traversed, while x^, y^, and z^ (translation's comment: x^ is the same as x with a bar above it, y^ is the same as y with a bar above it, and z^ is the same as z with a bar above it) represent the x-axis, y-axis, and z-axis coordinates at the position of the center point of the feature region currently being traversed, respectively.

number

[0079] Furthermore, when the model processing device determines the radius of each vertex in the currently traversed feature region from its corresponding center point, and the average radius of each vertex in the currently traversed feature region, based on the position of each vertex and the position of the center point in the currently traversed feature region, the radius of any vertex in the currently traversed feature region from its corresponding center point may be the distance between that vertex and its corresponding center point, and the average radius of each vertex in the currently traversed feature region is obtained by averaging the radii of each vertex from its corresponding center point in the currently traversed feature region, and the radius of a vertex in the currently traversed feature region from its corresponding center point is shown by the following equation 10.4, and the average radius of each vertex in the currently traversed feature region is shown by the following equation 10.5, R i R^ represents the radius from the corresponding center point of the vertex counted as i in the feature region currently being traversed, and R^ (translation comment: R^ is the same as the symbol R with a bar above it) represents the average radius corresponding to each vertex in the feature region currently being traversed.

number

[0080] The model processing device can determine the spherical similarity of the currently traversed feature region based on the difference between the radius from the corresponding center point of each vertex in the currently traversed feature region and the mean radius. For example, the average of the absolute differences between the radius from the corresponding center point of each vertex in the currently traversed feature region and the mean radius is taken as the spherical similarity of the currently traversed feature region. The spherical similarity of the currently traversed feature region is shown by the following formula 10.6, where Circularity represents the spherical similarity.

number

[0081] As can be seen from equation 10.6, the spherical similarity of one feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere. Based on this, the model processing device can determine eye regions from multiple feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere as indicated by its spherical similarity. This step may include selecting feature regions whose spherical similarity is less than a similarity threshold as candidate eye regions, determining the distance between the center point of each candidate eye region and the eyeball keypoint among the face keypoints of the adjusted target head model, and determining the candidate eye region indicated by the minimum distance as the eye region. The similarity threshold can be set according to specific needs. The eyeball keypoint is a facial keypoint that represents an eyeball. Since the eyeball can include both the left and right eyeballs, the eyeball keypoint can include both the left and right eyeball keypoints. Based on this, when determining the distance between the center point of each candidate eyeball region and the eyeball keypoint of the adjusted target head model, the distances to the left and right eyeball keypoints can be determined. Furthermore, among the candidate eyeball regions, the one with the smallest distance to the left eyeball keypoint is determined as the left eyeball region, and the one with the smallest distance to the right eyeball keypoint is determined as the right eyeball region. In other words, the left and right eyeball regions can be determined based on the similarity between the center point of the candidate eyeball region and the left and right eyeball keypoints. In 3D modeling, the eyeball is typically a multilayered concentric shell representing the constituent elements of the eye, such as the iris, pupil, sclera, and cornea. Therefore, the determined left and right eyeball regions may contain multiple concentric shells (also called multilayered eyeball components). Determining the eyeball region based on the above method is necessary because if the spherical similarity is lower than the similarity threshold, or if the center point of the feature region is too far from the center of the eyeball space with the highest confidence level, it will not be considered an eyeball region.The eye features can be determined more accurately using the calculation method described above, which allows for more precise control of the eye region and eyeball position in the model. This enables more appropriate control of feature point movement in the case of facial expression transformations related to the eyeballs, resulting in a more natural representation of eye-related facial expressions in the target head model.

[0082] The linked feature region corresponding to the eyeball appendage region is the eyeball region, meaning that the eyeball appendage region may adjust in accordance with the adjustment of the eyeball region. Therefore, after determining the eyeball region from multiple feature regions, the model processing device may perform facial expression adjustment processing on the eyeball region. Typically, facial expression adjustment processing of the eyeball region mainly involves eyeball rotation, and the position of the corresponding vertex in the adjusted target head model can be adjusted based on the vertex deformation parameters of each vertex of the eyeball region in the target head model, i.e., it is performed based on blendshape rotation. Referring to Figure 9, which is a schematic diagram of eyeball region positioning and rotation according to an embodiment of the present application, the model processing device determines a plurality of feature regions constituting the target head model, determines the eyeball region from the plurality of feature regions, and in the process of determining the eyeball region from the plurality of feature regions, the eyeball region (i.e., multilayer eyeball components constituting the eyeball region) can be determined based on the center point and spherical similarity of each determined feature region, the center point of the eyeball region, i.e., the center point of the multilayer eyeball component is determined based on the position of each vertex in the multilayer eyeball component, and since the multilayer eyeball component is a concentric spherical shell, the center point of the eyeball region can be determined based on the position of each vertex in any of the eyeball components. Furthermore, the model processing device can perform facial expression adjustment processing on the eyeball region, for example, based on blendshape rotation.

[0083] In one embodiment, after determining the eyeball region and the eyeball appendage region from a plurality of feature regions, a linked feature region corresponding to the target feature region (i.e., the eyeball appendage region) can be determined from the plurality of feature regions. That is, if the target feature region belongs to the eyeball appendage region, the positional relationship between the target feature region and the eyeball region in the plurality of feature regions is determined, and based on the positional relationship between the target feature region and the eyeball region, a linked feature region can be determined from the eyeball region and the face feature region. If the distance between the target feature region and the eyeball region is smaller than a distance threshold, the eyeball region is determined as a linked feature region; otherwise, the face feature region is determined as a linked feature region. The eyeball appendage region is a feature region that indicates an eyeball appendage, and the eyeball appendage may include structural components related to the eyeball, such as eyeball contents, eyelids, eyelashes, and eyebrows. The model processing device can determine linked feature regions from the eyeball region and the face feature region based on the positional relationship between the target feature region and the eyeball region, and can make this determination based on the distance between the target feature region and the eyeball region. If the distance between the target feature region and the eyeball region is less than a distance threshold, the eyeball region is determined as a linked feature region; otherwise, the face feature region is determined as a linked feature region. This distance threshold can be set according to specific needs. For example, the eyeball region can be determined as a linked feature region corresponding to the eyeball appendage region showing the contents of the eyeball, and the face feature region can be determined as a linked feature region corresponding to the eyeball appendage region showing the eyelids, the eyeball appendage region showing the eyelashes, and the eyeball appendage region showing the eyebrows.

[0084] Furthermore, optionally, when the model processing device determines linked feature regions from the eyeball region and the face feature region based on the positional relationship between the target feature region and the eyeball region, it may also determine the type to which each eyeball appendage region belongs based on the positional relationship between the target feature region (i.e., the eyeball appendage region) and the eyeball region, and determine linked feature regions from the eyeball region and the face feature region based on the type to which each eyeball appendage region belongs. For example, among the eyeball appendage regions, the eyeball appendage region that overlaps with the eyeball region may be determined as the eyeball appendage region indicating the contents of the eyeball, and among the eyeball appendage regions, the eyeball appendage region located above the eyeball region and close in distance from the face keypoint indicating the eyebrows may be determined as the eyeball appendage region indicating the eyebrows. Furthermore, when determining linked feature regions from the eyeball region and facial feature region based on the type to which each ocular appendage region belongs, the eyeball region can be determined as a linked feature region corresponding to the ocular appendage region showing the contents of the eyeball, and the facial feature region can be determined as a linked feature region corresponding to the ocular appendage region showing the eyelids, the ocular appendage region showing the eyelashes, and the ocular appendage region showing the eyebrows.

[0085] In one embodiment, the model processing device determines the oral cavity region from a plurality of feature regions, and further determines the oral appendage region, which is a feature region indicating the oral appendage region, and the oral appendage may include structural components related to the oral cavity such as the maxilla, mandible, teeth, and tongue. After determining the oral appendage region, the model processing device can determine the feature regions indicating the maxilla and mandible as linked feature regions corresponding to the oral appendage region, thereby allowing the oral appendage region to be adjusted in accordance with the adjustment of the feature regions indicating the maxilla and mandible. For example, stiffness adjustment can be performed in accordance with the adjustment of the feature regions indicating the maxilla and mandible. In the oral appendage region, the oral appendage region indicating the tongue may be adjusted based on the vertex deformation parameters of the vertices in the oral appendage region indicating the tongue in the target head model.

[0086] Referring to Figure 10, this is a schematic diagram of the facial expression transition processing performed on a target head model according to an embodiment of the present application. Based on the connected region of the target head model, a plurality of feature regions constituting the target head model are determined in the target head model. From the plurality of feature regions, the face feature region having the largest surface area is determined. Based on the vertex deformation parameters of each vertex of the face feature region in the target head model, the positions of the corresponding vertices in the target head model having the first model facial expression are adjusted to obtain the adjusted target head model. The eyeball region and eyeball appendage region are determined from the plurality of feature regions. Based on the adjusted target head model, facial expression adjustment processing is performed on the eyeball region and eyeball appendage region. For example, the eyeball appendage region showing eyelashes is adjusted to follow the adjustment of the face feature region. The oral appendage region is determined from the plurality of feature regions. After performing facial expression adjustment processing on the eyeball region and eyeball appendage region, facial expression adjustment processing is performed on the oral appendage region. For example, the oral appendage region showing the maxilla and mandible and the oral appendage region showing teeth are adjusted to follow the adjustment of the feature region showing the maxilla and mandible.

[0087] In one embodiment, when the target feature region belongs to the eyelid region, the first model expression in the source head model includes an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes an expression indicating that the source head model is in a closed-eye state. The model processing device performs an expression adjustment process on the target feature region based on the adjustment rules for the target feature region among the multiple feature regions, which determines the lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid region. Based on the size parameter and position of the eyeball region among the multiple feature regions, a collision deformation process is performed on the eyelid region using the Laplace equation to obtain the deformed eyelid region. This process causes the upper eyelid boundary and the lower eyelid boundary in the deformed eyelid region to overlap as much as possible. Since the eyes include the left and right eyes, in the above process, the adjustment to the eyelid region may be an adjustment to the eyelid region corresponding to the left eye, an adjustment to the eyelid region corresponding to the right eye, or an adjustment to the eyelid region corresponding to both eyes. For example, if the second model expression in the source head model indicates that the source head model has its left eye closed, the eyelid region corresponding to the left eye is adjusted accordingly. If the second model expression in the source head model indicates that the source head model has its right eye closed, the eyelid region corresponding to the right eye is adjusted accordingly. If the second model expression in the source head model indicates that the source head model has both eyes closed, the eyelid region corresponding to both eyes is adjusted accordingly.

[0088] In one selectable embodiment, the eyelid region may be determined from a plurality of feature regions, or it may be determined based on the distribution of facial keypoints in a target head model, and in specific implementation, facial keypoints indicating the lower part of the eyebrows and facial keypoints indicating the upper part of the eyes in the target head model can be connected by the shortest distance, and the connected region is determined as the eyelid region. Referring to Figure 11, this is a schematic diagram of eyelid region determination according to an embodiment of the present application, where the facial keypoint indicating the lower part of the eyebrows is shown as mark 1101, the facial keypoint indicating the upper part of the eyes may be shown as mark 1102, and the determined eyelid region is shown as mark 1103. Referring to Figure 12, this is a schematic diagram of the lower eyelid boundary and upper eyelid boundary according to an embodiment of the present application, where the upper eyelid boundary is shown as mark 1201 and the lower eyelid boundary is shown as mark 1202.

[0089] In the process where the model processing device performs collision deformation on the eyelid region using the Laplace equation based on the size parameters and position of the eyeball region in multiple feature regions to obtain the deformed eyelid region, the Laplace equation is solved with the size parameters and position of the eyeball region as newly added constraints. This resolves the collision between the eyelid vertices and the eyeball while maintaining the agreement of the grid surface normal vectors and the positions of the original vertices as much as possible. This reduces the possibility of the eyeball penetrating the model that can occur in the process of performing collision deformation on the eyelid region. Furthermore, when performing collision deformation on the eyelid region, the boundary points except for the upper eyelid boundary region are not changed, and the remaining vertices are deformed to make the upper and lower eyelid boundaries overlap. This process, by adding constraints to the Laplace equation, reduces the possibility of the eyeball penetrating the model that can occur in the process of performing collision deformation on the eyelid region, and improves the transition effect of the second model expression showing the closed-eye state to the target head model, that is, the target head model can achieve a better "closed-eye" effect.

[0090] Referring to Figure 13, a schematic diagram of adjusting the eyelid region according to an embodiment of the present application, the reference facial keypoint in the 2D facial image is shown as mark 1301, the model processing device can determine the facial keypoint in the target head model based on the reference facial keypoint in the 2D facial image, the distribution of which is shown as mark 1302, the eyelid region is determined from the target head model based on the shortest distance connection (also called the shortest path connection), the determined eyelid region is shown as mark 1303, after determining the lower eyelid boundary and upper eyelid boundary, collision deformation processing is performed on the eyelid region based on the Laplace equation to obtain the deformed eyelid region, the upper eyelid boundary and lower eyelid boundary in the deformed eyelid region overlap, and the corresponding adjustment effect is shown as mark 1304.

[0091] In one embodiment, the target head model is obtained by simplifying the model of the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model. In this case, after obtaining the target head model with the second model expression, the model processing device may determine the grid region in the target head model corresponding to the texture feature point based on the position of the texture feature point in the head model region of the interaction model, determine the position mapping relationship between the texture feature point in the head model region of the interaction model and the corresponding grid region in the target head model, determine the position of the texture feature point in the head model region of the interaction model after positional deformation based on the positional mapping relationship and the position of each vertex included in the corresponding grid region, and perform deformation processing on the texture feature point in the head model region of the interaction model based on the position of the texture feature point in the head model region of the interaction model after positional deformation to obtain the interaction model with the second model expression. Furthermore, the interaction model may be a model for interacting with an object, for example, a model obtained by a user pinching a face, a model in a movie or television production, a model in a game scenario, etc. Optionally, the interaction model may be a human body model, a head model, etc. Texture feature points may be texture coordinate points corresponding to vertices. Optionally, the position of a texture feature point may be the position indicated by the texture coordinates of the texture coordinate point. The position of each vertex included in the grid region may be the position indicated by the texture coordinates of the corresponding vertex. The position mapping relationship between the texture feature points in the head model region of the interaction model and the corresponding grid region in the target head model may be such that the texture feature point is located at the centroid coordinates of the corresponding grid region, and the grid region is triangular.

[0092] In an executable embodiment, the model processing device can simplify the head model region of an interaction model by performing texture expansion and decomposition on the head model region of the interaction model, and further by reducing the number of vertices. The number of vertices that need to be retained for model simplification can be set according to specific needs. For example, when simplifying the model by retaining 10,000 vertices, it is necessary to retain the boundary points of the head model region of the interaction model. Referring to Figure 14, which is a schematic diagram of the texture coordinate relationship according to an embodiment of the present application, the texture feature point in the head model region of the interaction model is shown as mark 1401, the texture coordinate of the texture feature point is represented as P', the corresponding grid region is shown as mark 1402, the texture coordinates of the three vertices are represented as P1, P2, and P3, respectively, the centroid coordinate of the texture feature point in the corresponding grid region is represented as [w1w2w3], and the position of the texture feature point after positional deformation is shown by the following formula 11.

number

[0093] Since the target head model is obtained by simplifying the model in the head model region of the interaction model, the head model of the interaction model can be considered a high-poly model and the target head model a low-poly model. This process is also a high-to-low-poly expression transition process, that is, the deformation of the low-poly target head model is transferred to the high-poly head model region of the interaction model, enabling the realization of a second model expression in the head model region of the interaction model. Based on the above method, the deformation transition from low-poly to high-poly is performed without loss of accuracy, and the generated blend shape (blend shape corresponding to the second model expression) does not require secondary smoothing, resulting in a more accurate blend shape result on the high-poly model. In other words, the effect of the second model expression realized on the high-poly model is good.

[0094] Referring to Figure 15, which is a schematic diagram of the facial expression transition to the interaction model according to an embodiment of the present application, the model processing device acquires a source head model in response to the acquired facial expression movement command, performs model simplification on the head model region of the interaction model, and acquires a target head model. The model processing device determines the face keypoints in the target head model, and based on the model features of the target head model, performs deformation matching on the source head model to acquire the source head model after deformation matching, and the model features of the target head model may include the positions of the face keypoints in the target head model. Furthermore, the model processing device determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, and performs facial expression transition processing on the target head model based on the deformation parameters to acquire a target head model having the second model expression. In the process of obtaining a target head model with a second model expression by performing an expression transition process on the target head model based on deformation parameters, the position of the corresponding vertices in the target head model with the first model expression can be adjusted based on the vertex deformation parameters of each vertex in the facial feature region of the target head model to obtain an adjusted target head model. Based on the adjusted target head model, an expression adjustment process is performed on the target feature region based on adjustment rules for the target feature region in multiple feature regions to obtain a target head model with a second model expression. This process may also be called a post-transition processing process.Furthermore, the model processing device determines the grid region in the target head model that corresponds to the texture feature point in the head model region of the interaction model, based on the position of the texture feature point in the head model region of the interaction model, and determines the position mapping relationship between the texture feature point in the head model region of the interaction model and the corresponding grid region in the target head model. Based on the position mapping relationship and the position of each vertex included in the corresponding grid region, it determines the position of the texture feature point in the head model region of the interaction model after positional deformation, and performs deformation processing on the texture feature point in the head model region of the interaction model based on the position of the texture feature point in the head model region of the interaction model after positional deformation to obtain an interaction model with a second model expression.

[0095] Referring to Figure 16, which is a schematic diagram of the effect of facial expression transition according to the embodiment of this application, as shown in 1601 and 1602, the model processing method proposed in this solution can achieve a good "closed eyes" effect on the target head model. Furthermore, since this method can perform deformation transitions from a low-poly model to a high-poly model without compromising accuracy, the generated blend shape (blend shape corresponding to the second model expression) does not require secondary smoothing processing, and a more accurate blend shape result can be obtained on the high-poly model. In other words, the effect of the second model expression realized on the high-poly model is good, and an example of the effect when the second model expression is realized on the high-poly model is shown in 1603.

[0096] In one embodiment, the above model processing method can be applied to an object interaction scenario. For example, it can be used to generate a model with a specific model expression in order to realize object interaction. For example, if an expression action command includes target text for instructing the generation of a second model expression, the model processing device can perform speech synthesis processing based on the target text to obtain a target voice corresponding to the target text. After obtaining a target head model with a second model expression, the device outputs the target head model with the second model expression and the target voice, and performs interaction processing based on the target head model. The speech synthesis processing is performed by a TTS (Text To Speech) module. The target text may be input by an object, and the target head model may be set by an object. For example, it may be the head model of a game character set by an object in a game scenario. The target head model and target voice can be output to the object's terminal device to realize interaction with the object. Referring to Figure 17, which is a schematic diagram of an application scenario according to an embodiment of the present application, the model processing device, upon receiving an expression action command containing target text for instructing the generation of a second model expression, performs speech synthesis processing based on the target text to obtain a target voice corresponding to the target text, obtains a source head model, performs command analysis processing on the expression action command to generate a second model expression expected by the expression action command for the source head model, transfers the second model expression to the target head model to obtain a target head model having the second model expression, outputs the target head model having the second model expression and a target voice, and can perform interaction processing based on the target head model.In the embodiments of this application, it is possible to display the facial expression of the source head model by voice and to instruct the transfer of the facial expression to the target head model, thereby making it easier to realize facial expression control for the designed target head model and meeting the needs for automation and intelligence in head model facial expression control.

[0097] In the embodiments of this application, the process of obtaining a target head model with a second model expression by performing an expression transition process based on deformation parameters is as follows: First, the position of the corresponding vertices in the target head model with the first model expression is adjusted based on the vertex deformation parameters of each vertex in the facial feature region of the target head model to obtain the adjusted target head model. Furthermore, based on the adjusted target head model, an expression adjustment process is performed on the target feature region based on the adjustment rules for the target feature region among a plurality of feature regions to obtain a target head model with a second model expression. The deformation transition of the eyeball region, eyeball appendage region, oral appendage region and eyelid region in the target head model can be processed to improve the expression transition effect of the corresponding region. Furthermore, a low-poly target head model may be obtained by simplifying the high-poly head model, thereby enabling expression transitions in the low-poly target head model. After obtaining the target head model with the second model expression, the corresponding deformations can be transferred from the low-poly model to the high-poly model to complete the transition to the second model expression in the high-poly head model, thereby speeding up the expression transition process for the high-poly model.

[0098] Based on the embodiments of the model processing method described above, the embodiments of this application provide a model processing apparatus. Referring to Figure 18, which is a schematic diagram of the configuration of the model processing apparatus according to the embodiments of this application, the model processing apparatus may include an acquisition unit 1801 and a processing unit 1802. The model processing apparatus shown in Figure 18 is used to perform the following operations.

[0099] The acquisition unit 1801 acquires a source head model in response to the acquired facial motion command, and the source head model is equipped with neutral feature data and multiple facial feature data for the neutral feature data. The processing unit 1802 performs deformation matching on the source head model based on the model features of the target head model and acquires the source head model after deformation matching. The processing unit 1802 determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model. The first model expression in the source head model is determined from the neutral feature data of the source head model, and the second model expression in the source head model is determined from the facial feature data indicated by the facial motion command among the multiple facial feature data. The processing unit 1802 further performs facial transition processing on the target head model based on the deformation parameters and acquires a target head model with the second model expression.

[0100] In one embodiment, the processing unit 1802 performs deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching. Specifically, Based on the position of each feature point in the source head model and the position of each feature point in the target head model, a matching relationship is determined between each feature point in the source head model and each feature point in the target head model. Each feature point includes either or both of the vertices and / or face keypoints in the corresponding model. Based on the matching relationship, the positions of the vertices constituting the source head model are adjusted using the position of the feature point in the target head model as the reference position to obtain the source head model after deformation matching. The distance between the position of the feature point in the source model after deformation matching and the corresponding reference position determined according to the matching relationship satisfies the proximity condition.

[0101] In one embodiment, when the processing unit 1802 determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, specifically, Based on the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model, the deformation relationship between the source head model and the source head model after deformation matching is determined. The feature points include one or both of the vertices and / or face keypoints in the corresponding model. Based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, the deformation parameters are determined.

[0102] In one embodiment, when the processing unit 1802 determines deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, specifically, Based on the deformation relationship between the source head model and the source head model after deformation matching, a first vertex deformation relationship is determined between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching. Based on the deformation relationship from the first model expression to the second model expression in the source head model, a second vertex deformation relationship is determined between each reference vertex in the source head model having the first model expression and the corresponding vertex in the source head model having the second model expression. Based on the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex, the vertex deformation parameter of the corresponding target vertex in the target head model for each reference vertex is determined.

[0103] In one embodiment, when the processing unit 1802 performs facial expression transition processing on the target head model based on deformation parameters to obtain a target head model having a second model facial expression, specifically, In the target head model, multiple feature regions constituting the target head model are determined, and based on the vertex deformation parameters of each vertex in the face feature region of the target head model, the positions of the corresponding vertices in the target head model with the first model expression are adjusted to obtain the adjusted target head model. The face feature region is the feature region with the largest surface area among the multiple feature regions, and based on the adjusted target head model, an expression adjustment process is performed on the target feature region based on the adjustment rules for the target feature region in the multiple feature regions to obtain the second target head model with the model expression.

[0104] In one embodiment, when the processing unit 1802 performs facial expression adjustment processing on a target feature region based on the adjustment rules for the target feature region among a plurality of feature regions, specifically, From multiple feature regions, linked feature regions corresponding to the target feature region are determined, and based on the instructions of the adjustment rules for the target feature region, a follow-up adjustment process is performed on the target feature region in accordance with the adjustment of the linked feature region.

[0105] In one embodiment, when the processing unit 1802 performs follow-up adjustment processing on the target feature region in accordance with the adjustment of the linked feature region based on the instructions of the adjustment rule for the target feature region, specifically, If the overlapping area between the target feature region and the linked feature region is smaller than the area threshold, the vertex positions in the target feature region are adjusted according to the stiffness adjustment method, following the adjustment of the vertex positions in the linked feature region. If the overlapping area between the target feature region and the linked feature region is greater than or equal to the area threshold, the vertex positions in the target feature region are adjusted according to the Laplace method, following the adjustment of the vertex positions in the linked feature region.

[0106] In one embodiment, when the processing unit 1802 determines a linked feature region corresponding to a target feature region from a plurality of feature regions, specifically, If the target feature region belongs to the eyeball appendage region, the positional relationship between the target feature region and the eyeball region among multiple feature regions is determined. Based on the positional relationship between the target feature region and the eyeball region, linked feature regions are determined from the eyeball region and the face feature region. If the distance between the target feature region and the eyeball region is smaller than the distance threshold, the eyeball region is determined as a linked feature region; otherwise, the face feature region is determined as a linked feature region.

[0107] In one embodiment, the multiple feature regions include the eyeball region, and when the processing unit 1802 determines the eyeball region from the multiple feature regions, specifically, The center point and spherical similarity of each feature region are determined. The spherical similarity of one feature region is used to indicate the degree to which the corresponding feature region belongs to the sphere. Based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to the sphere as indicated by its spherical similarity, the eyeball region is determined from multiple feature regions.

[0108] In one embodiment, the spherical similarity of a feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere. When the processing unit 1802 determines the eyeball region from multiple feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity, it specifically determines the eyeball region. For each feature region, feature regions where the spherical similarity is less than the similarity threshold are selected as candidate eyeball regions. The distance between the center point of each candidate eyeball region and the eyeball keypoint among the face keypoints of the adjusted target head model is determined, and the candidate eyeball region indicated by the minimum distance is selected as the eyeball region.

[0109] In one embodiment, when the processing unit 1802 determines the center point and spherical similarity of each feature region, specifically, For each feature region being traversed, the center point of the currently traversed feature region is determined according to the position of each vertex in that feature region. Based on the positions of each vertex and the center point in the currently traversed feature region, the radius of each vertex in the currently traversed feature region from the corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature region are determined. The spherical similarity of the currently traversed feature region is determined based on the difference between the radius of each vertex from the corresponding center point and the average radius.

[0110] In one embodiment, the target feature region belongs to the eyelid region, the first model expression in the source head model includes an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes an expression indicating that the source head model is in a closed-eye state. When processing unit 1802 performs facial expression adjustment processing on a target feature region based on the adjustment rules for the target feature region among multiple feature regions, it specifically performs the following: The lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid region are determined, and based on the size parameter and position of the eyeball region among multiple feature regions, collision deformation processing is performed on the eyelid region using the Laplace equation to obtain the deformed eyelid region.

[0111] In one embodiment, when the processing unit 1802 determines a plurality of feature regions that constitute the target head model in the target head model, specifically, In the target head model, the connected regions of the target head model are determined, the weight boundaries between each connected region are determined, and based on the length of each determined weight boundary, a region merging process is performed on each connected region, and the multiple connected regions obtained by region merging are treated as multiple feature regions.

[0112] In one embodiment, the target head model is obtained by simplifying the model of the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model, and the processing unit 1802 further... Based on the position of texture feature points in the head model region of the interaction model, the grid region corresponding to the texture feature points in the target head model is determined, and the positional mapping relationship between the texture feature points in the head model region of the interaction model and the corresponding grid region in the target head model is determined. Based on the positional mapping relationship and the position of each vertex included in the corresponding grid region, the position after deformation of the texture feature points in the head model region of the interaction model is determined, and a deformation process is performed on the texture feature points in the head model region of the interaction model based on the position after deformation to obtain an interaction model with a second model expression.

[0113] In one embodiment, the facial expression command includes target text for instructing the generation of a second model facial expression, and the processing unit 1802 further, Based on the target text, speech synthesis processing is performed to obtain the target audio corresponding to the target text, and after obtaining a target head model with a second model facial expression, the target head model with the second model facial expression and the target audio are output, and interaction processing is performed based on the target head model.

[0114] According to one embodiment of this application, each step of the model processing method shown in Figures 2, 3, and 7 may be performed by each unit in the model processing apparatus shown in Figure 18. For example, step S201 shown in Figure 2 can be performed by the acquisition unit 1801 in the model processing apparatus shown in Figure 18, and steps S202 to S204 shown in Figure 2 can be performed by the processing unit 1802 in the model processing apparatus shown in Figure 18. Also, step S301 shown in Figure 3 can be performed by the acquisition unit 1801 in the model processing apparatus shown in Figure 18, and steps S302 to S305 shown in Figure 3 can be performed by the processing unit 1802 in the model processing apparatus shown in Figure 18. Furthermore, step S701 shown in Figure 7 can be performed by the acquisition unit 1801 in the model processing apparatus shown in Figure 18, and steps S702 to S706 shown in Figure 7 can be performed by the processing unit 1802 in the model processing apparatus shown in Figure 18.

[0115] According to another embodiment of this application, each unit in the model processing apparatus shown in Figure 18 may be individually or collectively integrated into one or more other units, or one (or several) of these units may be further divided into several functionally smaller units, which can achieve similar operation and does not affect the realization of the technical effects of the embodiment of this application. The above units are divided based on logical function, and in practice, the function of one unit may be realized by multiple units, or the function of multiple units may be realized by one unit, for example, the function realized by each of the above units may be realized by one processing unit. In another embodiment of this application, the model processing apparatus by logical function division may include other units, and in actual application, these functions may be realized in cooperation with other units, or coordinated by multiple units.

[0116] According to another embodiment of this application, a model processing device shown in Figure 18 can be constructed on a general computing device such as a computer equipped with processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), by executing a computer program (including program code) for each step of the corresponding method shown in Figures 2, 3, and 7, thereby realizing the model processing method in the embodiment of this application. The computer program can be stored, for example, in a computer-readable storage medium, loaded onto the computing device by the computer-readable storage medium, and executed.

[0117] In the embodiments of this application, after obtaining an expression motion command, the source head model can be deformed to match the target head model so that the source head model after deformation matching approaches the target head model. Furthermore, deformation parameters can be determined based on the source head model after deformation matching and the deformation relationship from the first model expression in the source head model to the second model expression indicated by the expression motion command. Based on the deformation parameters, an expression transition process can be performed in the target head model to obtain a target head model with the second model expression. The transition of the second model expression from the source head model to the target head model can be automatically realized, that is, the second model expression can be automatically generated for the target head model, the second model expression in the source head model can be reused in different head models, and the efficiency of generating model expressions for different head models can be improved, reducing the consumption of human resources.

[0118] Based on the relevant embodiments of the model processing method and the embodiments of the model processing apparatus described above, this application further provides a model processing device. Referring to Figure 19, a schematic diagram of the configuration of a model processing device according to an embodiment of this application is shown. The model processing device shown in Figure 19 may include at least a processor 1901, an input interface 1902, an output interface 1903, and a computer storage medium 1904. The processor 1901, the input interface 1902, the output interface 1903, and the computer storage medium 1904 may be connected via a bus or other means.

[0119] The computer storage medium 1904 is stored in the memory of the model processing device, and the computer storage medium 1904 is used to store a computer program, which includes program instructions, and the processor 1901 is used to store the program instructions stored in the computer storage medium 1904. The processor 1901 (or referred to as the CPU (Central Processing Unit)) is the computational and control core of the model processing device and is suitable for executing one or more instructions, specifically, for realizing the flow or corresponding function of the model processing method described above by loading and executing one or more instructions.

[0120] Embodiments of this application further provide a computer memory medium, which is a memory device in a model processing device for storing programs and data. The computer memory medium may include a memory medium built into a terminal, and of course, may include an extended memory medium supported by the terminal. The computer memory medium provides a memory space in which the terminal's operating system is stored. The memory space also contains one or more instructions suitable for being loaded and executed by the processor 1901, which may be one or more computer programs (including program code). The computer memory medium may be a high-speed random access memory (RAM) memory, or a non-volatile memory such as at least one magnetic disk memory, and optionally, at least one computer memory medium located away from the aforementioned processor.

[0121] In one embodiment, the processor 1901 loads and executes one or more instructions stored in the computer storage medium to realize the corresponding steps in the embodiment of the model processing method shown in Figures 2, 3 and 7 above. In a specific embodiment, one or more instructions in the computer storage medium are loaded by the processor 1901 and the following steps are executed. In response to the acquired facial expression command, a source head model is acquired. The source head model contains neutral feature data and multiple facial expression feature data relative to the neutral feature data. Based on the model features of the target head model, a deformation matching process is performed on the source head model to acquire the source head model after deformation matching. Based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, deformation parameters are determined. The first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial expression feature data indicated by the facial expression command among the multiple facial expression feature data. In the target head model, a facial expression transition process is performed based on the deformation parameters to acquire the target head model with the second model expression.

[0122] In one embodiment, when the processor 1901 performs deformation matching on the source head model based on the model features of the target head model to obtain the source head model after deformation matching, specifically, The process involves: determining a matching relationship between each feature point in the source head model and each feature point in the target head model based on the positions of each feature point in the source head model and each feature point in the target head model, wherein the feature point includes one or both of the vertices and / or face keypoints in the corresponding model; and obtaining a source head model after deformation matching by adjusting the positions of the vertices constituting the source head model, using the positions of the feature points in the target head model as reference positions, based on the matching relationship, wherein the distance between the positions of the feature points in the source head model after deformation matching and the corresponding reference positions determined based on the matching relationship satisfies the proximity condition.

[0123] In one embodiment, when the processor 1901 determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, specifically, The process involves determining the deformation relationship between the source head model and the source head model after deformation matching, based on the position of each feature point in the source head model after deformation matching and the position of the corresponding feature point in the source head model, wherein the feature point includes one or both of the vertices and / or face keypoints in the corresponding model; and determining deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from a first model expression to a second model expression in the source head model.

[0124] In one embodiment, when the processor 1901 determines deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, specifically, The following steps are performed: determining a first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching, based on the deformation relationship between the source head model and the source head model after deformation matching; determining a second vertex deformation relationship between each reference vertex in the source head model having the first model expression and the corresponding vertex in the source head model having the second model expression, based on the deformation relationship from the first model expression to the second model expression in the source head model; and determining the vertex deformation parameter of the corresponding target vertex in the target head model for each reference vertex, based on the first vertex deformation relationship of each reference vertex and the second vertex deformation relationship of each reference vertex.

[0125] In one embodiment, when the processor 1901 performs facial expression transition processing on the target head model based on deformation parameters to obtain a target head model having a second model facial expression, specifically, The process involves three steps: determining a target head model, determining multiple feature regions that constitute the target head model, adjusting the positions of corresponding vertices in the target head model having a first model expression based on vertex deformation parameters of each vertex in the face feature region of the target head model, and obtaining an adjusted target head model, wherein the face feature region is the feature region having the largest surface area among the multiple feature regions; and performing an expression adjustment process on the target feature region based on the adjusted target head model and an adjustment rule for the target feature region among the multiple feature regions, to obtain a target head model having a second model expression.

[0126] In one embodiment, when the processor 1901 performs facial expression adjustment processing on a target feature region based on an adjustment rule for the target feature region among a plurality of feature regions, specifically, The process involves two steps: determining linked feature regions corresponding to a target feature region from multiple feature regions, and performing follow-up adjustment processing on the target feature region according to the adjustment of the linked feature regions, based on the instructions of the adjustment rules for the target feature region.

[0127] In one embodiment, when the processor 1901 performs facial expression adjustment processing on a target feature region based on an adjustment rule for the target feature region among a plurality of feature regions, specifically, The process involves the steps of determining linked feature regions corresponding to a target feature region from multiple feature regions, and performing follow-up adjustment processing on the target feature region in accordance with the adjustment of the linked feature regions, based on the instructions of the adjustment rules for the target feature region.

[0128] In one embodiment, when the processor 1901 determines a linked feature region corresponding to a target feature region from a plurality of feature regions, specifically, If the target feature region belongs to the eyeball appendage region, the system performs the following steps: determine the positional relationship between the target feature region and the eyeball region among multiple feature regions, and determine the linked feature region from the eyeball region and the face feature region based on the positional relationship between the target feature region and the eyeball region; and if the distance between the target feature region and the eyeball region is less than a distance threshold, the eyeball region is determined as the linked feature region; otherwise, the face feature region is determined as the linked feature region.

[0129] In one embodiment, the multiple feature regions include the eyeball region, and when the processor 1901 determines the eyeball region from the multiple feature regions, specifically, The steps include determining the center point and spherical similarity of each feature region, wherein the spherical similarity of one feature region indicates the degree to which the corresponding feature region belongs to a sphere, and determining the eyeball region from multiple feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity of each feature region.

[0130] In one embodiment, the spherical similarity of one feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere, and when the processor 1901 determines the eyeball region from multiple feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity of each feature region, specifically, The following steps are performed: selecting feature regions from among the feature regions in which the spherical similarity is less than the similarity threshold as candidate eyeball regions; determining the distance between the center point of each candidate eyeball region and the eyeball keypoint among the face keypoints of the adjusted target head model; and determining the candidate eyeball region indicated by the minimum distance from among the candidate eyeball regions as the eyeball region.

[0131] In one embodiment, when the processor 1901 determines the center point and spherical similarity of each feature region, specifically, For each feature region being traversed, the following steps are performed: determining the center point of the currently traversed feature region based on the position of each vertex in the currently traversed feature region; determining the radius of each vertex in the currently traversed feature region from its corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature region, based on the position of each vertex in the currently traversed feature region and the position of the center point; and determining the spherical similarity of the currently traversed feature region based on the difference between the radius of each vertex from its corresponding center point and the average radius.

[0132] In one embodiment, the target feature region belongs to the eyelid region, the first model expression in the source head model includes an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes an expression indicating that the source head model is in a closed-eye state. When the processor 1901 performs expression adjustment processing on the target feature region based on the adjustment rules for the target feature region among the multiple feature regions, specifically, The process involves the steps of determining the lower eyelid boundary in the adjusted target head model and the upper eyelid boundary in the eyelid region, and performing collision deformation on the eyelid region using the Laplace equation based on the size parameter and position of the eyeball region among multiple feature regions to obtain the deformed eyelid region.

[0133] In one embodiment, when the processor 1901 determines a plurality of feature regions that constitute the target head model in the target head model, specifically, The process involves determining the connected regions of the target head model and determining the weight boundaries between each connected region, and then performing a region merge process on each connected region based on the length of each determined weight boundary, thereby creating multiple feature regions from the multiple connected regions obtained by the region merge.

[0134] In one embodiment, the target head model is obtained by simplifying the model of the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model, and the processor 1901 further The following steps are performed: determining the grid region corresponding to the texture feature point in the target head model based on the position of the texture feature point in the head model region of the interaction model, and determining the position mapping relationship between the texture feature point in the head model region of the interaction model and the corresponding grid region in the target head model; determining the position after the texture feature point in the head model region of the interaction model has been deformed based on the position mapping relationship and the position of each vertex included in the corresponding grid region; and performing a deformation process on the texture feature point in the head model region of the interaction model based on the position after the texture feature point in the head model region of the interaction model has been deformed to obtain an interaction model with a second model expression.

[0135] In one embodiment, the facial expression command includes target text for instructing the generation of a second model facial expression, and the processor 1901 further, The process involves performing speech synthesis based on the target text to obtain a target voice corresponding to the target text, and then obtaining a target head model with a second model facial expression, outputting the target head model with the second model facial expression and the target voice, and performing interaction processing based on the target head model.

[0136] Embodiments of this application provide a computer program product including a computer program, which is stored in a computer storage medium. The processor of a model processing device reads the computer program from the computer storage medium and executes the computer program, causing the model processing device to perform embodiments of the method shown in Figures 2, 3, and 7. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0137] The above are merely specific embodiments of the present application, and the scope of protection of this application is not limited to these. Any person skilled in the art will readily conceive of modifications and substitutions within the scope of the technical information disclosed herein, and these should be included within the scope of protection of this application. Therefore, the scope of protection of this application is based on the scope of protection of the claims.

Claims

1. A model processing method performed by an electronic device, A step of acquiring a source head model in response to an acquired facial expression command, wherein the source head model is configured with neutral feature data and a plurality of facial expression feature data corresponding to the neutral feature data. The steps include: performing a deformation matching process on the source head model based on the model features of the target head model to obtain the source head model after deformation matching; A step of determining deformation parameters based on the deformation-matched source head model and the deformation relationship from a first model expression to a second model expression in the source head model, wherein the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the facial feature data indicated by the facial motion command in the plurality of facial feature data, The steps include: performing facial expression transition processing on the target head model based on the deformation parameters to obtain a target head model having a second model facial expression; A method characterized by including the following.

2. The step of performing a deformation matching process on the source head model based on the model features of the target head model to obtain the source head model after deformation matching is: A step of determining a matching relationship between each feature point in the source head model and each feature point in the target head model, based on the position of each feature point in the source head model and the position of each feature point in the target head model, wherein the feature point includes one or both of vertices and face keypoints in the corresponding model. Based on the matching relationship, the position of the feature points in the target head model is used as the reference position, and the position of the vertices constituting the source head model is adjusted to obtain the source head model after deformation matching. Includes, The distance between the position of the feature point in the source head model after deformation matching and the corresponding reference position determined based on the matching relationship satisfies the proximity condition. The method according to claim 1, characterized by the features described above.

3. The step of determining deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model is as follows: A step of determining the deformation relationship between the source head model and the source head model after deformation matching, based on the positions of each feature point in the source head model after deformation matching and the positions of corresponding feature points in the source head model, wherein the feature point includes one or both of vertices and face keypoints in the corresponding model. The steps include determining the deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, The method according to 1 or 2, characterized by including the following:

4. The deformation relationship between the source head model and the deformation-matched source head model includes a transformation matrix from the vertices of the target triangle in the source head model to the corresponding vertices in the deformation-matched source head model, wherein the transformation matrix between vertices is determined based on affine transformation parameters and translation parameters. The method according to any one of claims 1 to 3, characterized by the following:

5. The step of determining the deformation parameters based on the deformation relationship between the source head model and the source head model after deformation matching, and the deformation relationship from the first model expression to the second model expression in the source head model, A step of determining a first vertex deformation relationship between each reference vertex in the source head model and the corresponding vertex in the source head model after deformation matching, based on the deformation relationship between the source head model and the source head model after deformation matching. The steps include determining a second vertex deformation relationship between each reference vertex in the source head model having the first model expression and the corresponding vertex in the source head model having the second model expression, based on the deformation relationship from the first model expression to the second model expression in the source head model, The steps include determining the vertex deformation parameters of the corresponding target vertices in the target head model for each of the reference vertices based on the first vertex deformation relationship of each of the reference vertices and the second vertex deformation relationship of each of the reference vertices, The method according to any one of claims 1 to 4, characterized by including the following:

6. The step of obtaining a target head model having a second model expression by performing an expression transition process based on the deformation parameters in the target head model is: The steps include determining a plurality of feature regions that constitute the target head model, A step of obtaining an adjusted target head model by adjusting the position of the corresponding vertex in the target head model having a first model expression based on the vertex deformation parameter of each vertex in the facial feature region of the target head model, wherein the facial feature region is the feature region having the largest surface area among the plurality of feature regions. The steps include: obtaining a target head model having the second model facial expression by performing facial expression adjustment processing on the target feature region based on the adjustment rules for the target feature region among the plurality of feature regions, using the adjusted target head model as a basis; The method according to any one of claims 1 to 5, characterized by including

7. The step of performing facial expression adjustment processing on the target feature region based on the adjustment rules for the target feature region among the multiple feature regions is: The steps include determining an interlocked feature region corresponding to the target feature region from the plurality of feature regions, The steps include: performing a follow-up adjustment process on the target feature region in accordance with the adjustment of the linked feature region, based on the instructions of the adjustment rule for the target feature region; The method according to any one of claims 1 to 6, characterized by including the following:

8. The step of performing a follow-up adjustment process on the target feature region in accordance with the adjustment of the linked feature region, based on the instructions of the adjustment rule for the target feature region, is: If the overlapping area between the target feature region and the linked feature region is smaller than the area threshold, the steps include adjusting the position of the vertices in the target feature region in accordance with the adjustment of the vertex positions in the linked feature region, according to the stiffness adjustment method, If the overlapping area of ​​the target feature region and the linked feature region is greater than or equal to the area threshold, the Laplace method is used to adjust the position of the vertices in the target feature region in accordance with the adjustment of the vertex positions in the linked feature region. The method according to any one of claims 1 to 7, characterized by including the following:

9. The step of determining the linked feature region corresponding to the target feature region from the plurality of feature regions is: If the target feature region belongs to the ocular appendage region, the step of determining the positional relationship between the target feature region and the ocular region in the plurality of feature regions, The steps include determining the linked feature region from the eye region and the face feature region based on the positional relationship between the target feature region and the eye region, Includes, If the distance between the target feature region and the eyeball region is less than the distance threshold, the eyeball region is determined to be the linked feature region; otherwise, the face feature region is determined to be the linked feature region. The method according to any one of claims 1 to 8, characterized by...

10. The plurality of feature regions include the eyeball region, and the method for determining the eyeball region from the plurality of feature regions is: A step of determining the center point and spherical similarity of each feature region, wherein the spherical similarity of one feature region indicates the degree to which the corresponding feature region belongs to a sphere, The steps include determining the eyeball region from the plurality of feature regions based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity of each feature region, The method according to any one of claims 1 to 9, characterized by including the following:

11. The spherical similarity of a feature region is negatively correlated with the degree to which the corresponding feature region belongs to a sphere. The step of determining the eyeball region from the plurality of feature regions, based on the relationship between the distance between the center point of each feature region and the eyeball keypoint among the face keypoints of the adjusted target head model, and the degree to which each feature region belongs to a sphere, as indicated by the spherical similarity, is as follows: The step of selecting a candidate eyeball region from among the aforementioned feature regions in which the spherical similarity is less than the similarity threshold, The steps include determining the distance between the center point of each candidate eyeball region and the eyeball keypoint in the face keypoint of the adjusted target head model, The step of selecting the candidate eyeball region indicated by the minimum distance from each of the candidate eyeball regions mentioned above as the eyeball region, The method according to any one of claims 1 to 10, characterized by including the following:

12. The steps to determine the center point and spherical similarity of each feature region are as follows: The steps include determining the center point of the currently traversed feature region based on the position of each vertex in the currently traversed feature region, among the feature regions mentioned above, The steps include determining the radius of each vertex in the currently traversed feature region from its corresponding center point, and the average radius corresponding to each vertex in the currently traversed feature region, based on the position of each vertex and the position of its center point in the feature region being traversed, The steps include determining the spherical similarity of the currently traversed feature region based on the difference between the radius of each vertex in the currently traversed feature region from its corresponding center point and the average radius, The method according to any one of claims 1 to 11, characterized by including the following:

13. The target feature region belongs to the eyelid region, the first model expression in the source head model includes an expression indicating that the source head model is in an open-eye state, and the second model expression in the source head model includes an expression indicating that the source head model is in a closed-eye state. The step of performing facial expression adjustment processing on the target feature region based on the adjustment rules for the target feature region among the multiple feature regions is: The steps include determining the lower eyelid boundary and the upper eyelid boundary in the eyelid region of the adjusted target head model, The steps include: performing collision deformation on the eyelid region using the Laplace equation based on the size parameter and position of the eyeball region among the multiple feature regions, and obtaining the deformed eyelid region; The method according to any one of claims 1 to 12, characterized by including the following:

14. The step of determining a plurality of feature regions constituting the target head model in the target head model is: The steps include determining the connection regions of the target head model and determining the weight boundaries between each connection region, The steps include: performing a region merge process on each of the connected regions based on the length of each determined weight boundary, and making the multiple connected regions obtained by region merging the multiple feature regions; The method according to any one of claims 1 to 13, characterized by including the following:

15. The target head model is obtained by simplifying the head model region of the interaction model, and the number of vertices in the target head model is smaller than the number of vertices in the head model region of the interaction model. The method further... The steps include determining the grid region in the target head model that corresponds to the texture feature point based on the position of the texture feature point in the head model region of the interaction model, and determining the positional mapping relationship between the texture feature point in the head model region of the interaction model and the corresponding grid region in the target head model, The steps include determining the position of the texture feature points in the head model region of the interaction model after positional deformation, based on the positional mapping relationship and the position of each vertex included in the corresponding grid region, The steps include: obtaining an interaction model having a second model expression by performing a deformation process on the texture feature points in the head model region of the interaction model based on the position of the texture feature points in the head model region of the interaction model after the deformation of the texture feature points in the head model region of the interaction model; The method according to any one of claims 1 to 14, characterized by including the following:

16. The facial expression command includes target text for instructing the generation of a second model facial expression, and the method further includes The steps include: performing speech synthesis processing based on the target text to obtain a target voice corresponding to the target text; After obtaining a target head model having the second model facial expression, the steps include outputting the target head model having the second model facial expression and the target voice, and performing interaction processing based on the target head model, The method according to any one of claims 1 to 15, characterized by including the following:

17. A model processing device, An acquisition unit that acquires a source head model in response to an acquired facial movement command, wherein the source head model has neutral feature data and a plurality of facial feature data corresponding to the neutral feature data arranged in the acquisition unit, A processing unit that performs deformation matching on the source head model based on the model features of the target head model and obtains the source head model after deformation matching, Includes, The processing unit further determines deformation parameters based on the source head model after deformation matching and the deformation relationship from the first model expression to the second model expression in the source head model, the first model expression in the source head model is determined based on the neutral feature data of the source head model, and the second model expression in the source head model is determined based on the expression feature data indicated by the expression motion command from among a plurality of expression feature data. The processing unit further performs facial expression transition processing on the target head model based on the deformation parameters to obtain a target head model having a second model facial expression. A model processing device characterized by the following:

18. A model processing device including an input interface and an output interface, A processor suitable for implementing one or more instructions, A computer storage medium storing one or more instructions suitable for the processor to load and execute the model processing method described in any one of claims 1 to 16, A model processing device further comprising:

19. A computer storage medium in which computer program instructions are stored, When the computer program instruction is executed by the processor, the model processing method described in any one of claims 1 to 16 is executed. A computer storage medium characterized by the following features.

20. It is a computer program, When the computer program is executed by the processor, the steps of the model processing method described in any one of claims 1 to 16 are realized. A computer program characterized by the following features.

Images