Facial model reconstruction method, device, equipment and storage medium

Abstract

The application discloses a face model reconstruction method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring first face key point position information corresponding to a first face model, the first face model being generated based on first generation parameters; acquiring a first target function for updating the first generation parameters based on the first face key point position information, the first target function being a function matched with a reconstruction direction of the first face model; updating the first generation parameters by using the first target function to obtain second generation parameters; and generating a second face model based on the second generation parameters in response to the second generation parameters satisfying an update termination condition. The face model reconstruction process does not need to rely on manual work and can be automatically executed. The reliability and efficiency of the face model reconstruction process are good, which can save human resources and time cost and is beneficial to improving the quality of the reconstructed face model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and particularly relate to a face model reconstruction method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, there are more and more application scenarios that require generating face models, for example, in a game character shaping scenario, a face model of a game character needs to be generated, in a virtual image construction scenario, a face model of a virtual image needs to be generated, etc. After generating a face model, the face model needs to be reconstructed to obtain a face model that is more suitable for the application scenario.

[0003] In the related art, in the process of reconstructing a face model, the generated face model is imported into editing software, and then the face model is adjusted by a person using the editing software to obtain a reconstructed face model. This face model reconstruction process relies on manual execution, has poor reliability and low update efficiency, not only requires a large amount of human resources and time cost, but also leads to poor quality of the reconstructed face model. SUMMARY

[0004] Embodiments of the present application provide a face model reconstruction method, device, equipment and storage medium, which can be used to improve the reliability and efficiency of the face model reconstruction process. The technical solution is as follows:

[0005] In one aspect, the present application provides a face model reconstruction method, the method comprising:

[0006] obtaining first face key point position information corresponding to a first face model, the first face model being generated based on first generation parameters;

[0007] based on the first face key point position information, obtaining a first target function for updating the first generation parameters, the first target function being a function matched with the reconstruction direction of the first face model;

[0008] updating the first generation parameters using the first target function to obtain second generation parameters;

[0009] in response to the second generation parameters satisfying an update termination condition, generating a second face model based on the second generation parameters.

[0010] In another aspect, a face model reconstruction device is provided, the device comprising:

[0011] a first obtaining unit configured to obtain first face key point position information corresponding to a first face model, the first face model being generated based on first generation parameters;

[0012] The second acquisition unit is used to acquire a first objective function for updating the first generated parameters based on the first facial key point location information. The first objective function is a function that matches the reconstruction direction of the first facial model.

[0013] An update unit is used to update the first generated parameters using the first objective function to obtain the second generated parameters;

[0014] A generation unit is configured to generate a second facial model based on the second generation parameters in response to the second generation parameters satisfying the update termination condition.

[0015] In one possible implementation, the second acquisition unit is configured to acquire a first loss function and a second loss function based on the first facial key point location information. The first loss function is used to constrain the overall facial features of the reconstructed facial model, and the second loss function is used to optimize the reference features of the reconstructed facial model, wherein the reference features are different from the overall facial features. Based on the first loss function and the second loss function, a first objective function is acquired to update the first generation parameters.

[0016] In one possible implementation, the second acquisition unit is further configured to acquire reference facial key point location information corresponding to the reference facial model, wherein the reference facial model is used to provide constraint direction for the overall facial features of the reconstructed facial model; and to acquire the first loss function based on the first facial key point location information and the reference facial key point location information.

[0017] In one possible implementation, the second acquisition unit is further configured to: acquire first target key point location information based on the first facial key point location information, wherein the first target key point location information includes the location information corresponding to the target key points in the first facial model; acquire second target key point location information based on the reference facial key point location information, wherein the second target key point location information includes the location information corresponding to the target key points in the reference facial model, and the target key points in the reference facial model correspond to the target key points in the first facial model; and acquire the first loss function based on the first target key point location information and the second target key point location information.

[0018] In one possible implementation, the reference facial model is the original facial model of the first facial model, the reference features include local features, and the second loss function includes a local adjustment loss function, which is obtained directly based on the location information of the first facial key points.

[0019] In a possible implementation, the local adjustment loss function comprises at least one of a cheek adjustment loss function, a first nose adjustment loss function, a first mouth adjustment loss function, a second nose adjustment loss function, a chin adjustment loss function, a third nose adjustment loss function, a philtrum adjustment loss function, an eye adjustment loss function, a fourth nose adjustment loss function, and a second mouth adjustment loss function.

[0020] In a possible implementation, the local adjustment loss function comprises a cheek adjustment loss function; and the second obtaining unit is further configured to: obtain cheek left key point position information and cheek right key point position information based on the first facial key point position information; and obtain the cheek adjustment loss function based on the cheek left key point position information and the cheek right key point position information.

[0021] In a possible implementation, the local adjustment loss function comprises a second nose adjustment loss function; and the second obtaining unit is further configured to: obtain a nose bridge key point position information based on the first facial key point position information; obtain nose bridge guide point position information corresponding to the nose bridge key point position information; and obtain the second nose adjustment loss function based on the nose bridge key point position information and the nose bridge guide point position information.

[0022] In a possible implementation, the reference facial model is a template facial model, the reference feature comprises a scale feature, and the second loss function comprises a reference scale loss function, which is obtained based on the first facial key point position information and template facial key point position information corresponding to the template facial model.

[0023] In a possible implementation, the reference scale loss function comprises at least one of a horizontal scale loss function, a vertical scale loss function, a depth scale loss function, and a length-width scale loss function.

[0024] In a possible implementation, the reference scale loss function comprises a horizontal scale loss function; and the second obtaining unit is further configured to: obtain a first horizontal distance corresponding to the first facial model and at least one second horizontal distance corresponding to the first facial model based on the first facial key point position information; obtain a third horizontal distance corresponding to the template facial model and at least one fourth horizontal distance corresponding to the template facial model based on the template facial key point position information; and obtain the horizontal scale loss function based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance, and the at least one fourth horizontal distance.

[0025] In a possible implementation, the first obtaining unit is further configured to, in response to the second generation parameter not satisfying the update termination condition, obtain second face key point position information based on the second generation parameter.

[0026] The second obtaining unit is further configured to obtain a second target function for updating the second generation parameter based on the second face key point position information.

[0027] The updating unit is further configured to update the second generation parameter by using the second target function to obtain a third generation parameter.

[0028] The generating unit is further configured to, in response to the third generation parameter satisfying the update termination condition, generate a third face model based on the third generation parameter.

[0029] In a possible implementation, the generating unit is configured to apply the second generation parameter to a face model reconstruction model, and generate the second face model by using the face model reconstruction model with the second generation parameter.

[0030] In another aspect, a computer device is provided, which includes a processor and a memory. The memory stores at least one computer program. The at least one computer program is loaded and executed by the processor to implement the face model reconstruction method described above.

[0031] In another aspect, a computer readable storage medium is also provided. The computer readable storage medium stores at least one computer program. The at least one computer program is loaded and executed by a processor to implement the face model reconstruction method described above.

[0032] In another aspect, a computer program product or a computer program is also provided. The computer program product or the computer program includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computer device performs the face model reconstruction method described above.

[0033] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects:

[0034] In the embodiment of the present application, the generation parameters of the generated face model are updated by using the target function matched with the reconstruction direction of the first face model, and then the reconstructed face model is obtained based on the updated generation parameters. The face model reconstruction process does not need to rely on manual operation, can be automatically executed, has good reliability and high efficiency in the face model reconstruction process, can save human resources and time cost, and is beneficial to improve the quality of the reconstructed face model. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0036] Figure 1 is a schematic diagram of an implementation environment of a face model reconstruction method provided by an embodiment of the present application;

[0037] Figure 2 is a flowchart of a face model reconstruction method provided by an embodiment of the present application;

[0038] Figure 3 is a schematic diagram of 86 target key points in a face model provided by an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of a left cheek key point and a right cheek key point in a first face model provided by an embodiment of the present application;

[0040] Figure 5 is a schematic diagram of a left nose key point and a right nose key point in a first face model provided by an embodiment of the present application;

[0041] Figure 6 is a schematic diagram of an upper lip upper key point, an upper lip lower key point, a lower lip upper key point and a lower lip lower key point in a first face model provided by an embodiment of the present application;

[0042] Figure 7 is a schematic diagram of a nose bridge key point in a first face model and a nose bridge guide point corresponding to the nose bridge key point in the first face model provided by an embodiment of the present application;

[0043] Figure 8 is a schematic diagram of a chin key point in a first face model and a chin guide point corresponding to the chin key point in the first face model provided by an embodiment of the present application;

[0044] Figure 9is a schematic diagram of a left nostril key point, a right nostril key point, an upper nostril key point and a lower nostril key point in a first face model provided by an embodiment of the present application;

[0045] Figure 10 is a schematic diagram of a mouth key point and a nasal key point in a first face model provided by an embodiment of the present application;

[0046] Figure 11 is a schematic diagram of a lower eyelid key point in an eye in a first face model and a lower eyelid guide point corresponding to the lower eyelid key point in the eye in the first face model provided by an embodiment of the present application;

[0047] Figure 12 is a schematic diagram of a nasal key point in a first face model and a nasal guide point corresponding to the nasal key point in the first face model provided by an embodiment of the present application;

[0048] Figure 13 is a schematic diagram of a lower lip key point in a first face model and a lower lip guide point corresponding to the lower lip key point in the first face model provided by an embodiment of the present application;

[0049] Figure 14 is a schematic diagram of a first horizontal distance corresponding to a first face model and at least one second horizontal distance provided by an embodiment of the present application;

[0050] Figure 15 is a schematic diagram of a first vertical distance corresponding to a first face model, at least one second direct vertical distance and at least one second average vertical distance provided by an embodiment of the present application;

[0051] Figure 16 is a schematic diagram of a first depth distance corresponding to a first face model and at least one second depth distance provided by an embodiment of the present application;

[0052] Figure 17 is a schematic diagram of a reference vertical distance corresponding to a first face model and at least one reference horizontal distance provided by an embodiment of the present application;

[0053] Figure 18 is a schematic diagram of a process of reconstructing a first face model according to a reference provided template face model "three courts five eyes" ratio, so that each part of the reconstructed face model is close to the template face model in proportion provided by an embodiment of the present application;

[0054] Figure 19 is a schematic diagram of a face model reconstruction device provided by an embodiment of the present application;

[0055] Figure 20is a structural schematic diagram of a terminal provided by an embodiment of the present application.

[0056] Figure 21 is a structural schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application embodiments will be further described in detail below with reference to the drawings.

[0058] In order to facilitate understanding of the technical process of the present application embodiments, some terms related to the present application embodiments are explained below.

[0059] 3DMM (Three Dimensional Morphable Model, three-dimensional morphable model): a general three-dimensional face parameterization model, which uses a fixed number of points to represent a face. The core idea of 3DMM is that a face can be one-to-one matched in a three-dimensional space, and can be linearly added by a plurality of face orthogonal bases.

[0060] Exemplarily, a three-dimensional face can be represented by two feature vectors, a shape vector S and a texture vector W:

[0061]

[0062]

[0063] wherein, each element constituting the shape vector S is represented; it is illustrated that S is in a three-dimensional space; each element constituting the texture vector W is represented; it is illustrated that W is in a three-dimensional space; T represents the transpose of a vector; and n is an integer not less than 1.

[0064] The core idea of 3DMM is that any new face can be generated by linear combination of feature vectors of basic faces. Based on this core idea, the shape vector S' and the texture vector W' of any new face are obtained based on the following formula 1:

[0065] (Formula 1)

[0066] wherein, the average vector of the shape vectors of the basic faces is represented; the shape vector of the i-th basic face is represented; l l the shape vector of the i-th basic face is represented; the shape vector of the i-th basic face is represented; l ​a weight coefficient corresponding to the shape vector of the m-th basis face; m (m is an integer not less than 1) represents the number of basis faces; an average vector representing the texture vector of the basis faces; a texture vector representing the m-th basis face; l a texture vector representing the m-th basis face; a texture vector representing the m-th basis face; l a weight coefficient corresponding to the texture vector of the m-th basis face.

[0067] Optimization: Given a function , find an element such that for all in A, (minimize); or (maximize). Such formulations are sometimes also called "mathematical programming" (e.g., linear programming). Many real and theoretical problems can be modeled into such a general framework. The elements in A are called feasible solutions. The function f is called the objective function, or the cost function. A feasible solution that minimizes (or maximizes) the objective function is called an optimal solution.

[0068] Blend shape: A technique of a single mesh deforming to achieve a combination between many predefined shapes and any number, called a deformation target in Maya / 3ds Max (3D modeling and animation software). For example, a single mesh is a basic shape of the default shape (e.g., an expressionless face), and other shapes of the basic shape are used for mixing / deformation, which are different expressions (laugh, frown, close eyelids, etc.), collectively referred to as blend shapes or deformation targets.

[0069] In the example embodiment, the face model reconstruction method provided by the embodiment of the present application can be applied to the field of artificial intelligence technology. Next, the artificial intelligence technology is introduced.

[0070] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0071] Artificial intelligence technology is a comprehensive discipline involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0072] Among them, computer vision (CV) technology is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further process graphics, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, face model reconstruction, 3D (Three Dimensional) technology, virtual reality, augmented reality, map construction, etc. It also includes common biometric identification technologies.

[0073] Machine learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0074] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0075] Figure 1A schematic diagram of an implementation environment of the face model reconstruction method provided by the embodiments of the present application is shown. The implementation environment can include a terminal 11 and a server 12.

[0076] The face model reconstruction method provided by the embodiments of the present application can be executed by the terminal 11, or executed by the server 12, or executed by the terminal 11 and the server 12 jointly, and the embodiments of the present application do not limit this. For the case that the face model reconstruction method provided by the embodiments of the present application is executed by the terminal 11 and the server 12 jointly, the server 12 undertakes the main computing work, and the terminal 11 undertakes the secondary computing work; or the server 12 undertakes the secondary computing work, and the terminal 11 undertakes the main computing work; or the server 12 and the terminal 11 adopt a distributed computing architecture for collaborative computing.

[0077] In a possible implementation manner, the terminal 11 can be any electronic product that can interact with a user through one or more manners such as a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, or a handwriting device, for example, a PC (Personal Computer), a mobile phone, a smartphone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart car, a smart television, a smart speaker, and the like. The server 12 can be a server or a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 and the server 12 establish a communication connection through a wired or wireless network.

[0078] Those skilled in the art shall understand that the terminal 11 and the server 12 described above are only examples, and other existing or future terminal or server, such as applicable to the present application, shall also be included in the protection scope of the present application, and are hereby included by reference.

[0079] Based on the implementation environment shown above, Figure 1 The embodiments of the present application provide a face model reconstruction method. Taking the case that the method is applied to a computer device, the computer device can refer to a terminal or a server. As shown in the figure, Figure 2 The face model reconstruction method provided by the embodiments of the present application includes the following steps 201 to 204:

[0080] In step 201, first face key point position information corresponding to a first face model is acquired, and the first face model is generated based on first generation parameters.

[0081] The first face model refers to a face model to be reconstructed. The face model to be reconstructed can refer to a human face model or an animal face model, and the like, which is related to the actual application scenario. Embodiments of the present application take the face model to be reconstructed as a human face model as an example for illustration. The face model to be reconstructed can refer to a three-dimensional face model or a two-dimensional face model, and the like, and embodiments of the present application take the face model to be reconstructed as a three-dimensional face model as an example for illustration.

[0082] In an example embodiment, reconstructing the first face model can refer to directly reconstructing the first face model, or can refer to reconstructing the first face model based on a template face model, and the like, which is not limited in embodiments of the present application.

[0083] In an example embodiment, the reconstruction direction of reconstructing the first face model is set according to the actual application scenario, which is not limited in embodiments of the present application. The reconstruction direction of reconstructing the first face model refers to beautifying the first face model, for example. The reconstruction direction of reconstructing the first face model refers to uglifying the first face model, for example. Embodiments of the present application take the reconstruction direction of reconstructing the first face model as beautifying the first face model as an example for illustration. In addition, for the case that the reconstruction direction of reconstructing the first face model refers to beautifying the first face model, the reconstruction direction of reconstructing the first face model can be further refined as beautifying the cheeks in the first face model, beautifying the nose in the first face model, and the like.

[0084] In embodiments of the present application, the first face model is generated based on first generation parameters. The first generation parameters are parameters for generating the first face model. Embodiments of the present application do not limit the way of generating the first face model based on the first generation parameters, which is exemplified as follows: generating the first face model by using the first generation parameters to write an algorithm program; or generating the first face model by applying the first generation parameters to a face model reconstruction model.

[0085] The face model reconstruction model refers to a model for generating a face model. Depending on the type of face model to be generated, the type of face model reconstruction model can be different. The embodiments of the present application do not limit the type of face model reconstruction model, as long as the face model can be generated based on the generation parameter. Illustratively, the face model reconstruction model refers to 3DMM. For the case where the face model reconstruction model refers to 3DMM, the first face model refers to a three-dimensional face. Illustratively, for the case where the face model reconstruction model refers to 3DMM, the process of generating the first face model refers to a three-dimensional face reconstruction process based on 3DMM.

[0086] The first face key point position information corresponding to the first face model includes position information corresponding to each key point in the first face model. The position information corresponding to any key point in the first face model is used to indicate the position coordinates of the any key point in the first face model. After the first face model is generated based on the first generation parameter, the face model topology of the first face model can be obtained, and then the first face key point position information corresponding to the first face model can be obtained by analyzing the face model topology of the generated first face model. Illustratively, the way of analyzing the face model topology of the generated first face model is to perform data scanning on the face model topology of the generated first face model.

[0087] The embodiments of the present application do not limit the representation form of the position information corresponding to any key point in the first face model. Illustratively, the position information corresponding to any key point in the first face model is directly the position coordinates of the any key point in the first face model; or the position information corresponding to any key point in the first face model includes an index of the any key point in the first face model and the position coordinates pointed to by the index of the any key point. The position coordinates pointed to by the index of the any key point are the position coordinates of the any key point. It should be noted that when the first face model is a three-dimensional face model, the position coordinates of the key points in the first face model are three-dimensional coordinates (x, y, z); when the first face model is a two-dimensional face model, the position coordinates of the key points in the first face model are two-dimensional coordinates (x, y).

[0088] The type and number of key points in the first face model are set according to experience or flexibly adjusted according to application scenarios, and the embodiments of the present application do not limit this. In an illustrative embodiment, the first face model generated based on the first generation parameter can include a specified number of basis points. The specified number of basis points are used to constitute the first face model, and the specified number is a pre-set number which can be flexibly adjusted according to application scenarios. The key points in the first face model can refer to all the basis points in the first face model, or can refer to part of the basis points in the first face model, and the embodiments of the present application do not limit this.

[0089] In step 202, based on the first face key point position information, a first target function for updating the first generation parameter is obtained, and the first target function is a function matched with a reconstruction direction of the first face model.

[0090] After obtaining the first face key point position information corresponding to the first face model, based on the first face key point position information, a first target function for updating the first generation parameter is obtained. In the embodiment of the present application, by updating the generation parameter, the automatic reconstruction of the face model is realized, and the efficiency of the face model reconstruction is relatively high. The first target function is a function matched with a reconstruction direction of the first face model, and the reconstruction direction of the first face model is the reconstruction direction of the first face model. The reconstruction direction of the first face model is set according to experience or is flexibly adjusted according to application requirements, which is not limited in the embodiment of the present application. Since the first target function is a function matched with the reconstruction direction of the first face model, the first face model is reconstructed by using the first target function, which is beneficial to improve the quality of the reconstructed face model.

[0091] The embodiment of the present application does not limit the obtaining method of the first target function, which can be flexibly adjusted according to the actual application scene. In a possible implementation manner, the process of obtaining the first target function for updating the first generation parameter based on the first face key point position information includes the following steps 2021 and 2022:

[0092] Step 2021: based on the first face key point position information, a first loss function and a second loss function are obtained, the first loss function is used to constrain the overall face feature of the reconstructed face model, and the second loss function is used to optimize the reference feature of the reconstructed face model, and the reference feature is different from the overall face feature.

[0093] The first target function is obtained based on the first loss function for constraining the overall face feature of the reconstructed face model and the second loss function for optimizing the reference feature of the reconstructed face model. That is, based on the first target function, the overall face feature of the reconstructed face model can be constrained and the reference feature of the reconstructed face model can be optimized. Wherein, the reference feature is different from the overall face feature. Next, the obtaining process of the first loss function and the second loss function will be introduced respectively.

[0094] I. First loss function

[0095] The first loss function is used to constrain the overall facial features of the reconstructed face model, so as to avoid unnecessary overall changes in the subsequent reconstruction process and avoid losing identity features. In a possible implementation manner, based on the first face key point position information, the implementation manner of the first loss function is as follows: reference face key point position information corresponding to a reference face model is obtained; and the first loss function is obtained based on the first face key point position information and the reference face key point position information.

[0096] The reference face model is used to provide a constraint direction for the overall facial features of the reconstructed face model. The type of the reference face model is not limited in the embodiments of the present application, and can be flexibly set according to actual application scenarios. Exemplarily, the actual application scenarios include, but are not limited to, the following two kinds:

[0097] Application scenario one: directly reconstructing the first face model.

[0098] In this application scenario, a reconstructed face model similar in appearance to the first face model can be obtained. In an exemplary embodiment, in this application scenario, the reference face model is an original face model of the first face model. The original face model of the first face model refers to a face model directly relied on to obtain the first face model. In an exemplary embodiment, the first face model itself is the original face model of the first face model. In an exemplary embodiment, the first face model is a model obtained by reconstructing the original face model of the first face model at least once.

[0099] Application scenario two: reconstructing the first face model based on a template face model.

[0100] Exemplarily, in this application scenario two, the reference face model is a template face model. The template face model can provide a reconstruction direction for the reconstruction process of the first face model, and the template face model is selected according to experience or flexibly adjusted according to actual application scenarios, which is not limited in the embodiments of the present application. Exemplarily, the template face model and the first face model are face models of the same type, for example, the template face model and the first face model are both three-dimensional faces. Exemplarily, the template face model is a face model with a certain specific style, such as a realistic style, an ancient style, a cartoon style, etc. Based on the template face model with a specific style, the first face model can be reconstructed in the specific style. The reconstructed face model is similar in appearance to the first face model and has the specific style. In this application scenario two, a reconstructed face model similar in appearance to the first face model and similar in aesthetics to the template face model can be obtained.

[0101] The reference face key point position information includes position information corresponding to each key point in the reference face model. The position information corresponding to any key point in the reference face model indicates the position coordinates of the any key point in the reference face model. For example, the position information corresponding to any key point in the reference face model directly indicates the position coordinates of the any key point in the reference face model; or the position information corresponding to any key point in the reference face model includes an index of the any key point in the reference face model and position coordinates pointed to by the index of the any key point. The position coordinates pointed to by the index of the any key point are the position coordinates of the any key point.

[0102] In an example embodiment, when the reference face model is the first face model itself, the first face key point position information is directly used as the reference face key point position information. In an example embodiment, when the reference face model is a face model other than the first face model, the reference face key point position information is obtained by analyzing the reference face model.

[0103] After obtaining the reference face key point position information, a first loss function for constraining the overall face feature of the reconstructed face model is obtained based on the first face key point position information and the reference face key point position information. In a possible implementation, the implementation process of obtaining the first loss function based on the first face key point position information and the reference face key point position information is as follows: first target key point position information is obtained based on the first face key point position information; second target key point position information is obtained based on the reference face key point position information; and the first loss function is obtained based on the first target key point position information and the second target key point position information.

[0104] The first target key point position information includes position information corresponding to a target key point in the first face model; the second target key point position information includes position information corresponding to a target key point in the reference face model; and the target key point in the reference face model and the target key point in the first face model correspond to each other. In an example embodiment, the target key point in the reference face model and the target key point in the first face model correspond to each other means that the number of target key points in the reference face model is the same as the number of target key points in the first face model, and for each target key point in the reference face model, there is a target key point with the same semantic in the first face model.

[0105] It should be noted that the number of target key points in the face model (the reference face model or the first face model) is one or more, which is not limited in the embodiments of the present application. For example, the number of target key points in the face model is 86, and the 86 target key points in the face model are as shown in the following table. Figure 3as shown.

[0106] The embodiments of the present application do not limit the manner of determining the target key points in the first face model and the target key points in the reference face model, as long as the target key points in the reference face model and the target key points in the first face model correspond to each other. For example, the manner of determining the target key points in the first face model and the target key points in the reference face model is as follows: determining a selection condition; taking the key points in the first face model that meet the selection condition as the target key points in the first face model; and taking the key points in the reference face model that meet the selection condition as the target key points in the reference face model. The selection condition is set according to experience or is flexibly adjusted according to an application scenario, and the embodiments of the present application do not limit this. For example, the key points that meet the selection condition are the key points with a specified semantic, and the specified semantic is set according to experience or is flexibly adjusted according to an application scenario.

[0107] For example, the number and semantic of each key point in the reference face model can be the same as those of each key point in the first face model, or can be different from those of each key point in the first face model, and the embodiments of the present application do not limit this.

[0108] In exemplary embodiments, in different application scenarios, the target key points in the face model (the reference face model or the first face model) can be the same or different, and the embodiments of the present application do not limit this. For example, in an application scenario of directly reconstructing the first face model, the target key points in the face model are the first specified key points in the face model; and in an application scenario of reconstructing the first face model based on a template face model, the target key points in the face model are the second specified key points in the face model. The first specified key points in the face model can be the same as or different from the second specified key points in the face model.

[0109] The position information corresponding to any target key point in the first face model is used to indicate the position coordinates of the any target key point in the first face model. For example, the position information corresponding to any target key point in the first face model is directly the position coordinates of the any target key point in the first face model; or the position information corresponding to any target key point in the first face model includes the index of the any target key point in the first face model and the position coordinates pointed to by the index of the any target key point. The position coordinates pointed to by the index of the any target key point are the position coordinates of the any target key point.

[0110] The position information corresponding to any target key point in the reference face model indicates the position coordinates of the any target key point in the reference face model. For example, the position information corresponding to any target key point in the reference face model directly indicates the position coordinates of the any target key point in the reference face model; or the position information corresponding to any target key point in the reference face model includes an index of the any target key point in the first target face model and the position coordinates pointed to by the index of the any target key point. The position coordinates pointed to by the index of the any target key point are the position coordinates of the any target key point.

[0111] In the embodiments of the present application, since the key points in the first face model include the target key points, the first face key point position information includes the position information corresponding to each key point in the first face model, and therefore in the first face key point position information, the position information corresponding to each target key point in the first face model can be extracted to obtain the first target key point position information. Similarly, since the key points in the reference face model include the target key points, the reference face key point position information includes the position information corresponding to each key point in the reference face model, and therefore in the reference face key point position information, the position information corresponding to each target key point in the reference face model can be extracted to obtain the second target key point position information.

[0112] After obtaining the first target key point position information and the second target key point position information, the first loss function is obtained based on the first target key point position information and the second target key point position information. In a possible implementation manner, the manner of obtaining the first loss function based on the first target key point position information and the second target key point position information is as follows: the position coordinates of the target key points in the first face model are determined based on the first target key point position information; the position coordinates of the target key points in the reference face model are determined based on the second target key point position information; and the first loss function is obtained based on the difference between the position coordinates of the target key points in the first face model and the position coordinates of the corresponding target key points in the reference face model.

[0113] In the example embodiment, in the application scenario of directly reconstructing the first face model, the reference face model is the original face model of the first face model, and the second target key point position information includes the position information of the target key point in the original face model of the first face model. Assuming that the position information of the target key point in the first face model and the position information of the target key point in the original model of the first face model both include the index of the target key point and the position coordinates pointed by the index of the target key point, and the index of a certain target key point in the original model of the first face model is the same as the index of the corresponding target key point in the first face model. In this case, based on the first target key point position information and the second target key point position information, the process of obtaining the first loss function is based on formula 2:

[0114] (Formula 2)

[0115] Wherein, represents the first loss function obtained when the reference face model is the original face model of the first face model; represents the first generation parameter; represents the generation parameter corresponding to the original face model of the first face model; N1 (N1 is an integer not less than 1) represents the number of target key points; i1 represents the index of any target key point; Q represents the set of indexes of each target key point determined based on the first target key point position information or the second target key point position information; represents the position coordinates of the target key point with index i1 in the original face model of the first face model based on the second target key point position information; represents the position coordinates of the target key point with index i1 in the first face model based on the first target key point position information.

[0116] It should be noted that for the case that the original face model of the first face model is the first face model itself, the first loss function calculated based on the above formula 2 is 0. It should be noted that in the case of multiple updates of the first generation parameter, the second target key point position information remains unchanged in the process of calculating the first loss function required in each update process.

[0117] In the example embodiment, in the application scenario of reconstructing the first face model based on the template face model, the reference face model is the template face model, and the second target key point position information includes the position information of the target key point in the template face model. Assuming that the position information of the target key point in the first face model and the position information of the target key point in the template face model both include the index of the target key point and the position coordinates pointed by the index of the target key point, and the index of a target key point in the template face model is different from the index of the corresponding target key point in the first face model. In this case, based on the first target key point position information and the second target key point position information, the process of obtaining the first loss function is implemented based on formula 3:

[0118] (Formula 3)

[0119] Wherein, represents the first loss function obtained when the reference face model is the template face model; represents the first generation parameter; T represents the template face model; M1 (M1 is an integer not less than 1) represents the number of target key points; represents the set of indexes of the target key points in the first face model determined based on the first target key point position information; represents the set of indexes of the target key points in the template face model determined based on the second target key point position information; t1 represents the index of any target key point in the first face model; r1 represents the index of any target key point in the template face model; represents the position coordinates of the target key point with the index t1 in the first face model determined based on the first target key point position information; represents the position coordinates of the target key point with the index r1 in the template face model determined based on the second target key point position information, and the target key point with the index t1 in the first face model and the target key point with the index r1 in the template face model correspond to each other.

[0120] The first loss function obtained when the reference face model is the template face model can make the overall face features of the reconstructed face model as close as possible to the overall face features of the template face model, maintain the main features of the face model, and reserve sufficient change space for the face model by pre-setting multiple target key points.

[0121] II. Second loss function

[0122] The second loss function is used to optimize the reference feature of the reconstructed face model, wherein the reference feature is different from the overall face feature. The reference feature is set according to experience or is flexibly adjusted according to an application scenario, and embodiments of the present application do not limit this. In different application scenarios, the reference feature and the second loss function are different. Embodiments of the present application take two application scenarios including directly reconstructing the first face model and reconstructing the first face model based on the template face model as examples. Next, the reference feature and the second loss function in the two different application scenarios are introduced respectively.

[0123] Application scenario one: directly reconstructing the first face model.

[0124] In this application scenario, the reference face model on which the first loss function is based is the original face model of the first face model. Exemplarily, in this application scenario, the reference feature includes a local feature, and the second loss function includes a local adjustment loss function, which is directly obtained based on the first face key point position information.

[0125] The local feature includes a feature of at least one part (such as cheeks, a nose, a mouth, a chin, a philtrum, eyes, etc.) in the face model. For the case where the reference feature includes a local feature, the second loss function includes a local adjustment loss function, and the process of obtaining the second loss function is the process of obtaining the local adjustment loss function. The local adjustment loss function is used to optimize the feature of at least one part in the reconstructed face model. The number of local adjustment loss functions included in the second loss function is one or more, and embodiments of the present application do not limit this. For the case where the number of local adjustment loss functions is more than one, different local adjustment loss functions may be used to optimize the feature of the same part in the reconstructed face model, or may be used to optimize the feature of different parts in the reconstructed face model, and embodiments of the present application do not limit this.

[0126] In the application scenario of directly reconstructing the first face model, the local adjustment loss function is directly obtained based on the first face key point position information corresponding to the first face model. In this case, the second loss function can be obtained without relying on other face models, and the efficiency of obtaining the second loss function is high.

[0127] In an exemplary embodiment, the local adjustment loss function includes at least one of a cheek adjustment loss function, a first nose adjustment loss function, a first mouth adjustment loss function, a second nose adjustment loss function, a chin adjustment loss function, a third nose adjustment loss function, a philtrum adjustment loss function, an eye adjustment loss function, a fourth nose adjustment loss function, and a second mouth adjustment loss function.

[0128] In each of the above local adjustment loss functions, different local adjustment loss functions can be used to optimize the features of different local parts (for example, the cheek adjustment loss function is used to optimize the features of the cheeks, and the first nose adjustment loss function is used to optimize the features of the nose), and different local adjustment loss functions can also be used to optimize the features of the same local part (for example, the first nose adjustment loss function, the second nose adjustment loss function, the third nose adjustment loss function, and the fourth nose adjustment loss function are all used to optimize the features of the nose). It should be noted that, for the case where different local adjustment loss functions are used to optimize the features of the same local part, different local adjustment loss functions optimize the features of the same local part from different angles, for example, the first nose adjustment loss function, the second nose adjustment loss function, the third nose adjustment loss function, and the fourth nose adjustment loss function optimize the features of the nose from four different angles.

[0129] The specific case of the local adjustment loss function is flexibly set according to actual needs, and the embodiments of the present application do not limit this. According to the difference of the specific case of the local adjustment loss function, the process of obtaining the local adjustment loss function is different. Next, the acquisition processes of the cheek adjustment loss function, the first nose adjustment loss function, the first mouth adjustment loss function, the second nose adjustment loss function, the chin adjustment loss function, the third nose adjustment loss function, the philtrum adjustment loss function, the eye adjustment loss function, the fourth nose adjustment loss function, and the second mouth adjustment loss function are introduced respectively.

[0130] 1. Cheek adjustment loss function

[0131] The cheek adjustment loss function is used to optimize the features of the cheeks in the reconstructed face model. In a possible implementation, based on the first face key point position information, the process of obtaining the cheek adjustment loss function is as follows: based on the first face key point position information, cheek left key point position information and cheek right key point position information are obtained; based on the cheek left key point position information and the cheek right key point position information, the cheek adjustment loss function is obtained.

[0132] The cheek left key point position information includes position information corresponding to each cheek left key point in the first face model, and the cheek left key point refers to a key point in the first face model that is used to indicate the left cheek. The cheek right key point position information includes position information corresponding to each cheek right key point in the first face model, and the cheek right key point refers to a key point in the first face model that is used to indicate the right cheek. It should be noted that which key point or which key points are used to indicate the left cheek and which key point or which key points are used to indicate the right cheek are set according to experience, and the embodiments of the present application do not limit this.

[0133] Since the key points in the first face model include the left cheek key point and the right cheek key point, and the first face model key point information includes the position information corresponding to each key point in the first face model, in the first face key point position information, the position information corresponding to each left cheek key point in the first face model and the position information corresponding to each right cheek key point in the first face model can be extracted, so as to obtain the left cheek key point position information and the right cheek key point position information.

[0134] Exemplarily, the position information corresponding to any left cheek key point in the first face model includes the index of the left cheek key point and the position coordinates pointed by the index of the left cheek key point, the position information corresponding to any right cheek key point in the first face model includes the index of the right cheek key point and the position coordinates pointed by the index of the right cheek key point, and the left cheek key point in the first face model corresponds to the right cheek key point one by one. The number of left cheek key points in the first face model and the number of right cheek key points are the same, which can be one or multiple. In this case, the process of obtaining the cheek adjustment loss function based on the left cheek key point position information and the right cheek key point position information is realized based on formula 4:

[0135] (Formula 4)

[0136] Wherein, represents the cheek adjustment loss function; represents the first generation parameter; ( represents the number of left cheek key points (or the number of right cheek key points) which is an integer not less than 1; represents the set of indexes of each left cheek key point determined based on the left cheek key point position information; represents the set of indexes of each right cheek key point determined based on the right cheek key point position information.

[0137] i2 represents the index of any left cheek key point; j2 represents the index of any right cheek key point; represents the position coordinates of the left cheek key point with index i2 in the first face model determined based on the left cheek key point position information; represents the position coordinates of the right cheek key point with index j2 in the first face model determined based on the right cheek key point position information, and the right cheek key point with index j2 is the right cheek key point corresponding to the left cheek key point with index i2.

[0138] Exemplarily, the cheek left side key point and the cheek right side key point in the first face model are as shown in FIG. 8A. Figure 4 As shown in FIG. 8B, the cheek adjustment loss function can pull the key points on the left and right sides of the cheeks closer, thereby achieving the effect of slimming the face.

[0139] 2. The first nose adjustment loss function

[0140] The first nose adjustment loss function is used to optimize the features of the nose in the reconstructed face model from the perspective of pulling the distance between the left and right sides of the nose. In a possible implementation manner, based on the first face key point position information, the process of obtaining the first nose adjustment loss function is as follows: based on the first face key point position information, nose left side key point position information and nose right side key point position information are obtained; and based on the nose left side key point position information and the nose right side key point position information, the first nose adjustment loss function is obtained.

[0141] The nose left side key point position information includes position information corresponding to each nose left side key point in the first face model, and the nose left side key point refers to a key point in the first face model for indicating the left side of the nose. The nose right side key point position information includes position information corresponding to each nose right side key point in the first face model, and the nose right side key point refers to a key point in the first face model for indicating the right side of the nose. It should be noted that which key point or which key points are used to indicate the left side of the nose and which key point or which key points are used to indicate the right side of the nose are set according to experience, and the embodiments of the present application do not limit this.

[0142] Since the key points in the first face model include the nose left side key points and the nose right side key points, and the first face key point information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each nose left side key point in the first face model and position information corresponding to each nose right side key point in the first face model can be extracted, thereby obtaining the nose left side key point position information and the nose right side key point position information.

[0143] Exemplarily, the position information corresponding to any left-nose key point in the first face model comprises an index of the any left-nose key point and a position coordinate pointed by the index of the any left-nose key point, the position information corresponding to any right-nose key point in the first face model comprises an index of the any right-nose key point and a position coordinate pointed by the index of the any right-nose key point, and the left-nose key point in the first face model corresponds to the right-nose key point in a one-to-one manner. The number of the left-nose key points in the first face model is the same as the number of the right-nose key points, and can be one or multiple. In this case, the process of obtaining the nose adjustment loss function based on the left-nose key point position information and the right-nose key point position information is implemented based on formula 5:

[0144] (Formula 5)

[0145] wherein, represents the first nose adjustment loss function; represents the first generation parameter; is an integer greater than or equal to 1) represents the number of the left-nose key points (or the number of the right-nose key points); represents a set of indexes of the left-nose key points determined based on the left-nose key point position information; represents a set of indexes of the right-nose key points determined based on the right-nose key point position information.

[0146] i3 represents an index of any left-nose key point; j3 represents an index of any right-nose key point; represents a position coordinate of the left-nose key point with the index i3 in the first face model determined based on the left-nose key point position information; represents a position coordinate of the right-nose key point with the index j3 in the first face model determined based on the right-nose key point position information, and the right-nose key point with the index j3 is the right-nose key point corresponding to the left-nose key point with the index i3.

[0147] Exemplarily, the left-nose key point and the right-nose key point in the first face model are as shown in Figure 5 The first nose adjustment loss function can narrow the key points on the left and right sides of the nose, thereby achieving the effect of slimming the nose.

[0148] 3. First mouth adjustment loss function

[0149] ​The first mouth adjustment loss function is used to optimize the features of the mouth in the reconstructed face model from the perspective of thinning the lips. In a possible implementation, based on the first face key point position information, the process of obtaining the first mouth adjustment loss function is as follows: based on the first face key point position information, obtaining upper lip upper key point position information, upper lip lower key point position information, lower lip upper key point position information, and lower lip lower key point position information; and based on the upper lip upper key point position information, the upper lip lower key point position information, the lower lip upper key point position information, and the lower lip lower key point position information, obtaining the first mouth adjustment loss function.

[0150] The upper lip upper key point position information includes position information corresponding to each upper lip upper key point in the first face model, and the upper lip upper key point refers to a key point in the first face model that is used to indicate the upper side of the upper lip. The upper lip lower key point position information includes position information corresponding to each upper lip lower key point in the first face model, and the upper lip lower key point refers to a key point in the first face model that is used to indicate the lower side of the upper lip. It should be noted that which key point or key points are used to indicate the upper side of the upper lip and which key point or key points are used to indicate the lower side of the upper lip are set according to experience, and embodiments of the present application do not limit this.

[0151] The lower lip upper key point position information includes position information corresponding to each lower lip upper key point in the first face model, and the lower lip upper key point refers to a key point in the first face model that is used to indicate the upper side of the lower lip. The lower lip lower key point position information includes position information corresponding to each lower lip lower key point in the first face model, and the lower lip lower key point refers to a key point in the first face model that is used to indicate the lower side of the lower lip. It should be noted that which key point or key points are used to indicate the upper side of the lower lip and which key point or key points are used to indicate the lower side of the lower lip are set according to experience, and embodiments of the present application do not limit this.

[0152] Since the key points in the first face model include the upper-lip upper-side key point, the upper-lip lower-side key point, the lower-lip upper-side key point and the lower-lip lower-side key point, and the first face key point position information includes the position information corresponding to each key point in the first face model, in the first face key point position information, the position information corresponding to each upper-lip upper-side key point in the first face model, the position information corresponding to each upper-lip lower-side key point in the first face model, the position information corresponding to each lower-lip upper-side key point in the first face model and the position information corresponding to each lower-lip lower-side key point in the first face model can be extracted, so as to obtain the upper-lip upper-side key point position information, the upper-lip lower-side key point position information, the lower-lip upper-side key point position information and the lower-lip upper-side key point position information.

[0153] Exemplarily, the position information corresponding to any upper-lip upper-side key point in the first face model includes the index of the upper-lip upper-side key point and the position coordinates pointed by the index of the upper-lip upper-side key point; the position information corresponding to any upper-lip lower-side key point in the first face model includes the index of the upper-lip lower-side key point and the position coordinates pointed by the index of the upper-lip lower-side key point; and the upper-lip upper-side key point in the first face model corresponds to the upper-lip lower-side key point in one-to-one manner.

[0154] The position information corresponding to any lower-lip upper-side key point in the first face model includes the index of the lower-lip upper-side key point and the position coordinates pointed by the index of the lower-lip upper-side key point; the position information corresponding to any lower-lip lower-side key point in the first face model includes the index of the lower-lip lower-side key point and the position coordinates pointed by the index of the lower-lip lower-side key point; and the lower-lip upper-side key point in the first face model corresponds to the lower-lip lower-side key point in one-to-one manner.

[0155] Exemplarily, the number of upper-lip upper-side key points, the number of upper-lip lower-side key points, the number of lower-lip upper-side key points and the number of lower-lip lower-side key points in the first face model are all the same, which can be one or multiple. Based on the upper-lip upper-side key point position information, the upper-lip lower-side key point position information, the lower-lip upper-side key point position information and the lower-lip upper-side key point position information, the process of obtaining the first mouth adjustment loss function is realized based on formula 6:

[0156] (Formula 6)

[0157] Wherein, represents the first mouth adjustment loss function; represents the first generation parameter; ( represents the number of upper-lip upper-side key points (or the number of upper-lip lower-side key points, or the number of lower-lip upper-side key points, or the number of lower-lip lower-side key points) in the first face model; represents a set of indexes of each upper-lip upper-side key point determined based on the upper-lip upper-side key point position information; represents a set of indexes of each upper-lip lower-side key point determined based on the upper-lip lower-side key point position information; represents a set of indexes of each lower-lip upper-side key point determined based on the lower-lip upper-side key point position information; represents a set of indexes of each lower-lip lower-side key point determined based on the lower-lip lower-side key point position information.

[0158] i4 represents an index of any upper-lip upper-side key point; j4 represents an index of any upper-lip lower-side key point; represents a position coordinate of the upper-lip upper-side key point with the index i4 in the first face model determined based on the upper-lip upper-side key point position information; represents a position coordinate of the upper-lip lower-side key point with the index j4 in the first face model determined based on the upper-lip lower-side key point position information, the upper-lip lower-side key point with the index j4 being the upper-lip lower-side key point corresponding to the upper-lip upper-side key point with the index i4.

[0159] m4 represents an index of any lower-lip upper-side key point; n4 represents an index of any lower-lip lower-side key point; represents a position coordinate of the lower-lip upper-side key point with the index m4 in the first face model determined based on the lower-lip upper-side key point position information; represents a position coordinate of the lower-lip lower-side key point with the index n4 in the first face model determined based on the lower-lip lower-side key point position information, the lower-lip lower-side key point with the index n4 being the lower-lip lower-side key point corresponding to the lower-lip upper-side key point with the index m4.

[0160] Exemplarily, the upper-lip upper-side key point, the upper-lip lower-side key point, the lower-lip upper-side key point and the lower-lip lower-side key point in the first face model are as shown in FIG. 2A. Figure 6 The first mouth adjustment loss function can narrow the key points on the upper and lower sides of the upper lip and narrow the key points on the upper and lower sides of the lower lip, thereby achieving the effect of thinning the lips.

[0161] 4. Second nose adjustment loss function

[0162] The second nose adjustment loss function is used to optimize the feature of the nose in the reconstructed face model from the perspective of straightening the nose bridge. In a possible implementation, based on the first face key point position information, the process of obtaining the second nose adjustment loss function is as follows: based on the first face key point position information, nose bridge key point position information is obtained; nose bridge guide point position information corresponding to the nose bridge key point position information is obtained; and based on the nose bridge key point position information and the nose bridge guide point position information, the second nose adjustment loss function is obtained.

[0163] The nose bridge key point position information includes position information corresponding to each nose bridge key point in the first face model. The nose bridge key point refers to a key point in the first face model that is used to indicate the nose bridge. It should be noted that which key point or which key points are used to indicate the nose bridge is set according to experience, and the embodiments of the present application do not limit this. Since the key points in the first face model include the nose bridge key points, and the first face key point position information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each nose bridge key point in the first face model can be extracted, thereby obtaining the nose bridge key point position information.

[0164] The nose bridge guide point position information corresponding to the nose bridge key point position information is used to provide a reconstruction direction for the feature of the nose bridge in the reconstructed face model. The nose bridge guide point position information includes position information corresponding to each nose bridge guide point. The position information corresponding to any nose bridge guide point is used to indicate the position coordinates of the any nose bridge guide point. The nose bridge key point and the nose bridge guide point are in one-to-one correspondence. That is, the number of nose bridge key points is the same as the number of nose bridge guide points. Exemplarily, the nose bridge guide point can also be referred to as a nose bridge anchor point.

[0165] Exemplarily, the position coordinates of any nose bridge guide point are set according to experience, in which case, the nose bridge guide point position information can be obtained by analyzing the position coordinates of each nose bridge guide point set according to experience. Exemplarily, the position coordinates of any nose bridge guide point are determined according to the position coordinates of the nose bridge key point corresponding to the any nose bridge guide point, in which case, the position coordinates of each nose bridge guide point are first determined based on the position coordinates of each nose bridge key point, and then the nose bridge guide point position information is obtained by analyzing the position coordinates of each nose bridge guide point determined.

[0166] In an example embodiment, the manner of determining the position coordinate of the nose bridge guide point corresponding to any of the nose bridge key points based on the position coordinate of the any of the nose bridge key points is: translating the position indicated by the position coordinate of the any of the nose bridge key points along a first specified direction by a first reference distance, and taking the coordinate at the moved-to position as the position coordinate of the nose bridge guide point corresponding to the any of the nose bridge key points. The first specified direction and the first reference distance are set according to experience or flexibly adjusted according to application requirements, and the embodiments of the present application do not limit this. Illustratively, the first specified direction is a direction perpendicular to the nose bridge in the first face model and close to the forehead.

[0167] Illustratively, the position information corresponding to any of the nose bridge key points in the first face model includes the index of the any of the nose bridge key points and the position coordinate pointed to by the index of the any of the nose bridge key points, and the position information corresponding to any of the nose bridge guide points is the position coordinate of the any of the nose bridge guide points. The number of nose bridge key points and the number of nose bridge guide points are the same, and can be one or multiple. In this case, the process of obtaining the second nose adjustment loss function based on the nose bridge key point position information and the nose bridge guide point position information is implemented based on formula 7:

[0168] (Formula 7)

[0169] wherein, represents the second nose adjustment loss function; represents the first generation parameter; ( represents the number of nose bridge key points (or the number of nose bridge guide points) and is an integer not less than 1; represents a set of indexes of each nose bridge key point determined based on the nose bridge key point position information; represents a set of each nose bridge guide point.

[0170] i5 represents the index of any of the nose bridge key points; v5 represents any of the nose bridge guide points; represents the position coordinate of the nose bridge key point with index i5 in the first face model determined based on the nose bridge key point position information; represents the position coordinate of the nose bridge guide point v5, which is the nose bridge guide point corresponding to the nose bridge key point with index i5.

[0171] Illustratively, the nose bridge key points in the first face model and the nose bridge guide points corresponding to the nose bridge key points in the first face model are as shown in Figure 7 Figure 7 ​In the first face model, the nose bridge key points are filled with black, and the nose bridge guide points corresponding to the nose bridge key points are filled with white. The second nose adjustment loss function can achieve the effect of straightening the nose bridge by pulling the nose bridge key points close to the nose bridge guide points.

[0172] 5. Chin adjustment loss function

[0173] The chin adjustment loss function is used to optimize the features of the chin in the reconstructed face model. In a possible implementation, based on the first face key point position information, the process of obtaining the chin adjustment loss function is as follows: based on the first face key point position information, obtain chin key point position information; obtain chin guide point position information corresponding to the chin key point position information; and based on the chin key point position information and the chin guide point position information, obtain the second nose adjustment loss function.

[0174] The chin key point position information includes position information corresponding to each chin key point in the first face model, and the chin key point refers to a key point in the first face model that is used to indicate the chin. It should be noted that which key point or which key points are used to indicate the chin is set according to experience, and the embodiments of the present application do not limit this. Since the key points in the first face model include chin key points, and the first face key point position information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each chin key point in the first face model can be extracted, thereby obtaining the chin key point position information.

[0175] The chin guide point position information corresponding to the chin key point position information is used to provide a reconstruction direction for the features of the chin in the reconstructed face model. The chin guide point position information includes position information corresponding to each chin guide point. The position information corresponding to any chin guide point is used to indicate the position coordinates of the chin guide point. The chin key point and the chin guide point correspond one-to-one. That is, the number of chin key points is the same as the number of chin guide points. Exemplarily, the chin guide point can also be referred to as a chin anchor point.

[0176] Exemplarily, the position coordinates of any chin guide point are set according to experience, in which case, the chin guide point position information can be obtained by analyzing the position coordinates of each chin guide point set according to experience. Exemplarily, the position coordinates of any chin guide point are determined according to the position coordinates of the chin key point corresponding to the chin guide point, in which case, first, the position coordinates of each chin guide point are determined based on the position coordinates of each chin key point, and then the chin guide point position information is obtained by analyzing the determined position coordinates of each chin guide point.

[0177] In an exemplary embodiment, the method for determining the position coordinates of the chin guide point corresponding to any chin key point based on its position coordinates is as follows: the position indicated by the position coordinates of any chin key point is translated along a second specified direction by a second reference distance, and the coordinates of the moved position are taken as the position coordinates of the chin guide point corresponding to the chin key point. The second specified direction and the second reference distance are set based on experience or can be flexibly adjusted according to application requirements; this embodiment does not limit them.

[0178] For example, the location information corresponding to any chin keypoint in the first facial model includes the index of the chin keypoint and the coordinates pointed to by the index. The location information corresponding to any chin guide point is the coordinates of the chin guide point. The number of chin keypoints and the number of chin guide points are the same; both can be one or multiple. In this case, the process of obtaining the chin adjustment loss function based on the chin keypoint location information and the chin guide point location information is implemented according to Formula 8:

[0179] (Formula 8)

[0180] in, This represents the chin adjustment loss function; Indicates the first generation parameter; ( (Integer not less than 1) represents the number of key points on the chin (or the number of chin guide points); This represents the set of indices of each chin keypoint determined based on the chin keypoint location information; This represents the set of all chin guide points.

[0181] i6 represents the index of any chin key point; v6 represents any chin leader point; This indicates the position coordinates of the chin keypoint with index i6 in the first facial model determined based on the chin keypoint location information; This indicates the coordinates of the chin guide point v6, which is the bridge of the nose guide point corresponding to the chin key point with index i6.

[0182] For example, the chin key points in the first facial model and the corresponding chin guide points in the first facial model are as follows: Figure 8 As shown, in Figure 8 In the first facial model, the chin key points are filled with black, while the corresponding chin guide points are filled with white. The chin adjustment loss function can achieve a raised chin effect by pulling the chin key points closer to the chin guide points.

[0183] 6. The third nose adjustment loss function

[0184] The third nose adjustment loss function is used to optimize the features of the nose in the reconstructed face model from the perspective of the thin nostril. In a possible implementation, based on the first face key point position information, the process of obtaining the third nose adjustment loss function is as follows: based on the first face key point position information, obtaining nostril left side key point position information, nostril right side key point position information, nostril upper side key point position information, and nostril lower side key point position information; and based on the nostril left side key point position information, the nostril right side key point position information, the nostril upper side key point position information, and the nostril lower side key point position information, obtaining the third nose adjustment loss function.

[0185] The nostril left side key point position information includes position information corresponding to each nostril left side key point in the first face model, and the nostril left side key point refers to a key point in the first face model for indicating the left side of the nostril, including the left side of the left nostril and the left side of the right nostril. The nostril right side key point position information includes position information corresponding to each nostril right side key point in the first face model, and the nostril right side key point refers to a key point in the first face model for indicating the right side of the nostril, including the right side of the left nostril and the right side of the right nostril. It should be noted that which key point or key points are used to indicate the left side of the nostril and which key point or key points are used to indicate the right side of the nostril are set according to experience, and the embodiments of the present application do not limit this. The nostril left side key point in the first face model corresponds to the nostril right side key point in one-to-one correspondence.

[0186] The nostril upper side key point position information includes position information corresponding to each nostril upper side key point in the first face model, and the nostril upper side key point refers to a key point in the first face model for indicating the upper side of the nostril, including the upper side of the left nostril and the upper side of the right nostril. The nostril lower side key point position information includes position information corresponding to each nostril lower side key point in the first face model, and the nostril lower side key point refers to a key point in the first face model for indicating the lower side of the nostril, including the lower side of the left nostril and the lower side of the right nostril. It should be noted that which key point or key points are used to indicate the upper side of the nostril and which key point or key points are used to indicate the lower side of the nostril are set according to experience, and the embodiments of the present application do not limit this. The nostril upper side key point in the first face model corresponds to the nostril lower side key point in one-to-one correspondence.

[0187] Since the key points in the first face model include the left nostril key point, the left nostril key point, the upper nostril key point, and the lower nostril key point, and the first face key point position information includes the position information corresponding to each key point in the first face model, in the first face key point position information, the position information corresponding to each left nostril key point in the first face model, the position information corresponding to each left nostril key point in the first face model, the position information corresponding to each upper nostril key point in the first face model, and the position information corresponding to each lower nostril key point in the first face model can be extracted, thereby obtaining the left nostril key point position information, the right nostril key point position information, the upper nostril key point position information, and the lower nostril key point position information.

[0188] Exemplarily, the position information corresponding to any left nostril key point in the first face model includes the index of the any left nostril key point and the position coordinates pointed to by the index of the any left nostril key point; the position information corresponding to any right nostril key point in the first face model includes the index of the any right nostril key point and the position coordinates pointed to by the index of the any right nostril key point; the left nostril key point in the first face model corresponds to the right nostril key point in one-to-one.

[0189] The position information corresponding to any upper nostril key point in the first face model includes the index of the any upper nostril key point and the position coordinates pointed to by the index of the any upper nostril key point; the position information corresponding to any lower nostril key point in the first face model includes the index of the any lower nostril key point and the position coordinates pointed to by the index of the any lower nostril key point; the upper nostril key point in the first face model corresponds to the lower nostril key point in one-to-one.

[0190] Exemplarily, the number of left nostril key points, the number of right nostril key points, the number of upper nostril key points, and the number of lower nostril key points in the first face model are all the same, which can be one or multiple. Based on the left nostril key point position information, the right nostril key point position information, the upper nostril key point position information, and the lower nostril key point position information, the process of obtaining the third nose adjustment loss function is implemented based on formula 9:

[0191] (Formula 9)

[0192] Wherein, represents the third nose adjustment loss function; represents the first generation parameter; ( represents the number of left nostril key points (or the number of right nostril key points, or the number of upper nostril key points, or the number of lower nostril key points) of the first face model; represents a set of indexes of each left nostril key point determined based on the left nostril key point position information; represents a set of indexes of each right nostril key point determined based on the right nostril key point position information; represents a set of indexes of each upper nostril key point determined based on the upper nostril key point position information; represents a set of indexes of each lower nostril key point determined based on the lower nostril key point position information.

[0193] i7 represents an index of any left nostril key point; j7 represents an index of any right nostril key point; represents a position coordinate of the left nostril key point with index i7 in the first face model determined based on the left nostril key point position information; represents a position coordinate of the right nostril key point with index j7 in the first face model determined based on the right nostril key point position information, the right nostril key point with index j7 being the right nostril key point corresponding to the left nostril key point with index i7.

[0194] m7 represents an index of any upper nostril key point; n7 represents an index of any lower nostril key point; represents a position coordinate of the upper nostril key point with index m7 in the first face model determined based on the upper nostril key point position information; represents a position coordinate of the lower nostril key point with index n7 in the first face model determined based on the lower nostril key point position information, the lower nostril key point with index n7 being the lower nostril key point corresponding to the upper nostril key point with index m7.

[0195] Exemplarily, the left nostril key point, the right nostril key point, the upper nostril key point and the lower nostril key point in the first face model are as shown in FIG. 7. Figure 9 The third nose adjusting loss function can pull the key points on the left and right sides of the nostrils closer and pull the key points on the upper and lower sides of the nostrils closer, thereby achieving the effect of slimming the nostrils.

[0196] 7. Philtrum adjusting loss function

[0197] The inter-nasal adjustment loss function is used to optimize features in the human in the reconstructed face model. In one possible implementation, based on the first face key point position information, the process of obtaining the inter-nasal adjustment loss function is as follows: based on the first face key point position information, mouth key point position information and inter-nasal key point position information are obtained; based on the mouth key point position information and the inter-nasal key point position information, the inter-nasal adjustment loss function is obtained.

[0198] The mouth key point position information includes position information corresponding to each mouth key point in the first face model, and the mouth key point refers to a key point in the first face model that indicates the mouth. The inter-nasal key point position information is used to indicate the position coordinates of the inter-nasal key point in the first face model. The inter-nasal key point refers to a key point in the first face model that indicates the nose. Exemplarily, the number of inter-nasal key points is one. It should be noted that which key point or key points are used to indicate the mouth and which key point or key points are used to indicate the nose are set empirically, and the embodiments of the present application do not limit this.

[0199] Since the key points in the first face model include mouth key points and inter-nasal key points, and the first face key point position information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each mouth key point in the first face model and position information corresponding to the inter-nasal key point in the first face model can be extracted, thereby obtaining the mouth key point position information and the inter-nasal key point position information.

[0200] Exemplarily, the position information corresponding to any mouth key point in the first face model includes the index of the any mouth key point and the position coordinates pointed to by the index of the any mouth key point, and the inter-nasal key point position information includes the index of the inter-nasal key point and the position coordinates pointed to by the index of the inter-nasal key point. The number of mouth key points in the first face model is one or more, and the number of inter-nasal key points in the first face model is one. In this case, the process of obtaining the inter-nasal adjustment loss function based on the mouth key point position information and the inter-nasal key point position information is implemented based on formula 10:

[0201] (Formula 10)

[0202] Wherein, represents the inter-nasal adjustment loss function; represents the first generation parameter; (Formula 10) represents the number of mouth key points, which is an integer not less than 1; a set of indexes representing each mouth key point determined based on the mouth key point position information; i8 represents an index of any mouth key point; k8 represents an index of a philtrum key point determined based on the philtrum key point position information; a position coordinate of a mouth key point with index i8 in the first face model determined based on the mouth key point position information; a position coordinate of a philtrum key point determined based on the philtrum key point position information; an average coordinate of position coordinates of each mouth key point in the first face model.

[0203] Exemplarily, the mouth key points and the philtrum key points in the first face model are as shown in FIG. 2. Figure 10 As shown in FIG. 2, the philtrum adjustment loss function can pull the distance between the average position of the mouth key points and the philtrum key point, so as to achieve the effect of pulling up the mouth and pulling down the philtrum.

[0204] 8. Eye adjustment loss function

[0205] The eye adjustment loss function is used to optimize the features of the eyes in the reconstructed face model. In a possible implementation, based on the first face key point position information, the process of obtaining the eye adjustment loss function is as follows: based on the first face key point position information, obtaining eye lower eyelid key point position information; obtaining eye lower eyelid guide point position information corresponding to the eye lower eyelid key point position information; based on the eye lower eyelid key point position information and the eye lower eyelid guide point position information, obtaining the eye adjustment loss function.

[0206] The eye lower eyelid key point position information includes position information corresponding to each eye lower eyelid key point in the first face model. The eye lower eyelid key point refers to a key point in the first face model for indicating the eye lower eyelid, and the eye lower eyelid includes the left eye lower eyelid and the right eye lower eyelid. It should be noted that which key point or which key points are used to indicate the eye lower eyelid is set according to experience, and the embodiments of the present application do not limit this. Since the key points in the first face model include the eye lower eyelid key points, and the first face key point position information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each eye lower eyelid key point in the first face model can be extracted, so as to obtain the eye lower eyelid key point position information.

[0207] The eye lower eyelid key point position information corresponds to eye lower eyelid guide point position information, which is used to provide a reconstruction direction for a feature of the eye lower eyelid in the reconstructed face model. The eye lower eyelid guide point position information includes position information corresponding to each eye lower eyelid guide point. The position information corresponding to any eye lower eyelid guide point is used to indicate the position coordinates of the any eye lower eyelid guide point. The eye lower eyelid key point and the eye lower eyelid guide point correspond to each other. That is, the number of eye lower eyelid key points is the same as the number of eye lower eyelid guide points. Exemplarily, the eye lower eyelid guide point can also be referred to as an eye lower eyelid anchor point.

[0208] Exemplarily, the position coordinates of any eye lower eyelid guide point are set according to experience, and in this case, the eye lower eyelid guide point position information can be obtained by analyzing the position coordinates of each eye lower eyelid guide point set according to experience. Exemplarily, the position coordinates of any eye lower eyelid guide point are determined according to the position coordinates of the eye lower eyelid key point corresponding to the any eye lower eyelid guide point, and in this case, the position coordinates of each eye lower eyelid guide point are first determined based on the position coordinates of each eye lower eyelid key point, and then the eye lower eyelid guide point position information is obtained by analyzing the determined position coordinates of each eye lower eyelid guide point.

[0209] In an exemplary embodiment, based on the position coordinates of any eye lower eyelid key point, the position coordinates of the eye lower eyelid guide point corresponding to the any eye lower eyelid key point are determined in the following manner: the position indicated by the position coordinates of the any eye lower eyelid key point is translated along a third specified direction by a third reference distance, and the coordinates at the moved position are taken as the position coordinates of the eye lower eyelid guide point corresponding to the any eye lower eyelid key point. The third specified direction and the third reference distance are set according to experience or are flexibly adjusted according to application requirements, and the embodiments of the present application do not limit them.

[0210] Exemplarily, the position information corresponding to any eye lower eyelid key point in the first face model includes the index of the any eye lower eyelid key point and the position coordinates pointed to by the index of the any eye lower eyelid key point, and the position information corresponding to any eye lower eyelid guide point is the position coordinates of the any eye lower eyelid guide point. The number of eye lower eyelid key points and the number of eye lower eyelid guide points are the same, and can each be one or multiple. In this case, based on the eye lower eyelid key point position information and the eye lower eyelid guide point position information, the process of obtaining the eye adjustment loss function is implemented based on formula 11:

[0211] (Formula 11)

[0212] Wherein, represents the eye adjustment loss function. represents the first generation parameter; represents the number of eye lower eyelid key points (or the number of eye lower eyelid guide points) which is an integer no less than 1; represents a set of indexes of each eye lower eyelid key point determined based on eye lower eyelid key point position information; represents a set of each eye lower eyelid guide point.

[0213] i9 represents an index of any eye lower eyelid key point; v9 represents any eye lower eyelid guide point; represents a position coordinate of the eye lower eyelid key point with index i9 in the first face model determined based on eye lower eyelid key point position information; represents a position coordinate of the eye lower eyelid guide point v9, which is the eye lower eyelid guide point corresponding to the eye lower eyelid key point with index i9.

[0214] Exemplarily, the eye lower eyelid key points in the first face model and the eye lower eyelid guide points corresponding to the eye lower eyelid key points in the first face model are as shown in Figure 11 Figure 11 In the figure, the eye lower eyelid key points in the first face model are filled with black, and the eye lower eyelid guide points corresponding to the eye lower eyelid key points in the first face model are filled with white. The eye adjustment loss function can achieve the effect of enlarging the eyes by pulling down the eye lower eyelid key points close to the eye lower eyelid guide points.

[0215] 9、Fourth nose adjustment loss function

[0216] The fourth nose adjustment loss function is used to optimize the features of the nose in the reconstructed face model from the perspective of pulling back the nose. In a possible implementation manner, based on the first face key point position information, the process of obtaining the fourth nose adjustment loss function is as follows: based on the first face key point position information, obtaining nose key point position information; obtaining nose key point position information corresponding nose guide point position information; based on the nose key point position information and the nose guide point position information, obtaining the fourth nose adjustment loss function.

[0217] ​​The position information of the nasal key points includes position information corresponding to each of the nasal key points in the first face model. The nasal key points refer to key points in the first face model that are used to indicate the nose. It should be noted that which key point or key points are used to indicate the nose is set according to experience, and embodiments of the present application do not limit this. Since the key points in the first face model include the nasal key points, and the first face key point position information includes position information corresponding to each of the key points in the first face model, in the first face key point position information, position information corresponding to each of the nasal key points in the first face model can be extracted, thereby obtaining the position information of the nasal key points.

[0218] The position information of the nasal guide points corresponding to the position information of the nasal key points is used to provide a reconstruction direction for the features in the nose of the reconstructed face model. The position information of the nasal guide points includes position information corresponding to each of the nasal guide points. The position information corresponding to any nasal guide point is used to indicate the position coordinates of the any nasal guide point. The nasal key points and the nasal guide points correspond to each other in one-to-one manner. That is, the number of the nasal key points is the same as the number of the nasal guide points. Exemplarily, the nasal guide points can also be referred to as nasal anchor points.

[0219] Exemplarily, the position coordinates of any nasal guide point are set according to experience, in which case, the position information of the nasal guide points can be obtained by analyzing the position coordinates of the nasal guide points set according to experience. Exemplarily, the position coordinates of any nasal guide point are determined according to the position coordinates of the nasal key point corresponding to the any nasal guide point, in which case, the position coordinates of the nasal guide points are first determined based on the position coordinates of the nasal key points, and then the position information of the nasal guide points is obtained by analyzing the determined position coordinates of the nasal guide points.

[0220] In an exemplary embodiment, based on the position coordinates of any nasal key point, the position coordinates of the nasal guide point corresponding to the any nasal key point are determined in the following manner: the position indicated by the position coordinates of the any nasal key point is translated along a fourth specified direction by a fourth reference distance, and the coordinates of the position moved to are taken as the position coordinates of the nasal guide point corresponding to the any nasal key point. The fourth specified direction and the fourth reference distance are set according to experience, or are flexibly adjusted according to application requirements, and embodiments of the present application do not limit this.

[0221] For example, the location information corresponding to any keypoint in the nose in the first facial model includes the index of that keypoint and the coordinates of the position pointed to by that index. The location information corresponding to any guide point in the nose is the coordinates of that guide point. The number of keypoints and guide points in the nose are the same; both can be one or multiple. In this case, the process of obtaining the fourth nose adjustment loss function based on the location information of the keypoints and guide points in the nose is implemented according to Formula 12:

[0222] (Formula 12)

[0223] in, This represents the loss function adjusted for the fourth nose; Indicates the first generation parameter; ( (Integer not less than 1) represents the number of key points in the nose (or the number of guiding points in the nose); This represents the set of indices of each nasal midpoint determined based on the location information of the nasal midpoints. This represents the set of all guiding points in the nose.

[0224] i 10 Represents the index of any key point in the nose; v 10 Indicates any guiding point in the nose; This indicates that the index i in the first facial model determined based on the location information of key points in the nose is... 10 The coordinates of the key points in the nose; Indicates the nasal guiding point v 10 Location coordinates, mid-nasal guide point v 10 For index i 10 The key point in the nose corresponds to the guiding point in the nose.

[0225] For example, the mid-nasal keypoints in the first facial model and the mid-nasal guide points corresponding to the mid-nasal keypoints in the first facial model are as follows: Figure 12 As shown, in Figure 12 In the first facial model, the mid-nasal keypoints are filled with black, while the corresponding mid-nasal guide points are filled with white. The fourth nose adjustment loss function can achieve the effect of pulling the mid-nasal keypoints closer to the mid-nasal guide points.

[0226] 10. Adjusting the loss function using the second mouth

[0227] The second mouth adjustment loss function is used to optimize the features of the mouth in the reconstructed face model from the perspective of pulling down the lower lip. In one possible implementation, based on the first face key point position information, the process of obtaining the second mouth adjustment loss function is as follows: based on the first face key point position information, obtain lower lip key point position information; obtain lower lip guide point position information corresponding to the lower lip key point position information; and based on the lower lip key point position information and the lower lip guide point position information, obtain the eye adjustment loss function.

[0228] The lower lip key point position information includes position information corresponding to each lower lip key point in the first face model. The lower lip key point refers to a key point in the first face model that is used to indicate the lower lip. It should be noted that which key point or which key points are used to indicate the lower lip is set empirically, and embodiments of the present application do not limit this. Since the key points in the first face model include lower lip key points, and the first face key point position information includes position information corresponding to each key point in the first face model, in the first face key point position information, position information corresponding to each lower lip key point in the first face model can be extracted, thereby obtaining the lower lip key point position information.

[0229] The lower lip guide point position information corresponding to the lower lip key point position information is used to provide a reconstruction direction for the features of the lower lip in the reconstructed face model. The lower lip guide point position information includes position information corresponding to each lower lip guide point. The position information corresponding to any lower lip guide point is used to indicate the position coordinates of the lower lip guide point. The lower lip key point and the lower lip guide point correspond one-to-one. That is, the number of lower lip key points is the same as the number of lower lip guide points. Exemplarily, the lower lip guide point can also be referred to as a lower lip anchor point.

[0230] Exemplarily, the position coordinates of any lower lip guide point are set empirically, in which case the lower lip guide point position information can be obtained by analyzing the position coordinates of each lower lip guide point set empirically. Exemplarily, the position coordinates of any lower lip guide point are determined according to the position coordinates of the lower lip key point corresponding to the lower lip guide point, in which case the position coordinates of each lower lip guide point are first determined based on the position coordinates of each lower lip key point, and then the lower lip guide point position information is obtained by analyzing the position coordinates of each lower lip guide point determined.

[0231] In an example embodiment, the manner of determining the position coordinate of the lower lip guide point corresponding to the any lower lip key point based on the position coordinate of the any lower lip key point is: translating the position indicated by the position coordinate of the any lower lip key point along a fifth specified direction by a fifth reference distance, and taking the coordinate at the moved-to position as the position coordinate of the lower lip guide point corresponding to the any lower lip key point. The fifth specified direction and the fifth reference distance are set according to experience or flexibly adjusted according to application requirements, and the embodiments of the present application do not limit this.

[0232] Exemplarily, the position information corresponding to any lower lip key point in the first face model includes the index of the any lower lip key point and the position coordinate pointed to by the index of the any lower lip key point, and the position information corresponding to any lower lip guide point is the position coordinate of the any lower lip guide point. The number of lower lip key points and the number of lower lip guide points are the same, and can each be one or multiple. In this case, the process of obtaining the second mouth adjustment loss function based on the lower lip key point position information and the lower lip guide point position information is implemented based on formula 13:

[0233] (Formula 13)

[0234] Wherein, represents the second mouth adjustment loss function; represents the first generation parameter; ( is an integer not less than 1) represents the number of lower lip key points (or the number of lower lip guide points); represents a set of indexes of each lower lip key point determined based on the lower lip key point position information; represents a set of each lower lip guide point.

[0235] i 11 represents the index of any lower lip key point; v 11 represents any lower lip guide point; represents the position coordinate of the lower lip key point with index i 11 in the first face model determined based on the lower lip key point position information; represents the position coordinate of the lower lip guide point v 11 , the lower lip guide point v 11 is the lower lip guide point corresponding to the lower lip key point with index i 11 .

[0236] Exemplarily, the lower lip key point in the first face model and the lower lip guide point corresponding to the lower lip key point in the first face model are as shown in Figure 13 , in Figure 13In the first face model, the lower lip key point is filled with black, and the lower lip guide point corresponding to the lower lip key point in the first face model is filled with white. The second mouth adjustment loss function can achieve the effect of pulling the lower lip back by pulling the lower lip key point back close to the lower lip guide point.

[0237] For the case where the local adjustment loss function includes at least one of the above cheek adjustment loss function, the first nose adjustment loss function, the first mouth adjustment loss function, the second nose adjustment loss function, the chin adjustment loss function, the third nose adjustment loss function, the philtrum adjustment loss function, the eye adjustment loss function, the fourth nose adjustment loss function, and the second mouth adjustment loss function, the above-mentioned manner of obtaining each local adjustment loss function and the specific case of the local adjustment loss function are referred to, and the local adjustment loss function can be obtained.

[0238] Application scenario two: reconstructing the first face model based on the template face model.

[0239] In this application scenario two, the reference face model for obtaining the first loss function is the template face model. Illustratively, in this application scenario two, the reference feature includes the proportion feature, the second loss function includes the reference proportion loss function, and the reference proportion loss function is obtained based on the first face key point position information and the template face model corresponding template face key point position information.

[0240] The proportion feature includes at least one reference proportion (such as horizontal proportion, vertical proportion, depth proportion, etc.) in the face model. For the case where the reference feature includes the proportion feature, the second loss function includes the reference proportion loss function, and the process of obtaining the second loss function is the process of obtaining the reference proportion loss function. The reference proportion loss function is used to optimize at least one reference proportion in the reconstructed face model. The number of reference proportion loss functions included in the second loss function is one or more, which is not limited by the embodiments of the present application.

[0241] The reference proportion loss function is obtained based on the first face key point position information and the template face model corresponding template face key point position information. In this case, in addition to being able to provide a constraint direction for the overall face feature of the reconstructed face model, the template face model can also provide an optimization direction for the proportion feature of the reconstructed face model.

[0242] The template face model corresponding template face key point position information is obtained by analyzing the template face model. The template face model corresponding template face key point position information includes position information corresponding to each key point in the template face model. The position information corresponding to any key point in the template face model indicates the position coordinates of the any key point in the template face model. For example, the position information corresponding to any key point in the template face model directly indicates the position coordinates of the any key point in the template face model. Alternatively, the position information corresponding to any key point in the template face model includes an index of the any key point in the template face model and the position coordinates pointed to by the index of the any key point. The position coordinates pointed to by the index of the any key point are the position coordinates of the any key point.

[0243] The reference ratio loss function is used to optimize the corresponding reference ratio in the reconstructed face model based on the reference ratio in the template face model. The type and number of the reference ratio loss function are not limited in the embodiments of the present application, which can be set according to experience or adjusted flexibly according to application requirements.

[0244] In a possible implementation, based on the first face key point position information and the template face key point position information, the process of obtaining the reference ratio loss function is as follows: based on the first face key point position information and the template face key point position information, the ratio difference between the reference ratio in the first face model and the corresponding reference ratio in the template face model is obtained, and based on the ratio difference, the reference ratio loss function is obtained. The reference ratio loss function is related to the type of the reference ratio to be considered. For example, the reference ratio to be considered includes at least one of the horizontal ratio, the vertical ratio, the depth ratio and the length-width ratio. In this case, the reference ratio loss function includes at least one of the horizontal ratio loss function, the vertical ratio loss function, the depth ratio loss function and the length-width ratio loss function.

[0245] The specific case of the reference ratio loss function is flexibly set according to actual requirements, which is not limited in the embodiments of the present application. According to the difference of the specific case of the reference ratio loss function, the process of obtaining the reference ratio loss function is different. Next, the processes of obtaining the horizontal ratio loss function, the vertical ratio loss function, the depth ratio loss function and the length-width ratio loss function are introduced respectively.

[0246] A: Horizontal ratio loss function

[0247] The horizontal proportion loss function is used to optimize the corresponding horizontal proportion in the reconstructed face model based on the horizontal proportion in the template face model. In one possible implementation, based on the first face key point position information and the template face key point position information, the process of obtaining the horizontal proportion loss function is as follows: based on the first face key point position information, obtaining a first horizontal distance corresponding to the first face model and at least one second horizontal distance corresponding to the first face model; based on the template face key point position information, obtaining a third horizontal distance corresponding to the template face model and at least one fourth horizontal distance corresponding to the template face model; based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance and the at least one fourth horizontal distance, obtaining the horizontal proportion loss function.

[0248] The first horizontal distance corresponding to the first face model refers to the reference horizontal distance of the first face model. For example, the first horizontal distance corresponding to the first face model refers to the horizontal distance between the left temple center point and the right temple center point in the first face model. For example, based on the first face key point position information, the position coordinates of the left temple center point and the right temple center point in the first face model can be determined, and then the horizontal distance between the left temple center point and the right temple center point in the first face model can be determined, that is, the first horizontal distance corresponding to the first face model can be determined.

[0249] The at least one second horizontal distance corresponding to the first face model is set according to experience or adjusted flexibly according to application scenarios. For example, the at least one second horizontal distance includes at least one of the horizontal distance between the left temple center point and the left eye corner in the first face model, the horizontal distance between the left eye corner and the right eye corner in the left eye in the first face model, the horizontal distance between the right eye corner and the left eye corner in the left eye in the first face model, the horizontal distance between the right eye corner and the left eye corner in the right eye in the first face model, the horizontal distance between the right eye corner and the right temple center point in the first face model, the horizontal distance between the left end point of the nose and the right end point of the nose in the first face model, and the horizontal distance between the left corner of the mouth and the right corner of the mouth in the first face model. Since the first face key point position information includes the position information of each key point in the first face model, the at least one second horizontal distance can be obtained based on the first face key point position information.

[0250] According to the introduction of the horizontal distance, any horizontal distance is determined based on two key points, and the indices of the two key points form an index pair, that is, the calculation of any horizontal distance needs a key point index pair.

[0251] Exemplarily, it is assumed that the at least one second horizontal distance includes a horizontal distance between a left temple center point in the first face model and a left eye corner of the left eye, a horizontal distance between the left eye corner of the left eye and a right eye corner of the left eye in the first face model, a horizontal distance between the right eye corner of the left eye and a left eye corner of the right eye in the first face model, a horizontal distance between the left eye corner of the right eye and a right eye corner of the right eye in the first face model, a horizontal distance between the right eye corner of the right eye and the right temple center point in the first face model, a horizontal distance between a left side end point of a nose and a right side end point of the nose in the first face model, and a horizontal distance between a left corner of a mouth and a right corner of the mouth in the first face model. In this case, the first face model corresponds to 7 second horizontal distances.

[0252] The first horizontal distance corresponding to the first face model is denoted as a horizontal distance between the left temple center point in the first face model and the left eye corner of the left eye is denoted as a horizontal distance between the left eye corner of the left eye and the right eye corner of the left eye in the first face model is denoted as a horizontal distance between the right eye corner of the left eye and the left eye corner of the right eye in the first face model is denoted as a horizontal distance between the left eye corner of the right eye and the right eye corner of the right eye in the first face model is denoted as a horizontal distance between the right eye corner of the right eye and the right temple center point in the first face model is denoted as a horizontal distance between the left side end point of the nose and the right side end point of the nose in the first face model is denoted as a horizontal distance between the left corner of the mouth and the right corner of the mouth in the first face model is denoted as The first horizontal distance corresponding to the first face model and the at least one second horizontal distance are as shown in Figure 14 .

[0253] The third horizontal distance corresponding to the template face model refers to a reference horizontal distance of the template face model, and the third horizontal distance corresponding to the template face model corresponds to the first horizontal distance corresponding to the first face model; the at least one fourth horizontal distance corresponding to the template face model one-to-one corresponds to the at least one second horizontal distance corresponding to the first face model. The number of the fourth horizontal distances corresponding to the template face model is the same as the number of the second horizontal distances corresponding to the first face model. For implementation manners of obtaining the third horizontal distance corresponding to the template face model and the at least one fourth horizontal distance corresponding to the template face model based on the template face key point position information, reference can be made to the implementation manners of obtaining the first horizontal distance corresponding to the first face model and the at least one second horizontal distance corresponding to the first face model based on the first face key point position information, which will not be described herein again.

[0254] In the example embodiment, the design concept of the horizontal proportion loss function is that: it is expected that the proportion of any second horizontal distance corresponding to the first face model on the first horizontal distance corresponding to the first face model is roughly consistent with the proportion of the fourth horizontal distance corresponding to the template face model on the third horizontal distance corresponding to the template face model, so as to optimize the corresponding horizontal proportion in the reconstructed face model based on the horizontal proportion in the template face model.

[0255] In the example, based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance and the at least one fourth horizontal distance, the process of obtaining the horizontal proportion loss function is implemented based on formula 14:

[0256]

[0257] (Formula 14)

[0258] wherein, represents the horizontal proportion loss function; M2 represents the number of the fourth horizontal distances corresponding to the template face model (or the number of the second horizontal distances corresponding to the first face model); represents a set of key point index pairs required for calculating each second horizontal distance corresponding to the first face model respectively; represents a key point index pair required for calculating any second horizontal distance corresponding to the first face model; represents a set of key point index pairs required for calculating each fourth horizontal distance corresponding to the template face model respectively; represents a key point index pair required for calculating any fourth horizontal distance corresponding to the template face model.

[0259] represents a key point index pair required for calculating the third horizontal distance corresponding to the template face model; represents a key point index pair required for calculating the first horizontal distance corresponding to the first face model; represents a second horizontal distance corresponding to the first face model calculated based on the key point index pair represents a fourth horizontal distance corresponding to the template face model calculated based on the key point index pair represents the third horizontal distance corresponding to the template face model; represents the first horizontal distance corresponding to the first face model; represents a second horizontal distance corresponding to the first face model calculated based on the key point index pair represents a fourth horizontal distance corresponding to the template face model calculated based on the key point index pair

[0260] In the above formula 14, is used to solve the key point index pair indicates two key points and horizontal distance between the two eyes, Based on formula 15, the following is calculated:

[0261] (Formula 15)

[0262] wherein, represents the horizontal component of the position coordinate of the key point . represents the horizontal component of the position coordinate of the key point . It should be noted that the calculation of , and in formula 14 is described in the calculation of , which will not be repeated here.

[0263] B: vertical proportion loss function

[0264] The vertical proportion loss function is used to optimize the corresponding vertical proportion in the reconstructed face model based on the vertical proportion in the template face model. In one possible implementation, based on the first face key point position information and the template face key point position information corresponding to the template face model, the process of obtaining the vertical proportion loss function includes the following steps 1 to step 5:

[0265] Step 1: Based on the first face key point position information, obtain the first vertical distance corresponding to the first face model, at least one second direct vertical distance corresponding to the first face model, and at least one second average vertical distance corresponding to the first face model.

[0266] The first vertical distance corresponding to the first face model refers to the reference vertical distance of the first face model. For example, the first vertical distance corresponding to the first face model refers to the vertical distance between the center point of the line connecting the right eye corner of the left eye and the left eye corner of the right eye in the first face model and the lowest point of the chin. For example, based on the first face key point position information, the position coordinates of the center point of the line connecting the right eye corner of the left eye and the left eye corner of the right eye in the first face model and the lowest point of the chin can be determined, and then the vertical distance between the center point of the line connecting the right eye corner of the left eye and the left eye corner of the right eye in the first face model and the lowest point of the chin can be determined, that is, the first vertical distance corresponding to the first face model can be determined.

[0267] The at least one second direct vertical distance corresponding to the first face model refers to a vertical distance directly calculated based on position coordinates of two key points in the first face model. For example, the at least one second direct vertical distance corresponding to the first face model includes at least one of a vertical distance between a highest point of a left eye and a lowest point of the left eye in the first face model, and a vertical distance between a highest point of a right eye and a lowest point of the right eye in the first face model. Since the first face key point position information includes position information of each key point in the first face model, the at least one second direct vertical distance can be obtained based on the first face key point position information.

[0268] According to the introduction of the at least one second direct vertical distance, any second direct vertical distance is determined based on two key points, and the indices of the two key points form an index pair, that is, the calculation of any second direct vertical distance needs an index pair of key points.

[0269] The at least one second average vertical distance corresponding to the first face model refers to a vertical distance calculated based on average positions of two local key points in the first face model. For example, the at least one second average vertical distance corresponding to the first face model includes at least one of a vertical distance between an average position of eye key points and an average position of nose key points in the first face model, a vertical distance between the average position of the nose key points and an average position of mouth key points in the first face model, and a vertical distance between the average position of the mouth key points and an average position of chin key points in the first face model. Since the first face key point position information includes position information of each key point in the first face model, the average positions of the key points in each local part can be obtained based on the first face key point position information, and then the at least one second average vertical distance can be obtained.

[0270] According to the introduction of the at least one second average vertical distance, any second average vertical distance is determined based on two average positions, and the two average positions form an average position pair, that is, the calculation of any second average vertical distance needs an average position pair.

[0271] For example, it is assumed that the at least one second direct vertical distance corresponding to the first face model includes a vertical distance between a highest point of a left eye and a lowest point of the left eye in the first face model, and a vertical distance between a highest point of a right eye and a lowest point of the right eye in the first face model; and the at least one second average vertical distance corresponding to the first face model includes at least one of a vertical distance between an average position of eye key points and an average position of nose key points in the first face model, a vertical distance between the average position of the nose key points and an average position of mouth key points in the first face model, and a vertical distance between the average position of the mouth key points and an average position of chin key points in the first face model.

[0272] the first vertical distance corresponding to the first face model is denoted as the vertical distance between the highest point of the left eye and the lowest point of the left eye in the first face model is denoted as the vertical distance between the highest point of the right eye and the lowest point of the right eye in the first face model is denoted as the vertical distance between the average position of the eye key point and the average position of the nose key point in the first face model is denoted as the vertical distance between the average position of the nose key point and the average position of the mouth key point in the first face model is denoted as the vertical distance between the average position of the mouth key point and the average position of the chin key point in the first face model is denoted as the first vertical distance corresponding to the first face model, the at least one second direct vertical distance, and the at least one second average vertical distance are as shown in Figure 15 .

[0273] Step 2: based on the template face key point position information, obtaining the third vertical distance corresponding to the template face model, the at least one fourth direct vertical distance corresponding to the template face model, and the at least one fourth average vertical distance corresponding to the template face model.

[0274] The third vertical distance corresponding to the template face model refers to the reference vertical distance of the template face model, and the third vertical distance corresponding to the template face model corresponds to the first vertical distance corresponding to the first face model; the at least one fourth direct vertical distance corresponding to the template face model corresponds to the at least one second direct vertical distance corresponding to the first face model one by one, and the number of the fourth direct vertical distances corresponding to the template face model is the same as the number of the second direct vertical distances corresponding to the first face model. The at least one fourth average vertical distance corresponding to the template face model corresponds to the at least one second average vertical distance corresponding to the first face model one by one, and the number of the fourth average vertical distances corresponding to the template face model is the same as the number of the second average vertical distances corresponding to the first face model. Based on the template face key point position information, the implementation of obtaining the third vertical distance corresponding to the template face model, the at least one fourth direct vertical distance corresponding to the template face model, and the at least one fourth average vertical distance corresponding to the template face model is referred to the implementation of obtaining the first vertical distance corresponding to the first face model, the at least one second direct vertical distance corresponding to the first face model, and the at least one second average vertical distance corresponding to the first face model based on the first face key point position information introduced in the above step 1, which will not be repeated here.

[0275] Step 3: Obtain a first vertical proportion sub-loss function based on the first vertical distance, the at least one second direct vertical distance, the third vertical distance, and the at least one fourth direct vertical distance.

[0276] The first vertical proportion sub-loss function is a part of the vertical proportion loss function. Illustratively, this step 3 is implemented based on equation 16:

[0277]

[0278] (Equation 16)

[0279] wherein, denotes the first vertical proportion sub-loss function; M 3-1 denotes the number of the fourth direct vertical distances corresponding to the template face model (or the number of the second direct vertical distances corresponding to the first face model); denotes a set of keypoint index pairs needed for calculating each of the second direct vertical distances corresponding to the first face model, respectively; denotes a keypoint index pair needed for calculating any of the second direct vertical distances corresponding to the first face model; denotes a set of keypoint index pairs needed for calculating each of the fourth direct vertical distances corresponding to the template face model, respectively; denotes a keypoint index pair needed for calculating any of the fourth direct vertical distances corresponding to the template face model.

[0280] denotes a keypoint index pair needed for calculating the third vertical distance corresponding to the template face model; denotes a keypoint index pair needed for calculating the first vertical distance corresponding to the first face model; denotes a keypoint index pair corresponding to the template face model based on which a fourth direct vertical distance is calculated; denotes the third vertical distance corresponding to the template face model; denotes the first vertical distance corresponding to the first face model; denotes a keypoint index pair corresponding to the first face model based on which a second direct vertical distance is calculated.

[0281] In the above equation 16, is used to solve the keypoint index pair denotes the vertical distance between the two keypoints and , which is calculated based on equation 17:

[0282] (Equation 17) ​

[0283] wherein, represents a vertical component of a position coordinate of a key point ; represents a vertical component of a position coordinate of a key point . It is to be noted that the calculation of , and refer to the calculation of , which will not be repeated here.

[0284] Step 4: obtaining a second vertical proportion sub-loss function based on the first vertical distance, the at least one second average vertical distance, the third vertical distance and the at least one fourth average vertical distance.

[0285] The second vertical proportion sub-loss function is another part of the vertical proportion loss function. In one possible implementation, this step 4 is implemented based on formula 18:

[0286]

[0287] (formula 18)

[0288] wherein, represents the second vertical proportion sub-loss function; M 3-2 represents the number of the fourth average vertical distances corresponding to the template face model (or the number of the second average vertical distances corresponding to the first face model); represents a set of average position pairs needed for calculating each of the second average vertical distances corresponding to the first face model respectively; represents an average position pair needed for calculating any of the second average vertical distances corresponding to the first face model; represents a set of average position pairs needed for calculating each of the fourth average vertical distances corresponding to the template face model respectively; represents an average position pair needed for calculating any of the fourth average vertical distances corresponding to the template face model.

[0289] represents a key point index pair needed for calculating the third vertical distance corresponding to the template face model; represents a key point index pair needed for calculating the first vertical distance corresponding to the first face model; represents one of the fourth average vertical distances corresponding to the template face model calculated based on the average position pair ; represents the third vertical distance corresponding to the template face model; represents the first vertical distance corresponding to the first face model; represents one of the fourth average vertical distances corresponding to the template face model calculated based on the average position pair The second average vertical distance corresponding to the calculated first face model.

[0290] The calculation of the above formula 18 is as shown in the calculation of the above formula 17. 、 、 and The calculation of the above formula 18 is as shown in the calculation of the above formula 17. The calculation of the above formula 18 is as shown in the calculation of the above formula 17.

[0291] Exemplarily, the second vertical proportion sub-loss function can constrain the overall smooth movement of the local (e.g., eyes, nose, mouth, chin, etc.) in the reconstruction process, and keep the local overall form unchanged, for example, keep the eye shape and mouth shape unchanged.

[0292] Step 5: Obtain a vertical proportion loss function based on the first vertical proportion sub-loss function and the second vertical proportion sub-loss function.

[0293] The vertical proportion loss function is composed of the first vertical proportion sub-loss function and the second vertical proportion sub-loss function. In an exemplary embodiment, the way to obtain the vertical proportion loss function based on the first vertical proportion sub-loss function and the second vertical proportion sub-loss function is: calculating a first product of the first vertical proportion sub-loss function and a first weight; calculating a second product of the second vertical proportion sub-loss function and a second weight; and taking the sum of the first product and the second product as the vertical proportion loss function. Exemplarily, the obtaining process of the vertical proportion loss function is implemented based on formula 19:

[0294] (Formula 19)

[0295] Wherein, represents the vertical proportion loss function; represents the first weight; represents the first vertical proportion sub-loss function; represents the first weight; represents the second vertical proportion sub-loss function. The first weight and the second weight can be set according to experience or flexibly adjusted according to application scenarios, and the embodiments of the present application do not limit this. Exemplarily, the first weight and the second weight are both set to 1, in which case the vertical proportion loss function refers to the sum of the first vertical proportion sub-loss function and the second vertical proportion sub-loss function.

[0296] In the example embodiment, the design concept of the vertical proportion loss function is that: the proportion of any second vertical distance (any second direct vertical distance or any second average vertical distance) corresponding to the first face model on the first vertical distance corresponding to the first face model is roughly consistent with the proportion of the fourth vertical distance corresponding to the template face model and corresponding to the any second vertical distance on the third vertical distance corresponding to the template face model, so as to optimize the corresponding vertical proportion in the reconstructed face model based on the vertical proportion in the template face model.

[0297] C: depth proportion loss function

[0298] The depth proportion loss function is used to optimize the corresponding depth proportion in the reconstructed face model based on the depth proportion in the template face model. In a possible implementation manner, based on the first face key point position information and the template face key point position information corresponding to the template face model, the process of obtaining the depth proportion loss function is: based on the first face key point position information, obtaining the first depth distance corresponding to the first face model and at least one second depth distance corresponding to the first face model; based on the template face key point position information, obtaining the third depth distance corresponding to the template face model and at least one fourth depth distance corresponding to the template face model; based on the first depth distance, the at least one second depth distance, the third depth distance and the at least one fourth depth distance, obtaining the depth proportion loss function.

[0299] The first depth distance corresponding to the first face model refers to the reference depth distance of the first face model. For example, the first depth distance corresponding to the first face model refers to the depth distance between the first nose key point on the nose and the first cheek key point on the cheek in the first face model. For example, based on the first face key point position information, the position coordinates of the first nose key point on the nose and the first cheek key point on the cheek in the first face model can be determined, and then the depth distance between the first nose key point on the nose and the first cheek key point on the cheek in the first face model can be determined, that is, the first depth distance corresponding to the first face model can be determined.

[0300] The first nose key point and the first cheek key point are set according to experience or flexibly adjusted according to application scenarios, and the embodiments of the present application are not limited thereto. For example, the calculation method of the depth distance between the two key points can refer to calculating the distance of the position coordinates of the two key points in the depth direction, for example, for the case that the first face model is a three-dimensional face model, the depth direction is the z-axis direction.

[0301] The at least one second depth distance corresponding to the first face model is set empirically or adjusted flexibly according to an application scenario. In addition, the number of the second depth distances corresponding to the first face model is one or more, which is not limited by the embodiments of the present application.

[0302] According to the introduction of the depth distance, any depth distance is determined based on two key points, and the indexes of the two key points form an index pair, that is, the calculation of any depth distance needs a key point index pair.

[0303] Exemplarily, assuming that the number of the second depth distances corresponding to the first face model is one, the one second depth distance is denoted as The first depth distance corresponding to the first face model is denoted as The first depth distance corresponding to the first face model and the at least one second depth distance are as shown in Figure 16 .

[0304] The third depth distance corresponding to the template face model refers to the reference depth distance of the template face model, and the third depth distance corresponding to the template face model corresponds to the first depth distance corresponding to the first face model; the at least one fourth depth distance corresponding to the template face model one-to-one corresponds to the at least one second depth distance corresponding to the first face model. The number of the fourth depth distances corresponding to the template face model is the same as the number of the second depth distances corresponding to the first face model. The implementation of obtaining the third depth distance corresponding to the template face model and the at least one fourth depth distance corresponding to the template face model based on the template face key point position information is referred to the implementation of obtaining the first depth distance corresponding to the first face model and the at least one second depth distance corresponding to the first face model based on the first face key point position information, which will not be described herein.

[0305] Exemplarily, based on the first depth distance, the at least one second depth distance, the third depth distance and the at least one fourth depth distance, the process of obtaining the depth proportion loss function is implemented based on formula 20:

[0306]

[0307] (Formula 20)

[0308] wherein, indicates the depth proportion loss function; M4 indicates the number of the fourth depth distances corresponding to the template face model (or the number of the second depth distances corresponding to the first face model); indicates a set of key point index pairs needed for calculating each second depth distance corresponding to the first face model; indicates a key point index pair needed for calculating any second depth distance corresponding to the first face model; a set of key point index pairs needed for calculating each fourth depth distance corresponding to the template face model; a key point index pair needed for calculating any fourth depth distance corresponding to the template face model.

[0309] a key point index pair needed for calculating a third depth distance corresponding to the template face model; a key point index pair needed for calculating a first depth distance corresponding to the first face model; a fourth depth distance corresponding to the template face model calculated based on the key point index pair a fourth depth distance corresponding to the template face model calculated based on the key point index pair a third depth distance corresponding to the template face model; a first depth distance corresponding to the first face model; a second depth distance corresponding to the first face model calculated based on the key point index pair a second depth distance corresponding to the first face model calculated based on the key point index pair

[0310] In the above formula 20, a horizontal distance between the two key points indicated by the key point index pair and is calculated based on the formula 21:

[0311] (formula 21)

[0312] wherein, represents a depth component of the position coordinate of the key point ; represents a depth component of the position coordinate of the key point . It is to be noted that the calculation of , and in the formula 20 can refer to the calculation of and will not be repeated here.

[0313] In the exemplary embodiments, the design concept of the depth proportion loss function is that the proportion of any second depth distance corresponding to the first face model in the first depth distance corresponding to the first face model is roughly consistent with the proportion of the fourth depth distance corresponding to the template face model and corresponding to the any second depth distance in the third depth distance corresponding to the template face model, so as to optimize the corresponding depth proportion in the reconstructed face model based on the depth proportion in the template face model.

[0314] D: aspect ratio loss function ​

[0315] The aspect ratio loss function is used to optimize the corresponding aspect ratio in the reconstructed face model based on the aspect ratio in the template face model. In one possible implementation, the process of obtaining the aspect ratio loss function based on the first face key point position information and the template face key point position information corresponding to the template face model is as follows: based on the first face key point position information, the reference vertical distance corresponding to the first face model and the at least one reference horizontal distance corresponding to the first face model are obtained; based on the template face key point position information, the reference vertical distance corresponding to the template face model and the at least one reference horizontal distance corresponding to the template face model are obtained; and based on the reference vertical distance corresponding to the first face model, the at least one reference horizontal distance corresponding to the first face model, the reference vertical distance corresponding to the template face model, and the at least one reference horizontal distance corresponding to the template face model, the aspect ratio loss function is obtained.

[0316] The reference vertical distance corresponding to the first face model and the at least one reference horizontal distance corresponding to the first face model are set according to experience or adjusted flexibly according to application scenarios. Illustratively, the reference vertical distance corresponding to the first face model refers to the vertical distance between the center point of the line connecting the right eye corner and the left eye corner of the right eye and the lowest point of the chin in the first face model. The at least one reference horizontal distance corresponding to the first face model includes at least one of the horizontal distance between the left temple center point and the right temple center point in the first face model and other customized horizontal distances. Since the first face key point position information includes the position information of each key point in the first face model, the reference vertical distance corresponding to the first face model and the at least one reference horizontal distance corresponding to the first face model can be obtained based on the first face key point position information.

[0317] Illustratively, it is assumed that the at least one reference horizontal distance corresponding to the first face model includes the horizontal distance between the left temple center point and the right temple center point in the first face model and other three customized horizontal distances. The reference vertical distance corresponding to the first face model is denoted as , the horizontal distance between the left temple center point and the right temple center point in the first face model is denoted as , and the other three customized horizontal distances are denoted as , and , respectively. Figure 17 The reference vertical distance corresponding to the first face model and the at least one reference horizontal distance are as shown in

[0318] The reference vertical distance corresponding to the template face model corresponds to the reference vertical distance corresponding to the first face model; and the at least one reference horizontal distance corresponding to the template face model one-to-one corresponds to the at least one reference horizontal distance corresponding to the first face model. The number of the reference horizontal distances corresponding to the template face model is the same as the number of the reference horizontal distances corresponding to the first face model. For implementation of obtaining the reference vertical distance corresponding to the template face model and the at least one reference horizontal distance corresponding to the template face model based on the template face key point position information, refer to the implementation of obtaining the reference vertical distance corresponding to the first face model and the at least one reference horizontal distance corresponding to the first face model based on the first face key point position information, which will not be described herein again.

[0319] Exemplarily, the process of obtaining the aspect ratio loss function based on the reference vertical distance corresponding to the first face model, the at least one reference horizontal distance corresponding to the first face model, the reference vertical distance corresponding to the template face model and the at least one reference horizontal distance corresponding to the template face model is implemented based on formula 22:

[0320]

[0321] (Formula 22)

[0322] wherein, represents the aspect ratio loss function; M5 represents the number of the reference horizontal distances corresponding to the template face model (or the number of the reference horizontal distances corresponding to the first face model); represents a set of key point index pairs required for calculating each reference horizontal distance corresponding to the first face model respectively; represents a key point index pair required for calculating any reference horizontal distance corresponding to the first face model; represents a set of key point index pairs required for calculating each reference horizontal distance corresponding to the template face model respectively; represents a key point index pair required for calculating any reference horizontal distance corresponding to the template face model.

[0323] represents a key point index pair required for calculating the reference vertical distance corresponding to the template face model; represents a key point index pair required for calculating the reference vertical distance corresponding to the first face model; represents a key point index pair based on which represents one reference horizontal distance corresponding to the template face model calculated based on the key point index pair represents the reference vertical distance corresponding to the template face model; represents the reference vertical distance corresponding to the first face model; represents a key point index pair based on which The calculated first face model corresponds to a reference horizontal distance.

[0324] In an example embodiment, the design concept of the length-width ratio loss function is that: the proportion of any reference horizontal distance corresponding to the first face model on the reference vertical distance corresponding to the first face model is expected to be roughly consistent with the proportion of the reference horizontal distance corresponding to the first face model on the reference vertical distance corresponding to the template face model, so as to optimize the corresponding length-width ratio in the reconstructed face model based on the length-width ratio in the template face model.

[0325] The reference ratio loss function includes at least one of the horizontal ratio loss function, the vertical ratio loss function, the depth ratio loss function, and the length-width ratio loss function. The reference ratio loss function can be obtained according to the above-mentioned manner of obtaining each reference ratio loss function and the specific conditions of the at least one reference ratio loss function.

[0326] In an example embodiment, in the application scenario of reconstructing the first face model based on the template face model, the reference features can also include local features, in which case the second loss function includes a local adjustment loss function. Unlike the manner of obtaining the local adjustment loss function in the application scenario of directly reconstructing the first face model, the local adjustment loss function is obtained based on the first face key point position information and the template face key point position information corresponding to the template face model in the application scenario of reconstructing the first face model based on the template face model.

[0327] The manner of obtaining the local adjustment loss function based on the first face key point position information and the template face key point position information corresponding to the template face model is not limited in the embodiments of the present application. For example, the cheek difference between the cheeks in the first face model and the cheeks in the template face model is obtained based on the first face key point position information and the template face key point position information corresponding to the template face model, and a cheek adjustment loss function is obtained based on the cheek difference; the eye difference between the eyes in the first face model and the eyes in the template face model is obtained based on the first face key point position information and the template face key point position information corresponding to the template face model, and an eye adjustment loss function is obtained based on the eye difference.

[0328] It should be noted that the embodiments of the present application are only described by taking the reference features including local features or ratio features as examples, and the embodiments of the present application are not limited thereto. In actual application process, other aspects of features can also be set as reference features according to requirements, such as style features.

[0329] Step 2022: based on the first loss function and the second loss function, a first target function for updating the first generation parameter is obtained.

[0330] After the first loss function and the second loss function are obtained, based on the first loss function and the second loss function, a first target function for updating the first generation parameter is obtained. The first target function is composed of multiple different loss functions, and different loss functions represent different functions. In an exemplary embodiment, the degree of adjustment of each loss function is determined by the weight value corresponding to the loss function. The weight value corresponding to the loss function can be a positive value or a negative value, and the embodiment of the present application does not limit this. If the weight value corresponding to the loss function is negative, it means adjusting in the opposite direction.

[0331] In an exemplary embodiment, based on the first loss function and the second loss function, the process of obtaining the first target function for updating the first generation parameter is as follows: calculating the product of the first loss function and the weight value corresponding to the first loss function; calculating the product of the second loss function and the weight value corresponding to the second loss function; and taking the sum of the products obtained above as the first target function. For example, the second loss function can include multiple sub-loss functions, and a sub-loss function can refer to a local adjustment loss function or a reference ratio loss function. In this case, the weight value corresponding to the second loss function includes the weight value corresponding to each sub-loss function. The process of calculating the product of the second loss function and the weight value corresponding to the second loss function refers to the process of calculating the product of each sub-loss function in the second loss function and the corresponding weight value.

[0332] In an exemplary embodiment, in the application scenario of directly reconstructing the first face model, it is assumed that the first loss function is calculated based on formula 2 , and the second loss function includes ten local adjustment loss functions, i.e., the cheek adjustment loss function , the first nose adjustment loss function , the first mouth adjustment loss function , the second nose adjustment loss function , the chin adjustment loss function , the third nose adjustment loss function , the philtrum adjustment loss function , the eye adjustment loss function , the fourth nose adjustment loss function , and the second mouth adjustment loss function . In this case, based on the first loss function and the second loss function, the process of obtaining the first target function for updating the first generation parameter is implemented based on formula 23:

[0333] (Formula 23)

[0334] wherein, denotes the first target function obtained in the application scenario of directly reconstructing the first face model; denotes the first generation parameter; denotes a set of weights of each loss function in the application scenario of directly reconstructing the first face model; denotes ~ the weight corresponding to the u-th (u is an integer not less than 1 and not greater than 11) loss function in the 11 loss functions; denotes ~ the u-th loss function in the 11 loss functions.

[0335] In an exemplary embodiment, in the application scenario of reconstructing the first face model based on the template face model, it is assumed that the first loss function is calculated based on Formula 3 , and the second loss function includes (G-1) reference ratio loss functions, wherein G is an integer not less than 1. In this case, based on the first loss function and the second loss function, the process of obtaining the first target function for updating the first generation parameter is implemented based on Formula 24:

[0336] (Formula 24)

[0337] wherein, denotes the first target function obtained in the application scenario of reconstructing the first face model based on the template face model; denotes the first generation parameter; denotes a set of weights of each loss function in the application scenario of reconstructing the first face model based on the template face model; T denotes the template face model; and G denotes the total number of the first loss function and the reference ratio loss function. denotes the weight corresponding to the o-th (o is an integer not less than 1) loss function in the G loss functions of the first loss function and the (G-1) reference ratio loss functions; denotes the o-th loss function in the G loss functions of the first loss function and the (G-1) reference ratio loss functions.

[0338] Exemplarily, in the process of reconstructing the first face model based on the template face model, the idea of designing the first objective function is to reconstruct the first face model with reference to the proportions of the provided template face model “three courts and five eyes”, so that the proportions of each part of the reconstructed face model are close to the template face model, while the overall characteristics of the original face model are maintained. Exemplarily, the process of reconstructing the first face model with reference to the proportions of the provided template face model “three courts and five eyes” so that the proportions of each part of the reconstructed face model are close to the template face model is as shown in FIG. 6. Figure 18

[0339] Whether it is the case of directly reconstructing the first face model or the case of reconstructing the first face model based on the template face model, the first face key point position information can be used to obtain the first objective function for updating the first generation parameter, and then step 203 is performed.

[0340] In step 203, the first generation parameter is updated using the first objective function to obtain a second generation parameter.

[0341] After obtaining the first objective function for updating the first generation parameter, the first generation parameter is updated using the first objective function to obtain a second generation parameter. The second generation parameter refers to the generation parameter obtained after the first generation parameter is updated once using the first objective function.

[0342] In one possible implementation, the process of updating the first generation parameter using the first objective function to obtain a second generation parameter is as follows: based on the first objective function, an update gradient for updating the first generation parameter is calculated; the first generation parameter is updated using the update gradient to obtain the second generation parameter.

[0343] Exemplarily, the process of updating the first generation parameter using the update gradient to obtain the second generation parameter is implemented based on formula 25:

[0344] (Formula 25)

[0345] wherein, represents the second generation parameter; represents the first generation parameter; represents the update gradient for updating the first generation parameter calculated based on the first objective function L; F represents an optimizer, and exemplarily, F is an Adam (Adaptive moment estimation) optimizer.

[0346] ​In a possible implementation, after obtaining the second generation parameter, it is determined whether the second generation parameter satisfies an update termination condition. If the second generation parameter satisfies the update termination condition, step 204 is performed. The update termination condition is set according to experience or is flexibly adjusted according to an application scenario, and embodiments of the present application do not limit this. Illustratively, the update termination condition includes any one of the following: the number of parameter updates performed when the second generation parameter is obtained reaches a threshold number; the first target function on which the second generation parameter is based converges.

[0347] In step 204, in response to the second generation parameter satisfying the update termination condition, a second face model is generated based on the second generation parameter.

[0348] When the second generation parameter satisfies the update termination condition, it indicates that it is not necessary to continue to update the second generation parameter. In this case, the second face model is generated based on the second generation parameter, and the second face model is the face model obtained after the first face model is reconstructed. Thus, the process of reconstructing the first face model is completed.

[0349] In a possible implementation, the manner in which the second face model is generated based on the second generation parameter is: applying the second generation parameter to a face model reconstruction model; and generating the second face model by using the face model reconstruction model with the second generation parameter. The face model reconstruction model refers to a model used to generate a face model. Depending on the type of face model that needs to be generated, the type of face model reconstruction model can be different, and embodiments of the present application do not limit the type of face model reconstruction model as long as the face model can be generated based on the generation parameter. Illustratively, the face model reconstruction model refers to 3DMM. For the case where the face model reconstruction model refers to 3DMM, the second face model refers to a three-dimensional face.

[0350] Illustratively, the second generation parameter can also not satisfy the update termination condition. If the second generation parameter does not satisfy the update termination condition, the second generation parameter needs to be updated. Illustratively, the process of continuing to update the second generation parameter is: in response to the second generation parameter not satisfying the update termination condition, obtaining second face key point position information based on the second generation parameter; obtaining a second target function used to update the second generation parameter based on the second face key point position information; updating the second generation parameter by using the second target function to obtain a third generation parameter; and in response to the third generation parameter satisfying the update termination condition, generating a third face model based on the third generation parameter. For implementation of the process, refer to steps 201 to 204, which will not be described here.

[0351] Illustratively, in the application scenario of directly reconstructing the first face model, the process of obtaining the generation parameter satisfying the termination condition is implemented based on the following algorithm flow:

[0352] Input: the first generation parameter p' corresponding to the first face model to be reconstructed; the set of weights ω corresponding to the loss function 1

[0353] Output: the generation parameter p* satisfying the termination condition (p* can be directly applied to the face model reconstruction model to obtain the reconstructed face model corresponding to the first face model)

[0354] Process:

[0355] 1. Initialize the generation parameter p*: p* = p'.

[0356] 2. Loop N (N is an integer not less than 1) steps:

[0357] a) Calculate the current objective function L(p*, ω 1 )

[0358] b) Update p* using the optimizer F, and the update formula is .

[0359] wherein, represents the update gradient for updating the generation parameter p* calculated based on the objective function L(p*, ω 1 ).

[0360] Exemplarily, in the application scenario of reconstructing the first face model based on the template face model, the process of obtaining the generation parameter satisfying the termination condition is implemented based on the following algorithm process:

[0361] Input: the first generation parameter p' corresponding to the first face model to be reconstructed; the set of weights ω corresponding to the loss function; the template face model T 2 .

[0362] Output: the generation parameter p* satisfying the termination condition (p* can be directly applied to the face model reconstruction model to obtain the reconstructed face model corresponding to the first face model)

[0363] Process:

[0364] 1. Initialize the generation parameter p*: p* = p'.

[0365] 2. Loop N (N is an integer not less than 1) steps:

[0366] a) Calculate the current objective function L(p*, ω 2 , T)

[0367] b) Update p* using the optimizer F, and the update formula is .

[0368] wherein, an update gradient representing an update of the generation parameter p* calculated based on the target function L(p*, ω 2 an update gradient representing an update of the generation parameter p* calculated based on the target function L(p*, ω

[0369] According to the above, the process of obtaining the generation parameter satisfying the termination condition is an iterative process. The embodiments of the present application convert the face model reconstruction problem into an optimization mathematical problem by designing a target function, so as to realize the reconstruction of the face model by obtaining the generation parameter satisfying the update termination condition.

[0370] Exemplarily, the method provided by the embodiments of the present application can be applied to the face reconstruction scene of a character in a 3D game. By using the 3D face reconstruction technology of 3DMM, a 3D face can be shaped according to a face photo provided by the 3D game, and after shaping the 3D face, the shaped 3D face can be reconstructed according to the method provided by the embodiments of the present application, so as to facilitate the user to shape a face that is more similar in appearance and more suitable for the actual game scene in the game. For example, a more beautiful face, etc. Exemplarily, in the case of reconstructing the shaped face based on a template face, a beautiful character similar in aesthetics to the template face can also be reconstructed.

[0371] The embodiments of the present application provide a scheme for automatically reconstructing a face model, which can beautify the face model reconstructed based on the 3DMM model and realize targeted local face model beautification by modifying the generation parameter. Compared with the scheme in the related art that reconstructs based on manual operation or relies on the blend shape resource corresponding to the face model, the scheme provided by the embodiments of the present application does not need manual operation and does not need to obtain the blend shape resource corresponding to the face model, which can effectively reduce the cost of human resources and the cost of obtaining the blend shape resource. In addition, the face model reconstruction scheme provided by the embodiments of the present application can be simply connected to the 3D face reconstruction technology based on 3DMM, which can serve different games or applications and has a wide application prospect.

[0372] In the embodiments of the present application, the generation parameter of the generated face model is updated by using a target function matched with the reconstruction direction of the first face model, and then the reconstructed face model is obtained based on the updated generation parameter. This face model reconstruction process does not need to rely on manual operation and can be automatically executed, the reliability of the face model reconstruction process is good and the efficiency is high, which can save human resources and time cost and is beneficial to improve the quality of the reconstructed face model.

[0373] Referring to Figure 19 The embodiments of the present application provide a face model reconstruction device, which comprises:

[0374] The first obtaining unit 1901 is configured to obtain first face key point position information corresponding to a first face model, the first face model being generated based on first generation parameters;

[0375] The second obtaining unit 1902 is configured to obtain, based on the first face key point position information, a first target function for updating the first generation parameters, the first target function being a function matched with a reconstruction direction of the first face model;

[0376] The updating unit 1903 is configured to update the first generation parameters by using the first target function to obtain second generation parameters.

[0377] The generating unit 1904 is configured to generate a second face model based on the second generation parameters in response to the second generation parameters satisfying an update termination condition.

[0378] In a possible implementation, the second obtaining unit 1902 is configured to obtain, based on the first face key point position information, a first loss function and a second loss function, the first loss function being used for constraining overall face features of the reconstructed face model, and the second loss function being used for optimizing reference features of the reconstructed face model, the reference features being different from the overall face features; and obtain, based on the first loss function and the second loss function, the first target function for updating the first generation parameters.

[0379] In a possible implementation, the second obtaining unit 1902 is further configured to obtain reference face key point position information corresponding to a reference face model, the reference face model being used for providing a constraint direction for the overall face features of the reconstructed face model; and obtain, based on the first face key point position information and the reference face key point position information, the first loss function.

[0380] In a possible implementation, the second obtaining unit 1902 is further configured to obtain, based on the first face key point position information, first target key point position information, the first target key point position information including position information corresponding to target key points in the first face model; obtain, based on the reference face key point position information, second target key point position information, the second target key point position information including position information corresponding to target key points in the reference face model, the target key points in the reference face model corresponding to the target key points in the first face model; and obtain, based on the first target key point position information and the second target key point position information, the first loss function.

[0381] In a possible implementation, the reference face model is an original face model of the first face model, the reference features include local features, and the second loss function includes a local adjustment loss function, the local adjustment loss function being directly obtained based on the first face key point position information.

[0382] In a possible implementation, the local adjustment loss function comprises at least one of a cheek adjustment loss function, a first nose adjustment loss function, a first mouth adjustment loss function, a second nose adjustment loss function, a chin adjustment loss function, a third nose adjustment loss function, a philtrum adjustment loss function, an eye adjustment loss function, a fourth nose adjustment loss function, and a second mouth adjustment loss function.

[0383] In a possible implementation, the local adjustment loss function comprises a cheek adjustment loss function; the second obtaining unit 1902 is further configured to obtain cheek left key point position information and cheek right key point position information based on the first facial key point position information; and obtain the cheek adjustment loss function based on the cheek left key point position information and the cheek right key point position information.

[0384] In a possible implementation, the local adjustment loss function comprises a second nose adjustment loss function; the second obtaining unit 1902 is further configured to obtain a nose bridge key point position information based on the first facial key point position information; obtain nose bridge guide point position information corresponding to the nose bridge key point position information; and obtain the second nose adjustment loss function based on the nose bridge key point position information and the nose bridge guide point position information.

[0385] In a possible implementation, the reference facial model is a template facial model, the reference feature comprises a scale feature, and the second loss function comprises a reference scale loss function, which is obtained based on the first facial key point position information and template facial key point position information corresponding to the template facial model.

[0386] In a possible implementation, the reference scale loss function comprises at least one of a horizontal scale loss function, a vertical scale loss function, a depth scale loss function, and a length-width scale loss function.

[0387] In a possible implementation, the reference scale loss function comprises a horizontal scale loss function; the second obtaining unit 1902 is further configured to obtain a first horizontal distance corresponding to the first facial model and at least one second horizontal distance corresponding to the first facial model based on the first facial key point position information; obtain a third horizontal distance corresponding to the template facial model and at least one fourth horizontal distance corresponding to the template facial model based on the template facial key point position information; and obtain the horizontal scale loss function based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance, and the at least one fourth horizontal distance.

[0388] In a possible implementation, the first obtaining unit 1901 is further configured to, in response to the second generation parameter not satisfying an update termination condition, obtain the second facial key point position information based on the second generation parameter;

[0389] The second obtaining unit 1902 is further configured to obtain a second target function for updating the second generation parameter based on the second face key point position information.

[0390] The updating unit 1903 is further configured to update the second generation parameter by using the second target function to obtain a third generation parameter.

[0391] The generating unit 1904 is further configured to generate a third face model based on the third generation parameter in response to the third generation parameter satisfying an update termination condition.

[0392] In a possible implementation manner, the generating unit 1904 is configured to apply the second generation parameter to the face model reconstruction model, and generate the second face model by using the face model reconstruction model with the second generation parameter.

[0393] In the embodiments of the present application, the generation parameter of the generated face model is updated by using the target function matched with the reconstruction direction of the first face model, and then the reconstructed face model is obtained based on the updated generation parameter. The face model reconstruction process does not need to rely on manual operation, can be automatically executed, has better reliability and higher efficiency in the face model reconstruction process, can save human resources and time cost, and is beneficial to improving the quality of the reconstructed face model.

[0394] It should be noted that the apparatus provided in the above embodiments is only used as an example for dividing the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0395] In the example embodiments, a computer device is also provided, which includes a processor and a memory. The memory stores at least one computer program. The at least one computer program is loaded and executed by one or more processors to implement any of the above face model reconstruction methods. The computer device can be a terminal or a server. Next, the structures of the terminal and the server will be introduced respectively.

[0396] Figure 20 is a structural schematic diagram of a terminal provided in the embodiments of the present application. The terminal can be a smart phone, a tablet computer, a notebook computer or a desktop computer. The terminal can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0397] Generally, the terminal includes a processor 2001 and a memory 2002.

[0398] The processor 2001 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 2001 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 2001 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 2001 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 2001 can further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0399] The memory 2002 can include one or more computer-readable storage media that can be non-transitory. The memory 2002 can also include high-speed random access memory and nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2002 is used to store at least one instruction for being executed by the processor 2001 to implement the face model reconstruction method provided by the method embodiments in the present application.

[0400] In some embodiments, the terminal can further optionally include a peripheral device interface 2003 and at least one peripheral device. The processor 2001, the memory 2002, and the peripheral device interface 2003 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 2003 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 2004, a display screen 2005, a camera assembly 2006, an audio circuit 2007, and a power supply 2009.

[0401] The peripheral interface 2003 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 2001 and the memory 2002. The radio frequency circuit 2004 is used to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The display screen 2005 is used to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. The camera assembly 2006 is used to capture images or videos. The audio circuit 2007 can include a microphone and a speaker. The microphone is used to capture sound waves of a user and the environment, and convert the sound waves into an electrical signal input to the processor 2001 for processing, or to the radio frequency circuit 2004 to achieve voice communication. The speaker is used to convert the electrical signal from the processor 2001 or the radio frequency circuit 2004 into sound waves.

[0402] The power supply 2009 is used to supply power to each component in the terminal. The power supply 2009 can be alternating current, direct current, disposable battery or rechargeable battery.

[0403] In some embodiments, the terminal also includes one or more sensors 2010. The one or more sensors 2010 include but are not limited to: an acceleration sensor 2011, a gyroscope sensor 2012, a pressure sensor 2013, an optical sensor 2015, and a proximity sensor 2016.

[0404] The acceleration sensor 2011 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal. The gyroscope sensor 2012 can detect the body direction and rotation angle of the terminal, and the gyroscope sensor 2012 can cooperate with the acceleration sensor 2011 to collect 3D actions of the user on the terminal. The pressure sensor 2013 can be arranged on the side frame of the terminal and / or under the display screen 2005. When the pressure sensor 2013 is arranged on the side frame of the terminal, the user's grip signal on the terminal can be detected, and the processor 2001 can perform left-hand or right-hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 2013. When the pressure sensor 2013 is arranged under the display screen 2005, the processor 2001 can control the operable control on the UI interface according to the pressure operation of the user on the display screen 2005.

[0405] The optical sensor 2015 is used to collect ambient light intensity. The proximity sensor 2016, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 2016 is used to collect the distance between the user and the front of the terminal.

[0406] Those skilled in the art can understand that, Figure 20The structure shown in the figure does not constitute a limitation on the terminal, and can include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0407] Figure 21 FIG. 1 is a structural schematic diagram of a server provided by an embodiment of the present application. The server can be different in configuration or performance, and can include one or more processors (CPU) 2101 and one or more memories 2102. The one or more memories 2102 store at least one computer program, which is loaded and executed by the one or more processors 2101 to implement the face model reconstruction method provided by each of the above-mentioned method embodiments. Of course, the server can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for implementing device functions, and will not be described here in detail.

[0408] In an example embodiment, a computer-readable storage medium is also provided, which stores at least one computer program. The at least one computer program is loaded and executed by a processor of a computer device to implement any of the above-mentioned face model reconstruction methods.

[0409] In a possible implementation manner, the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0410] In an example embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium. The processor executes the computer instructions, so that the computer device executes any of the above-mentioned face model reconstruction methods.

[0411] It should be noted that the terms "first", "second" and the like in the description and in the claims of the present application are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the data used in the examples described herein is for illustrative purposes only and thus is non-limiting. The embodiments described herein are not a limitation on the scope or complete description of all embodiments comprising the application as claimed. Rather, they are merely examples of apparatuses and methods in conformance with some aspects of the application as detailed in the appended claims.

[0412] It should be understood that the "multiple" mentioned herein refers to two or two more. "And / or", describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. The character " / " generally represents that the associated objects before and after are a kind of "or" relationship.

[0413] The above description is only an example of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of reconstruction of a face model, characterized in that, The method comprises: obtaining first face key point position information corresponding to a first face model, the first face model being generated based on first generation parameters; obtaining reference face key point position information corresponding to a reference face model, the reference face model being used to provide a constraint direction for overall face features of a reconstructed face model; based on the first face key point position information and the reference face key point position information, obtaining a first loss function, the first loss function being used to constrain the overall face features of the reconstructed face model; based on the first face key point position information, obtaining a second loss function, the second loss function being used to optimize reference features of the reconstructed face model, the reference features being different from the overall face features; in a case where the reference face model is a template face model, the reference features include scale features, and the second loss function includes a reference scale loss function, the reference scale loss function being obtained based on the first face key point position information and template face key point position information corresponding to the template face model; based on the first loss function and the second loss function, obtaining a first target function for updating the first generation parameters, the first target function being a function matched with a reconstruction direction of the first face model; updating the first generation parameters using the first target function to obtain second generation parameters; in response to the second generation parameters satisfying an update termination condition, generating a second face model based on the second generation parameters.

2. The method of claim 1, wherein, The first loss function is obtained based on the first face key point position information and the reference face key point position information, comprising: based on the first face key point position information, obtaining first target key point position information, the first target key point position information including position information corresponding to target key points in the first face model; based on the reference face key point position information, obtaining second target key point position information, the second target key point position information including position information corresponding to target key points in the reference face model, the target key points in the reference face model corresponding to the target key points in the first face model; based on the first target key point position information and the second target key point position information, obtaining the first loss function.

3. The method according to claim 1 or 2, characterized in that, In a case where the reference face model is an original face model of the first face model, the reference features include local features, and the second loss function includes a local adjustment loss function, the local adjustment loss function being obtained directly based on the first face key point position information.

4. The method of claim 3, wherein, The local adjustment loss function includes at least one of a cheek adjustment loss function, a first nose adjustment loss function, a first mouth adjustment loss function, a second nose adjustment loss function, a chin adjustment loss function, a third nose adjustment loss function, a philtrum adjustment loss function, an eye adjustment loss function, a fourth nose adjustment loss function, and a second mouth adjustment loss function.

5. The method of claim 4, wherein, The local adjustment loss function comprises a cheek adjustment loss function; before the first target function for updating the first generation parameter is obtained based on the first loss function and the second loss function, the method further comprises: Based on the first face key point position information, cheek left key point position information and cheek right key point position information are obtained; Based on the cheek left key point position information and the cheek right key point position information, the cheek adjustment loss function is obtained.

6. The method of claim 4, wherein, The local adjustment loss function comprises a second nose adjustment loss function; before the first target function for updating the first generation parameter is obtained based on the first loss function and the second loss function, the method further comprises: Based on the first face key point position information, the bridge of the nose key point position information is obtained; The bridge of the nose guide point position information corresponding to the bridge of the nose key point position information is obtained; Based on the bridge of the nose key point position information and the bridge of the nose guide point position information, the second nose adjustment loss function is obtained.

7. The method according to claim 1 or 2, characterized in that, The reference scale loss function comprises at least one of horizontal scale loss function, vertical scale loss function, depth scale loss function and length-width scale loss function.

8. The method of claim 7, wherein, The reference scale loss function comprises horizontal scale loss function; before the first target function for updating the first generation parameter is obtained based on the first loss function and the second loss function, the method further comprises: Based on the first face key point position information, the first face model corresponding first horizontal distance and the first face model corresponding at least one second horizontal distance are obtained; Based on the template face key point position information, the third horizontal distance corresponding to the template face model and the fourth horizontal distance corresponding to the template face model are obtained; Based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance and the at least one fourth horizontal distance, the horizontal scale loss function is obtained.

9. The method of claim 1 or 2, wherein, After the first generation parameter is updated by using the first target function to obtain the second generation parameter, the method further comprises: In response to the second generation parameter not satisfying the update termination condition, the second face key point position information is obtained based on the second generation parameter; Based on the second face key point position information, the second target function for updating the second generation parameter is obtained; The second generation parameter is updated by using the second target function to obtain the third generation parameter; In response to the third generation parameter satisfying the update termination condition, the third face model is generated based on the third generation parameter.

10. The method of claim 1 or 2, wherein, The second face model is generated based on the second generation parameter, comprising: The second generation parameter is applied to the face model reconstruction model; The second face model is generated by using the face model reconstruction model with the second generation parameter.

11. A device for reconstructing a face model, characterized by The device comprises: The first acquisition unit is used for obtaining the first face key point position information corresponding to the first face model, and the first face model is generated based on the first generation parameter; The second obtaining unit is configured to obtain reference face key point position information corresponding to a reference face model, the reference face model being used to provide a constraint direction for overall face features of the reconstructed face model; based on the first face key point position information and the reference face key point position information, obtain a first loss function, the first loss function being used to constrain the overall face features of the reconstructed face model; based on the first face key point position information, obtain a second loss function, the second loss function being used to optimize reference features of the reconstructed face model, the reference features being different from the overall face features; in a case where the reference face model is a template face model, the reference features include scale features, and the second loss function includes a reference scale loss function, the reference scale loss function being obtained based on the first face key point position information and template face key point position information corresponding to the template face model; based on the first loss function and the second loss function, obtain a first target function used to update the first generation parameter, the first target function being a function matched with a reconstruction direction of the first face model; The updating unit is configured to update the first generation parameter by using the first target function, to obtain a second generation parameter. The generating unit is configured to generate a second face model based on the second generation parameter in response to the second generation parameter satisfying an update termination condition.

12. The apparatus of claim 11, wherein, The second obtaining unit is configured to obtain first target key point position information based on the first face key point position information, the first target key point position information including position information corresponding to target key points in the first face model. The second obtaining unit is configured to obtain second target key point position information based on the reference face key point position information, the second target key point position information including position information corresponding to target key points in the reference face model, the target key points in the reference face model corresponding to the target key points in the first face model; and obtain the first loss function based on the first target key point position information and the second target key point position information.

13. The apparatus of claim 11 or 12, wherein, In a case where the reference face model is an original face model of the first face model, the reference features include local features, and the second loss function includes a local adjustment loss function, the local adjustment loss function being obtained directly based on the first face key point position information.

14. The apparatus of claim 13, wherein, The local adjustment loss function includes at least one of a cheek adjustment loss function, a first nose adjustment loss function, a first mouth adjustment loss function, a second nose adjustment loss function, a chin adjustment loss function, a third nose adjustment loss function, a philtrum adjustment loss function, an eye adjustment loss function, a fourth nose adjustment loss function, and a second mouth adjustment loss function.

15. The apparatus of claim 14, wherein, The local adjustment loss function comprises a cheek adjustment loss function; the second obtaining unit is further configured to obtain cheek left key point position information and cheek right key point position information based on the first face key point position information; and obtain the cheek adjustment loss function based on the cheek left key point position information and the cheek right key point position information.

16. The apparatus of claim 14, wherein, The local adjustment loss function comprises a second nose adjustment loss function; the second obtaining unit is further configured to obtain nose ridge key point position information based on the first face key point position information; and obtain nose ridge guide point position information corresponding to the nose ridge key point position information; The second obtaining unit is further configured to obtain the second nose adjustment loss function based on the nose ridge key point position information and the nose ridge guide point position information.

17. The apparatus of claim 11 or 12, wherein, The reference ratio loss function comprises at least one of a horizontal ratio loss function, a vertical ratio loss function, a depth ratio loss function and a length-width ratio loss function.

18. The apparatus of claim 17, wherein, The reference ratio loss function comprises a horizontal ratio loss function; the second obtaining unit is further configured to obtain a first horizontal distance corresponding to the first face model and at least one second horizontal distance corresponding to the first face model based on the first face key point position information; The second obtaining unit is further configured to obtain a third horizontal distance corresponding to the template face model and at least one fourth horizontal distance corresponding to the template face model based on the template face key point position information; and obtain the horizontal ratio loss function based on the first horizontal distance, the at least one second horizontal distance, the third horizontal distance and the at least one fourth horizontal distance.

19. The apparatus of claim 11 or 12, wherein, The first obtaining unit is further configured to obtain second face key point position information based on the second generation parameter in response to the second generation parameter not satisfying the update termination condition; The second obtaining unit is further configured to obtain a second target function for updating the second generation parameter based on the second face key point position information; The update unit is further configured to update the second generation parameter by using the second target function to obtain a third generation parameter; The generation unit is further configured to generate a third face model based on the third generation parameter in response to the third generation parameter satisfying the update termination condition.

20. The apparatus of claim 11 or 12, wherein, The generation unit is configured to apply the second generation parameter to a face model reconstruction model; and generate the second face model by using the face model reconstruction model with the second generation parameter.

21. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor to implement the face model reconstruction method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor to implement the face model reconstruction method according to any one of claims 1 to 10.

23. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the face model reconstruction method according to any one of claims 1 to 10.

Images