Non-rigid registration method, device, equipment and storage medium for three-dimensional face model

Abstract

The present application discloses a non-rigid registration method for a three-dimensional face model. The method includes: obtaining a target three-dimensional face model and multiple first key points of the target three-dimensional face model, aligning the target three-dimensional face model with a preset standard three-dimensional face model; obtaining the nearest neighbor points corresponding to multiple second key points preset in the standard three-dimensional face model in the target three-dimensional face model, where the nearest neighbor points are the first key points; establishing a correspondence between the nearest neighbor points and the second key points; and adjusting the second key points of the standard three-dimensional face model based on the correspondence to obtain an adjusted standard three-dimensional face model. In this way, the standard three-dimensional face model is adjusted based on the relationship between the nearest neighbor points in the target three-dimensional model and the second key points of the standard three-dimensional model, and the adjusted standard three-dimensional face model can maintain good shape information relative to the target three-dimensional face model.

Description

Technical Field

[0001] The present application relates to the field of digital geometry processing technology, and in particular to a non-rigid registration method, device, equipment and storage medium for a three-dimensional face model. Background Art

[0002] With the development of 3D facial reconstruction technology, people can collect high-precision 3D facial scanning data through multi-view camera arrays. Compared with general 3D facial models, these 3D scans usually have a particularly large number of face patches, which can capture the geometric details of facial wrinkles, eyebrows and other parts.

[0003] Different 3D facial models usually have different mesh topologies. Non-rigid registration of 3D faces refers to inputting two 3D facial models, with one model serving as a template and the other as a target model, performing non-rigid deformation on the template, and outputting a new 3D facial model so that the output face model and the target model have the same geometric shape and the same mesh topology as the template.

[0004] Currently, during the non-rigid registration process, shape information is lost severely, resulting in a significant difference in shape between the output 3D face model and the target model. Summary of the Invention

[0005] The present application provides a non-rigid registration method, apparatus, device and storage medium for a three-dimensional face model, which can maintain good shape information during the non-rigid registration process.

[0006] In a first aspect, the present application provides a non-rigid registration method for a three-dimensional face model, the method comprising: obtaining a target three-dimensional face model and multiple first key points of the target three-dimensional face model, and aligning the target three-dimensional face model with a preset standard three-dimensional face model; obtaining the nearest neighbor points corresponding to multiple second key points preset in the standard three-dimensional face model in the target three-dimensional face model, where the nearest neighbor points are the first key points; establishing a correspondence between the nearest neighbor points and the second key points; and adjusting the second key points of the standard three-dimensional face model based on the correspondence to obtain an adjusted standard three-dimensional face model.

[0007] According to a second aspect of the present application, a non-rigid registration device for a three-dimensional face model is provided, which includes: an acquisition module for acquiring a target three-dimensional face model and multiple first key points of the target three-dimensional face model; an alignment module connected to the acquisition module for aligning the target three-dimensional face model with a preset standard three-dimensional face model; the acquisition module is also used to acquire the nearest neighbor points corresponding to multiple second key points preset in the standard three-dimensional face model in the target three-dimensional face model, where the nearest neighbor points are first key points; the acquisition module is also used to establish a corresponding relationship between the nearest neighbor points and the second key points; and an adjustment module connected to the acquisition module for adjusting the second key points of the standard three-dimensional face model based on the corresponding relationship to obtain an adjusted standard three-dimensional face model.

[0008] A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement the above-mentioned non-rigid registration method of a three-dimensional face model.

[0009] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned non-rigid registration method of the three-dimensional face model when the program instructions are executed by a processor.

[0010] The present application has at least the following beneficial effects: compared with the existing technology, the present application adjusts the standard three-dimensional face model based on the relationship between the nearest neighbor point in the target three-dimensional model and the second key point of the standard three-dimensional model. The adjusted standard three-dimensional face model can maintain good shape information relative to the target three-dimensional face model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 This is a flow chart of an embodiment of a non-rigid registration method for a three-dimensional face model of the present application;

[0013] Figure 2 is a schematic diagram of the target three-dimensional face model of this application;

[0014] Figure 3 This is a flow chart of an embodiment of a non-rigid registration method for a three-dimensional face model of the present application;

[0015] Figure 4 yes Figure 2 A rendered image of the target 3D face model in ;

[0016] Figure 5 is a schematic diagram of facial key points in a face image of the present application;

[0017] Figure 6 This is a flow chart of an embodiment of a non-rigid registration method for a three-dimensional face model of the present application;

[0018] Figure 7 This is a flow chart of an embodiment of a non-rigid registration method for a three-dimensional face model of the present application;

[0019] Figure 8 This is a flow chart of an embodiment of a non-rigid registration method for a three-dimensional face model of the present application;

[0020] Figure 9 is a schematic diagram of the final model of this application;

[0021] Figure 10 Schematic diagram of the framework structure of an embodiment of a non-rigid registration device for a three-dimensional face model of the present application;

[0022] Figure 11 It is a schematic diagram of the framework structure of the electronic device of the present application;

[0023] Figure 12 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically specified.

[0026] A first aspect of the present application provides a non-rigid registration method for a three-dimensional face model. Based on this method, the adjusted three-dimensional face model can maintain good shape information relative to the target three-dimensional face model.

[0027] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an embodiment of a non-rigid registration method for a 3D face model of the present invention. As shown in the figure, the method includes:

[0028] Step S11: Acquire a target three-dimensional face model and a plurality of first key points of the target three-dimensional face model, and align the target three-dimensional face model with a preset standard three-dimensional face model.

[0029] Please combine Figure 2 , Figure 2 Specifically, a 3D face scan data acquisition device can acquire 3D face scan data, and a corresponding target 3D face model S1 can be established based on the acquired 3D face scan data.

[0030] For example, three-dimensional facial scanning data collected by a multi-view camera array can be used to obtain high-precision three-dimensional facial scanning data, and a target three-dimensional facial model can be established based on the three-dimensional facial scanning data.

[0031] The first key points in the target 3D face model are facial key points that can represent the facial contour, specific shape, etc. Facial key points can include key points corresponding to the eye area, key points corresponding to the nose area, key points corresponding to the mouth area, key points corresponding to the eyebrow area, key points corresponding to the chin area, and key points corresponding to the facial contour, or any combination of at least two of the following:

[0032] Specifically, the target 3D face model can be input into a facial key point detection model, and then the facial key points in the target 3D face model can be detected using facial key point detection technology to obtain the first key points. The facial key point detection model can be a machine learning model that supports detecting facial key point coordinates, and the facial key points output by the model are coordinate point information on the target 3D face model.

[0033] The standard 3D face model can be any standard 3D face model in the 3DMM library. A 3D face model that meets the requirements can be obtained from the 3DMM library as the standard 3D face model. The topology structure of the standard 3D face model is different from the topology structure of the target 3D face model.

[0034] It should be understood that a 3D face model is composed of many sheet-like structures, called facets. The facets of a 3D face model are spliced together to form a complete 3D face model, and each facet is part of the 3D face model. Different 3D face models can vary in facet type, number of facets, and arrangement of facets. Facet type refers to the shape of the facet. For example, facet types can include triangular facets, quadrilateral facets, and so on.

[0035] Different topological structures of 3D facial models can refer to differences in one or more of the facet type, facet number, or facet arrangement. For example, if one 3D facial model has triangular facets and another has quadrilateral facets, the two 3D facial models have different topological structures.

[0036] The standard 3D face model of this application has fewer facets than the target 3D face model, so the topological format of the standard 3D face model and the topological format of the target 3D face model can be different. For example, the standard 3D face model is composed of quadrilateral facets with a smaller number of facets, while the target 3D face model is composed of triangular facets with a larger number of facets.

[0037] It should be understood that for the same 3D face model, the more facets there are, the higher the resolution. Therefore, the present application can convert a high-resolution 3D face model into a low-resolution 3D face model.

[0038] In other embodiments, the number of facets in the standard 3D face model may be greater than the number of facets in the target 3D face model, which is not specifically limited here.

[0039] After obtaining the target 3D face model and the standard 3D face model, the target 3D face model and the standard 3D face model are aligned. This step aligns the 3D data of the target 3D face model with the 3D data of the standard 3D face model, that is, converts the target 3D face model and the standard 3D face model into the same coordinate system.

[0040] Step S12: Acquire the nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model, where the nearest neighbor points are the first key points.

[0041] The second key points are also facial key points, which are pre-set in the standard 3D face model. For example, multiple second key points can be set corresponding to areas such as the mouth, nose, and eyebrows. Of course, multiple second key points can also be set in other facial areas of the standard 3D face model.

[0042] After obtaining the second key point, the nearest neighbor point of the second key point in the target 3D face model is further obtained. Specifically, a point that meets the requirements among the numerous first key points in the target 3D face model is found as the nearest neighbor point. It should be understood that in the target 3D face model, the overall positional relationship of the nearest neighbor point in the target 3D face model is most similar to the overall positional relationship of the second key point corresponding to the nearest neighbor point in the standard 3D face model.

[0043] In some specific embodiments, the nearest neighbor points corresponding to the plurality of second key points preset in the standard 3D face model in the target 3D face model may be obtained using a proximity algorithm. For example, the nearest neighbor points corresponding to the second key points may be obtained using a KNN algorithm.

[0044] Step S13: establishing a correspondence between the nearest neighbor point and the second key point.

[0045] A one-to-one correspondence is established between the nearest neighbor point and the second key point. The correspondence may include a correspondence between the coordinate information of the nearest neighbor point and the coordinate information of the second key point. Thus, the correspondence may reflect the positional relationship between the nearest neighbor point and the second key point.

[0046] Step S14: adjusting the second key point of the standard three-dimensional face model based on the corresponding relationship to obtain an adjusted standard three-dimensional face model.

[0047] Based on the above, it can be seen that the correspondence relationship can include the positional relationship between the nearest neighbor point and the second key point. Therefore, adjusting the second key point of the standard 3D facial model based on the correspondence relationship involves adjusting the second key point based on the positional relationship between the nearest neighbor point and the second key point, and ensuring that the positional information of the adjusted second key point is the same as the positional information of the nearest neighbor point or meets a preset condition. After the second key point in the standard 3D facial model is adjusted, an adjusted standard 3D facial model is obtained and output.

[0048] It should be understood that the topological format of the adjusted standard three-dimensional face model is different from the topological format of the target three-dimensional face model, but the shape information of the two is consistent or the difference is within a certain preset condition range, so that the adjusted standard three-dimensional face model can ensure good shape information, that is, compared with the target three-dimensional face model, the adjusted standard three-dimensional face model has good shape retention.

[0049] In summary, this application adjusts the standard three-dimensional face model based on the relationship between the nearest neighbor point in the target three-dimensional face model and the second key point of the standard three-dimensional face model, and the adjusted standard three-dimensional face model can maintain good shape information relative to the target three-dimensional information.

[0050] Further combining the above content, this application can convert a 3D face model in one topological format into a 3D face model in another topological format, thereby achieving a change in the topological format of the 3D face model. In some application scenarios, a high-resolution 3D face model can be converted into a low-resolution 3D face model, thereby achieving a change in the resolution of the 3D face model.

[0051] See also Figure 3 , Figure 3It is a flowchart of an embodiment of the non-rigid registration method of a three-dimensional face model of the present application.

[0052] Specifically, in some embodiments, step S11 may specifically include:

[0053] Step S111: performing a guideable rendering process on the target 3D face model to obtain a rendered image and rendering data.

[0054] The rendering process may include obtaining rendering parameters, and performing rendering synthesis on the rendering parameters and feature data through a rendering algorithm to obtain a rendered image.

[0055] See also Figure 4 , Figure 4 yes Figure 2 Rendering image L of target 3D face model S1 in . After the target 3D face model S1 is subjected to the guideable rendering process, the corresponding rendering image L is obtained.

[0056] The rendering parameters may include head parameters and virtual camera parameters. The head parameters include at least one of the head's orientation, position, and size, and the virtual camera parameters include at least one of the camera's position, viewing angle, aperture, focal length, field of view aspect ratio, and image resolution. The feature data may include 3D face model data, facial expression model data, ambient lighting model data, and the like of the target 3D face model.

[0057] It should be understood that the rendering data may include rendering parameters, which are specific parameterized embodiments of the rendering process. The rendering process can be understood through the rendering parameters.

[0058] In some specific embodiments, the target three-dimensional face model is rendered in a pytorch-based rendering manner to obtain a rendered image and rendering data.

[0059] Step S112: Detect facial key points based on the rendered image to obtain facial key points.

[0060] Based on the facial key point detection technology, the rendered image is detected for facial key points to obtain facial key points. Figure 5 , Figure 5 This is a schematic diagram of facial key points L1 in the face image of this application. As shown in the figure, facial key points L1 are primarily distributed along the edges of the eyes, mouth, eyebrows, and other areas of the face in the rendered image, as well as along the facial contour lines. These facial key points L1 reflect the primary shape information of the face and provide a good representation of the primary shape of the face. Of course, facial key points L1 can also be distributed in other areas, such as the cheeks and forehead.

[0061] Step S113: obtaining a first key point corresponding to the facial key point in the target three-dimensional face model based on the rendering data.

[0062] In combination with the above content, a rendered image can be obtained by performing guided rendering based on the feature data and rendering data of the target three-dimensional face model, wherein the rendering parameters include the rendering data.

[0063] Therefore, the first key point corresponding to the facial key point in the target three-dimensional face model is obtained based on the rendering data, that is, a derivative operation is performed based on the rendering data and the data information of the facial key point to obtain the data information of the first key point corresponding to the facial key point in the target three-dimensional face data, and the data information includes position information.

[0064] See also Figure 6 , Figure 6 FIG. 1 is a flow chart of an embodiment of the non-rigid registration method for a 3D face model of the present application. In some specific embodiments, the above step S112 may specifically include the following steps:

[0065] S1121: Perform face recognition on the rendered image to obtain position information of the face in the rendered image.

[0066] In combination with the above content, face recognition is performed on the rendered image based on face recognition technology, and then the position information of the face in the rendered image is obtained. The face position information represents the position information of the face in the rendered image.

[0067] It should be understood that the rendered image may contain other images in addition to the facial image. The face recognition technology can recognize the facial image in the rendered image and obtain the position information of the facial image in the entire rendered image.

[0068] S1122: Obtain a face image in the rendered image based on the position information of the face.

[0069] The position of the face image in the rendered image can be obtained based on the face position information, and the face image in the rendered image can be obtained based on the position. The face image does not include the non-face part in the rendered image.

[0070] S1123: Detect facial key points based on the face image.

[0071] As can be seen from the above, the acquired facial image does not contain the non-face parts in the rendered image. Based on this, when performing facial landmark detection, it will not be interfered with by the non-face parts in the rendered image, and thus facial landmark detection is performed only on the facial image, which can improve the accuracy of facial landmark detection.

[0072] See also Figure 7 , Figure 71 is a flow chart of an embodiment of the non-rigid registration method of a 3D face model of the present application. In some specific embodiments, after the above step S111, the following steps may also be included:

[0073] S21: performing face segmentation on the face image in the rendered image to obtain different first face segmentation regions; wherein the different first face segmentation regions do not overlap with each other, and a facial key point is located in one first face segmentation region.

[0074] After the face image is segmented using the face segmentation technology, the face image is segmented into multiple parts, and the segmented parts are the first face segmentation areas.

[0075] The different first face segmentation regions do not overlap, meaning the multiple segments obtained from face segmentation are independent and non-overlapping, forming a complete face image. Consequently, a facial landmark is located within only one first face segmentation region and not within multiple first face segmentation regions simultaneously.

[0076] Specifically, after performing face segmentation on the face image, the first face segmentation region may include the eyes, nose, mouth, eyebrow region, and regions other than these regions, such as the forehead region.

[0077] After step S21, the following steps may be included:

[0078] S22: Acquire a preset second face segmentation region to which a preset second key point of the standard three-dimensional face model belongs.

[0079] A plurality of second face segmentation regions are pre-set in the standard three-dimensional face model. The preset second key points belong to different second face segmentation regions respectively, and one second key point belongs to only one second face segmentation region.

[0080] For example, the second face segmentation region includes the eyes, nose, mouth, eyebrows, and other regions except these regions.

[0081] S23: Acquire a nearest neighbor point in the first face segmentation area corresponding to the second face segmentation area in the target three-dimensional face model.

[0082] The first face segmentation region corresponds to the second face segmentation region one-to-one and forms a corresponding relationship, and the corresponding first face segmentation region and the second face segmentation region have the same semantics. For example, the eye region of the first face segmentation region corresponds to the eye region of the second face segmentation region and both are semantically identical eye regions, and the nose region of the first face segmentation region corresponds to the nose region of the second face segmentation region and both are semantically identical nose regions.

[0083] In the process of obtaining the nearest neighbor point, the first face segmentation area corresponding to the second face segmentation area to which the second key point belongs is first determined, and the nearest neighbor point is found in the first key point in the first face segmentation area.

[0084] It should be understood that by searching for the nearest neighbor point in the corresponding area, the first face segmentation area to which the obtained nearest neighbor point belongs must correspond to the second face segmentation area corresponding to the second key point. Therefore, the semantics of the nearest neighbor point is the same as that of the second key point, which improves the accuracy of the nearest neighbor point determination and ensures that the adjusted standard three-dimensional face model maintains good shape information.

[0085] See also Figure 8 , Figure 8 This is a flow chart of an embodiment of the non-rigid registration method for a 3D face model of the present application. In some embodiments, after step S14, the method further includes:

[0086] S31: Calculate the loss function value through the loss function.

[0087] A loss function is a function that maps the value of a random event or its related random variables into a non-negative real number to represent the "risk" or "loss" of the random event.

[0088] It should be understood that during the adjustment process of the standard 3D face model, the standard 3D face model is often adjusted multiple times to obtain a 3D face model that meets the preset conditions. The 3D face model obtained at this time is the final result, and the adjusted 3D face model obtained before the final result is obtained is the intermediate result. After obtaining the intermediate result, the loss value is calculated using the loss function.

[0089] Specifically, in some specific embodiments, the formula of the loss function can be:

[0090] Loss = A*Distance + B*Laplace + C*Edge.

[0091] Where A, B, and C are preset proportional constants, Distance is the square of the three-dimensional distance between the second key point and its nearest neighbor, Laplace is the Laplace smoothing term, and Edge is the average length of all edges of the adjusted standard three-dimensional face model.

[0092] S32: Determine whether the loss value of the loss function is greater than a preset threshold.

[0093] The loss function's loss value reflects the difference between the intermediate result and the standard result under the preset conditions. This difference reflects the deviation of the intermediate result from the standard result. Of course, the difference described here represents the gap between the intermediate result and the standard result.

[0094] The preset threshold is compared with the difference. If the difference is greater than the preset threshold, the difference is large, the intermediate result deviates significantly from the standard result, and does not meet the preset conditions. Step S33 is then executed. If the difference is less than or equal to the preset difference, the difference is small, the intermediate result deviates slightly from the standard result, and meets the preset conditions. Step S34 is then executed.

[0095] S33: Obtain the nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model.

[0096] If the difference is greater than the preset threshold, it means that the intermediate result is significantly different from the standard result. In this case, the process returns to step S12 to reacquire the adjusted target 3D face model as the intermediate result.

[0097] S34: Output the adjusted standard 3D face model as the final model.

[0098] If the difference is less than or equal to the preset threshold, the intermediate result is close to the standard result and meets the preset conditions. At this point, no further adjustments are required to the target 3D face model. The current standard 3D face model can be used as the final output. The final model is the target 3D face model after non-rigid registration.

[0099] See also Figure 9 , Figure 9 Schematic diagram of the final model S2 of the present application. Compared with the target 3D face model S1, the final model S2 retains good shape information, that is, the shape of the final model S2 is basically consistent with the shape of the target 3D face model S1.

[0100] Furthermore, the topological format of the final model S2 is different from that of the target 3D face model S1. Specifically, the target 3D face model S1 has triangular facets, a large number of facets, and a high resolution, while the final model S2 has quadrilateral facets, a small number of facets, and a low resolution.

[0101] Therefore, the non-rigid registration method of the three-dimensional face model of the present application can be applied to converting a high-resolution target three-dimensional face into a low-resolution final model while maintaining good shape information.

[0102] The second aspect of the present application provides a non-rigid registration device 20 for a three-dimensional face model. Figure 10 , Figure 10 Schematic diagram of the framework structure of the non-rigid registration device 20 of the three-dimensional face model of the present application.

[0103] The non-rigid registration device 20 for a three-dimensional face model includes an acquisition module 21 , an alignment module 22 , and an adjustment module 23 . The acquisition module 21 is connected to the alignment module 22 and the adjustment module 23 , respectively.

[0104] The acquisition module 21 is configured to acquire a target 3D facial model and multiple first key points of the target 3D facial model. The alignment module 22 is configured to align the target 3D facial model with a preset standard 3D facial model. The acquisition module 21 is further configured to obtain the nearest neighbor points corresponding to multiple second key points preset in the standard 3D facial model in the target 3D facial model, where the nearest neighbor points are first key points. The acquisition module 21 is further configured to establish a correspondence between the nearest neighbor points and the second key points. The adjustment module 23 is configured to adjust the second key points of the standard 3D facial model based on the correspondence to obtain an adjusted standard 3D facial model.

[0105] The above module can also be used to execute other corresponding steps in the above non-rigid registration method of the three-dimensional face model. Please refer to the above implementation content and will not be repeated here.

[0106] The third aspect of the present application provides an electronic device 30, see Figure 11 , Figure 11 It is a schematic diagram of the framework structure of the electronic device 30 of the present application.

[0107] Specifically, the electronic device 30 includes a memory 31 and a processor 32 coupled to each other. The processor 32 is used to execute program instructions stored in the memory 31 to implement the non-rigid registration method of the three-dimensional face model provided in the above embodiment.

[0108] In a specific implementation scenario, the electronic device 30 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 30 may also include mobile devices such as a laptop computer and a tablet computer, which are not limited here.

[0109] Specifically, the processor 32 is used to control itself and the memory 31 to implement the steps of the above-mentioned training method embodiment of any image detection model, or to implement the steps of the above-mentioned image detection method embodiment. The processor 32 can also be called a CPU (Central Processing Unit). The processor 32 may be an integrated circuit chip with signal processing capabilities. The processor 32 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 32 can be implemented by an integrated circuit chip.

[0110] See also Figure 12 , Figure 12 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 40 of the present application.

[0111] The computer-readable storage medium 40 stores program instructions 401 that can be executed by a processor. The program instructions 401 are used to implement the steps in the above-mentioned embodiment of the non-rigid registration method for a three-dimensional face model, which will not be described in detail here.

[0112] The computer-readable storage medium 40 may include various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0113] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments unless there is a conflict.

[0114] The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A non-rigid registration method for a 3D face model, characterized in that: The method comprises: Acquiring a target three-dimensional face model and a plurality of first key points of the target three-dimensional face model, and aligning the target three-dimensional face model with a preset standard three-dimensional face model; wherein the target three-dimensional face model corresponds to a plurality of different first face segmentation regions, the different first face segmentation regions do not overlap, and one of the first key points is located within one of the first face segmentation regions; Obtaining nearest neighbor points corresponding to a plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model, where the nearest neighbor points are the first key points, including: obtaining a preset second face segmentation region to which the preset second key points of the standard three-dimensional face model belong; the preset second key points are respectively located in different second face segmentation regions, and one second key point is located in one second face segmentation region; and obtaining the nearest neighbor points in a first face segmentation region corresponding to the second face segmentation region in the target three-dimensional face model; Establishing a correspondence between the nearest neighbor point and the second key point; The second key point of the standard three-dimensional face model is adjusted based on the corresponding relationship to obtain the adjusted standard three-dimensional face model.

2. The method according to claim 1, characterized in that The step of obtaining a plurality of first key points of the target three-dimensional face model includes: Performing a rendering process on the target three-dimensional face model to obtain a rendered image and rendering data; Performing facial key point detection based on the rendered image to obtain facial key points; A first key point corresponding to the facial key point in the target three-dimensional face model is obtained based on the rendering data.

3. The method according to claim 2, characterized in that The step of detecting facial key points based on the rendered image to obtain facial key points includes: Performing face recognition on the rendered image to obtain position information of the face in the rendered image; Acquire a face image in the rendered image based on the position information of the face; Perform facial key point detection based on the facial image.

4. The method according to claim 2, characterized in that After the step of performing a guideable rendering process on the target 3D face model to obtain a rendered image and rendering data, the method further includes: Performing face segmentation on the face image in the rendered image to obtain different first face segmentation regions; wherein the different first face segmentation regions do not overlap with each other, and one of the face key points is located within one of the first face segmentation regions.

5. The method according to claim 1, wherein After the step of adjusting the second key point of the standard three-dimensional face model based on the corresponding relationship to obtain an adjusted standard three-dimensional face model, the method further includes: Calculate the loss function value through the loss function; Determine whether the loss value of the loss function is greater than a preset threshold; If yes, continue to execute the step of obtaining the nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model; If not, the adjusted standard three-dimensional face model is output as the final model.

6. The method according to claim 5, characterized in that The formula of the loss function includes: Loss = A*Distance + B*Laplace + C*Edge; Where A, B, and C are preset proportional constants, Distance is the square of the three-dimensional distance between the second key point and its nearest neighbor, Laplace is the Laplace smoothing term, and Edge is the average length of all edges of the adjusted standard three-dimensional model.

7. The method according to claim 1, characterized in that The step of obtaining the nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model includes: The nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model are obtained through a proximity algorithm.

8. A non-rigid registration device for a three-dimensional face model, characterized in that: The device comprises: an acquisition module, configured to acquire a target three-dimensional face model and a plurality of first key points of the target three-dimensional face model; wherein the target three-dimensional face model corresponds to a plurality of different first face segmentation regions, the different first face segmentation regions do not overlap with each other, and one of the first key points is located within one of the first face segmentation regions; an alignment module, connected to the acquisition module, for aligning the target three-dimensional face model with a preset standard three-dimensional face model; The acquisition module is further configured to acquire the nearest neighbor points corresponding to the plurality of second key points preset in the standard three-dimensional face model in the target three-dimensional face model, wherein the nearest neighbor points are the first key points, including: acquiring a preset second face segmentation region to which the preset second key points of the standard three-dimensional face model belong; the preset second key points are respectively located in different second face segmentation regions, and one second key point is located in one second face segmentation region; and acquiring the nearest neighbor points in the first face segmentation region corresponding to the second face segmentation region in the target three-dimensional face model; The acquisition module is further configured to establish a correspondence between the nearest neighbor point and the second key point; An adjustment module is connected to the acquisition module and is used to adjust the second key point of the standard three-dimensional face model based on the corresponding relationship to obtain the adjusted standard three-dimensional face model.

9. An electronic device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the non-rigid registration method of the three-dimensional face model according to any one of claims 1 to 7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the non-rigid registration method of a three-dimensional face model according to any one of claims 1 to 7 is implemented.

Images