Method for creating 3D models based on 2D key points, computer and storage media

Abstract

This invention relates to artificial intelligence technology, and its embodiments provide a method for building a 3D model based on 2D key points, a computer, and a storage medium. The method includes the following steps: acquiring a target 2D face image; performing predictive processing on the target 2D face image using a target 2D face key point recognition model to obtain a first key point image including target key points; performing 3D reconstruction processing on the target 2D face image to obtain a 3D texture image; performing coordinate transformation processing on the key point coordinates in the first key point image based on the texture map coordinate data of the 3D texture image to obtain a second key point image based on the texture map coordinate data; and building a target 3D face model based on the second key point image and the 3D texture image. This method can improve the accuracy of binding corresponding points in the 3D face model, making the built 3D face model closer to real-world usage scenarios.

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of artificial intelligence digital medical technology, and particularly to a method for establishing a three-dimensional model based on two-dimensional key points, a computer, and a storage medium. Background Technology

[0002] Two-dimensional facial landmark detection is a hot topic in computer vision and image processing, widely used in financial identity authentication, access control, intelligent video surveillance, liveness detection, and entertainment, significantly replacing manual resources and effectively improving productivity. However, the quality and quantity of labeled two-dimensional facial landmark data—the "raw material" for 2D facial landmark detection algorithms—have long plagued industry and academia. When the number of points is large, it's difficult to guarantee both accuracy and speed. Furthermore, when faces are occluded or exhibit large poses, different annotators use inconsistent standards for labeling landmarks, leading to ambiguity in data labels. To address the difficulty of labeling data, the common approach is to augment the data in 2D space to obtain more data, such as scaling, rotating, translating, occluding, and enhancing images, while simultaneously transforming the landmarks accordingly. However, faces are three-dimensional in the real world, so 2D augmentation alone is insufficient to simulate and cover real-world application scenarios. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] The main objective of this invention is to propose a method, computer, and storage medium for establishing a three-dimensional model based on two-dimensional key points, which can improve the accuracy of binding corresponding points in a three-dimensional face model, making the established three-dimensional face model closer to real-world usage scenarios.

[0005] In a first aspect, embodiments of the present invention provide a method for establishing a three-dimensional model based on two-dimensional key points, including:

[0006] Acquire the target 2D face image;

[0007] The target two-dimensional face image is predicted by the target two-dimensional face key point recognition model to obtain a first key point image including the target key points, and the coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image.

[0008] The target two-dimensional face image is subjected to three-dimensional reconstruction processing to obtain a three-dimensional texture image;

[0009] Based on the texture map coordinate data of the three-dimensional texture image, the coordinates of the key points in the first key point image are transformed to obtain the second key point image based on the texture map coordinate data;

[0010] A target 3D face model is obtained based on the second key point image and the 3D texture image.

[0011] In one embodiment, before performing predictive processing on the target two-dimensional face image using a two-dimensional facial landmark recognition model to obtain a first keypoint image including the target keypoints, the process includes:

[0012] Obtain a public dataset of key points for 2D face images;

[0013] The two-dimensional face key point recognition model is trained using the publicly available dataset of key points of the target two-dimensional face image to obtain the target two-dimensional face key point recognition model.

[0014] In one embodiment, before acquiring the target two-dimensional face image, the method includes:

[0015] Acquire a two-dimensional image, the two-dimensional image including several personal portrait information;

[0016] The two-dimensional image is processed by a face generation module to extract the face, thereby obtaining the target two-dimensional face image.

[0017] In one embodiment, the step of performing face extraction processing on the two-dimensional image through a face generation module to obtain a target two-dimensional face image includes:

[0018] The two-dimensional image is processed by a face generation module to extract faces, thereby obtaining a two-dimensional face image.

[0019] The two-dimensional face image is processed to obtain a target two-dimensional face image with an unobstructed frontal view.

[0020] In one embodiment, the step of establishing the target 3D face model based on the second key point image and the 3D texture image includes:

[0021] The second key point image is pasted onto the first 3D face model to obtain the second 3D face model;

[0022] The target key points are determined from the second 3D face model;

[0023] The three-dimensional texture image is pasted onto the second three-dimensional face model based on the target key points on the second three-dimensional face model to obtain the target three-dimensional face model.

[0024] In one embodiment, the step of performing predictive processing on the target two-dimensional face image using a target two-dimensional face key point recognition model to obtain a first key point image including target key points includes:

[0025] The target two-dimensional face image is input into the target two-dimensional face key point recognition model to obtain the target key points output by the target two-dimensional face key point recognition model;

[0026] In a completely black image with coordinates consistent with the target two-dimensional face image, the target key points are whitened to obtain the first key point image.

[0027] In one embodiment, the step of establishing the target 3D face model based on the second key point image and the 3D texture image includes:

[0028] The second key point image is pasted onto the first 3D face model to obtain the second 3D face model;

[0029] The point with the largest RGB value in the second 3D face model is determined as the target key point;

[0030] The 3D texture image is applied to the second 3D face model based on the target key points on the second 3D face model to obtain the target 3D face model.

[0031] Secondly, embodiments of the present invention provide a three-dimensional model building device based on two-dimensional key points, comprising:

[0032] The acquisition module is used to acquire the target 2D face image;

[0033] The prediction module is used to perform prediction processing on the target two-dimensional face image through the target two-dimensional face key point recognition model to obtain a first key point image including the target key points, wherein the coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image.

[0034] The reconstruction module is used to perform three-dimensional reconstruction processing on the target two-dimensional face image to obtain a three-dimensional texture image;

[0035] The conversion module is used to perform coordinate transformation processing on the key point coordinates in the first key point image according to the texture map coordinate data of the three-dimensional texture image, so as to obtain a second key point image based on the texture map coordinate data;

[0036] A module is established to create a target 3D face model based on the second key point image and the 3D texture image.

[0037] In one embodiment, a training module is further included, which is used to acquire a public dataset of key points of a two-dimensional face image; and to train the two-dimensional face key point recognition model based on the public dataset of key points of the target two-dimensional face image to obtain the target two-dimensional face key point recognition model.

[0038] In one embodiment, an extraction module is further included, which is used to acquire a two-dimensional image, the two-dimensional image including several portrait information; and to perform face extraction processing on the two-dimensional image by a face generation module to obtain a target two-dimensional face image.

[0039] In one embodiment, the extraction module is further configured to perform face extraction processing on the two-dimensional image through the face generation module to obtain a two-dimensional face image; and to perform recognition processing on the two-dimensional face image to obtain a target two-dimensional face image with an unobstructed frontal view.

[0040] In one embodiment, the module is further configured to attach the second key point image to the first three-dimensional face model to obtain a second three-dimensional face model; determine the target key points from the second three-dimensional face model; and attach the three-dimensional texture image to the second three-dimensional face model based on the target key points on the second three-dimensional face model to obtain a target three-dimensional face model.

[0041] In one embodiment, the prediction module is further configured to input the target two-dimensional face image into the target two-dimensional face key point recognition model to obtain the target key points output by the target two-dimensional face key point recognition model; and to white-encode the target key points in a completely black image with coordinates consistent with the target two-dimensional face image to obtain a first key point image.

[0042] In one embodiment, the module is further configured to paste the second key point image onto the first three-dimensional face model to obtain a second three-dimensional face model; determine the target key point by identifying the point with the largest RGB value in the second three-dimensional face model; and paste the three-dimensional texture image onto the second three-dimensional face model based on the target key point on the second three-dimensional face model to obtain a target three-dimensional face model.

[0043] Thirdly, embodiments of the present invention provide a computer, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the three-dimensional model building method based on two-dimensional key points as described in the first aspect.

[0044] Fourthly, a computer-readable storage medium storing computer-executable instructions for performing the three-dimensional model building method based on two-dimensional key points described in the first aspect.

[0045] This invention includes a method for establishing a 3D model based on 2D keypoints, comprising the following steps: acquiring a target 2D face image; performing prediction processing on the target 2D face image using a target 2D face keypoint recognition model to obtain a first keypoint image including target keypoints, wherein the coordinate information of the target keypoints corresponds to the coordinate information of the target keypoints in the target 2D face image; performing 3D reconstruction processing on the target 2D face image to obtain a 3D texture image; performing coordinate transformation processing on the keypoint coordinates in the first keypoint image according to the texture map coordinate data of the 3D texture image to obtain a second keypoint image based on the texture map coordinate data; and establishing a target 3D face model based on the second keypoint image and the 3D texture image. The technical solution of this embodiment, by binding the mapping relationship between 2D keypoints and 3D texture images, can improve the accuracy of binding corresponding points in the 3D face model, making the established 3D face model closer to real-world usage scenarios.

[0046] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a system architecture platform for executing a three-dimensional model building method based on two-dimensional key points, provided in an embodiment of the present invention;

[0048] Figure 2 This is a flowchart of a method for establishing a three-dimensional model based on two-dimensional key points according to an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the first key point image in a three-dimensional model building method based on two-dimensional key points provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the second key point image in a three-dimensional model building method based on two-dimensional key points provided in an embodiment of the present invention;

[0051] Figure 5 This is a flowchart of training a two-dimensional face key point recognition model in a three-dimensional model building method based on two-dimensional key points provided in an embodiment of the present invention;

[0052] Figure 6 This is a flowchart illustrating the generation of a target 3D face model in a 3D model building method based on 2D key points provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of a three-dimensional model building device based on two-dimensional key points provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0055] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0056] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0057] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0058] Two-dimensional facial landmark detection is a hot topic in computer vision and image processing, widely used in financial identity authentication, access control, intelligent video surveillance, liveness detection, and entertainment, significantly replacing manual resources and effectively improving productivity. However, the quality and quantity of labeled two-dimensional facial landmark data—the "raw material" for 2D facial landmark detection algorithms—have long plagued industry and academia. When the number of points is large, it's difficult to guarantee both accuracy and speed. Furthermore, when faces are occluded or exhibit large poses, different annotators use inconsistent standards for labeling landmarks, leading to ambiguity in data labels. To address the difficulty of labeling data, the common approach is to augment the data in 2D space to obtain more data, such as scaling, rotating, translating, occluding, and enhancing images, while simultaneously transforming the landmarks accordingly. However, faces are three-dimensional in the real world, so 2D augmentation alone is insufficient to simulate and cover real-world application scenarios.

[0059] To address the aforementioned problems, this invention provides a method, computer, and storage medium for establishing a 3D model based on 2D keypoints. The method includes the following steps: acquiring a target 2D face image; performing predictive processing on the target 2D face image using a target 2D face keypoint recognition model to obtain a first keypoint image including target keypoints, wherein the coordinate information of the target keypoints corresponds to the coordinate information of the target keypoints in the target 2D face image; performing 3D reconstruction processing on the target 2D face image to obtain a 3D texture image; performing coordinate transformation processing on the keypoint coordinates in the first keypoint image based on the texture map coordinate data of the 3D texture image to obtain a second keypoint image based on the texture map coordinate data; and establishing a target 3D face model based on the second keypoint image and the 3D texture image. This embodiment's technical solution, by binding the mapping relationship between 2D keypoints and 3D texture images, can improve the accuracy of binding corresponding points in the 3D face model, making the established 3D face model closer to real-world usage scenarios.

[0060] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0061] like Figure 1 As shown, Figure 1 This is a schematic diagram of a system architecture platform 100 for executing a three-dimensional model building method based on two-dimensional key points, provided in an embodiment of the present invention.

[0062] exist Figure 1 In this example, the system architecture platform 100 includes a processor 110 and a memory 120, which can be connected via a bus or other means. Figure 1Taking the example of a connection between China and Israel via a bus.

[0063] Memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 120 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 120 may optionally include memory remotely located relative to processor 110, and these remote memories can be connected to the system architecture platform via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0064] Those skilled in the art will understand that this system architecture platform can be applied to existing communication network systems and subsequent evolved mobile communication network systems, and this embodiment does not specifically limit it.

[0065] It will be understood by those skilled in the art that Figure 1 The system architecture platform shown does not constitute a limitation on the embodiments of the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0066] The system architecture platform 100 can be an independent system architecture platform, or it can be a cloud system architecture platform 100 that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0067] Based on the above system architecture platform, the following are various embodiments of the method for establishing a three-dimensional model based on two-dimensional key points of the present invention.

[0068] like Figure 2 As shown, Figure 2 This is a flowchart of a method for establishing a three-dimensional model based on two-dimensional key points according to an embodiment of the present invention. The method for establishing a three-dimensional model based on two-dimensional key points is applied to the above-mentioned architecture platform, and the method for establishing a three-dimensional model based on two-dimensional key points includes, but is not limited to, steps S100, S200, S300, S400 and S500.

[0069] Step S100: Obtain the target two-dimensional face image.

[0070] Specifically, in order to address the poor performance of 3D face reconstruction, especially in cases of large poses, occlusion, and unclear faces, which would result in subsequent steps obtaining corresponding key points in 3D that differ significantly from the key points in the real image and thus become unusable, the input target 2D face image can be a frontal, unoccluded face image.

[0071] It should be noted that the target two-dimensional face image can be a pre-processed image to be processed, or it can be a directly captured face image. This embodiment does not specifically limit it.

[0072] In one embodiment, a two-dimensional image can be acquired, which includes several portrait information. Then, a face generation module is used to perform face extraction processing on the two-dimensional image to obtain a target two-dimensional face image.

[0073] In one embodiment, a two-dimensional image including several personal portrait information can be acquired. Then, a face generation module performs face extraction processing on the two-dimensional image to obtain all two-dimensional face images in the two-dimensional image. Then, the two-dimensional face images are processed for recognition to obtain a target two-dimensional face image with an unobstructed frontal face.

[0074] It should be noted that the face generation module can be a high-definition face generator or a StyleGan face generator; this embodiment does not specifically limit it.

[0075] Step S200: The target two-dimensional face image is predicted and processed by the target two-dimensional face key point recognition model to obtain the first key point image including the target key points. The coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image.

[0076] Specifically, after acquiring the target two-dimensional face image, the target two-dimensional face image can be input into the target two-dimensional face key point recognition model for prediction processing to obtain the first key point image including the target key points. The coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image. That is, the target key points are determined from the target two-dimensional face image, and then the coordinate information of the target key points in the target two-dimensional face image is obtained. Then, the first key point image is generated from the target key points and their coordinate information.

[0077] In one embodiment, reference is made to Figure 3 The target 2D face image is input into the target 2D face key point recognition model to obtain the target key point (the left eye of the face) output by the target 2D face key point recognition model. Then, the target key point is painted white in a completely black image with the same coordinates as the target 2D face image to obtain the first key point image. The coordinate information of the target key point in the first key point image is the same as the coordinate information of the target key point in the target 2D face image.

[0078] Step S300: Perform three-dimensional reconstruction processing on the target two-dimensional face image to obtain a three-dimensional texture image.

[0079] Specifically, while generating target key points, the target two-dimensional face image is reconstructed into three dimensions to obtain the 3DMM parameters, 3D point cloud and three-dimensional texture image corresponding to the target two-dimensional face image. In this embodiment, the three-dimensional texture image is used. It is understood that the coordinate system of the three-dimensional texture image is different from the coordinate system of the target two-dimensional face image. Therefore, subsequent steps require coordinate transformation processing.

[0080] Step S400: Perform coordinate transformation on the key point coordinates in the first key point image based on the texture map coordinate data of the three-dimensional texture image to obtain the second key point image based on the texture map coordinate data.

[0081] Specifically, after completing the first keypoint image and the 3D texture image, the coordinates of the first keypoint image are transformed using the texture map coordinate data of the 3D texture image as a reference. This transformation ensures that the coordinate information of the target keypoints is accurately converted from the coordinate system of the target 2D face image to the coordinate system of the 3D texture image. Based on the transformed coordinates, a second keypoint image is generated using the texture map coordinate data. The second keypoint image is shown below. Figure 4 As shown.

[0082] Step S500: A target 3D face model is obtained based on the second key point image and the 3D texture image.

[0083] Specifically, after obtaining the second key point image, the target 3D face model corresponding to the 3D texture image can be established using the second key point image. That is, binding is performed through the mapping relationship between the 2D key points and the 3D texture image, which can improve the accuracy of binding the corresponding points in the 3D face model and make the established 3D face model closer to the real use scenario.

[0084] In one embodiment, a target 2D face image without occlusion is first acquired. Then, a target 2D face keypoint recognition model is used to predict the target 2D face image to obtain a first keypoint image including target keypoints, wherein the coordinate information of the target keypoints corresponds to the coordinate information of the target keypoints in the target 2D face image. Simultaneously, the target 2D face image is subjected to 3D reconstruction processing to obtain a 3D texture image. Next, the coordinate transformation processing of the keypoint coordinates in the first keypoint image is performed according to the texture map coordinate data of the 3D texture image to obtain a second keypoint image based on the texture map coordinate data. Finally, a target 3D face model is established based on the second keypoint image and the 3D texture image. The technical solution of this embodiment, by binding the 2D keypoints and the 3D texture image through the mapping relationship, can improve the accuracy of binding the corresponding points in the 3D face model, making the established 3D face model closer to real-world usage scenarios.

[0085] It is understandable that, since the established target 3D face model has been accurately bound to the corresponding points in the 3D face, the rotation, scaling, translation and other transformations of the target 3D face model in 3D space are closer to real-world usage scenarios. Furthermore, the facial features and contour transformations can be controlled through 3DMM shape and expression parameters, resulting in richer augmented content than in 2D space. The target 3D face model established in this embodiment is not limited to scenarios where 2D key points are augmented in 3D space. It should also include any 2D face-related recognition, detection, and generation tasks, such as face recognition, face gender recognition, angle recognition, face occlusion detection, face generation, face occlusion recovery, and face correction. This embodiment does not impose specific limitations on these scenarios.

[0086] Reference Figure 5 In one embodiment, step S200 includes, but is not limited to, steps S710 and S720.

[0087] Step S710: Obtain the public dataset of key points for two-dimensional face images;

[0088] Step S720: Train the two-dimensional face key point recognition model based on the public dataset of key points of the target two-dimensional face image to obtain the target two-dimensional face key point recognition model.

[0089] In one embodiment, a publicly available dataset of key points of two-dimensional faces can be collected via the Internet. Then, a target two-dimensional face key point recognition model can be obtained by building and training a two-dimensional face key point recognition model. It should be noted that a larger model (such as HRnet) should be selected for the two-dimensional face key point recognition model to ensure the accuracy of the model on frontal, unobstructed face images.

[0090] It is understandable that, in addition to publicly available datasets, the data used in this embodiment to train the 2D face landmark recognition model does not require the collection of additional labeled datasets, which can effectively save manpower and resources.

[0091] It should be noted that the two-dimensional facial landmark recognition model can be HRnet or other network models. This embodiment does not limit it to a single model and can be set according to the actual situation.

[0092] Reference Figure 6 In one embodiment, step S500 includes, but is not limited to, steps S810, S820 and S830.

[0093] Step S810: Paste the second key point image onto the first 3D face model to obtain the second 3D face model;

[0094] Step S820: Determine the target key points from the second 3D face model;

[0095] Step S830: Apply the 3D texture image to the second 3D face model based on the target key points on the second 3D face model to obtain the target 3D face model.

[0096] Specifically, after generating a 3D texture image and a second keypoint image with the same coordinate system as the 3D texture image, the second keypoint image is first pasted onto a first 3D face model to obtain a second 3D face model. Then, target keypoints are determined from the second 3D face model to complete the texture mapping binding operation between the target keypoints in the 2D image and the 3D texture image. Finally, the 3D texture image is pasted onto the second 3D face model based on the target keypoints on the second 3D face model to obtain the target 3D face model. The technical solution of this embodiment, by binding the mapping relationship between 2D keypoints and 3D texture images, can improve the accuracy of binding corresponding points in the 3D face model, making the established 3D face model closer to real-world usage scenarios.

[0097] In one embodiment, the acquired target 2D face image is first input into a target 2D face keypoint recognition model to obtain target keypoints output by the model. Then, the target keypoints are painted white in a completely black image with coordinates consistent with the target 2D face image to obtain a first keypoint image. Next, the second keypoint image is pasted onto a first 3D face model to obtain a second 3D face model. Then, the point with the largest RGB value in the second 3D face model is determined as the target keypoint, thus completing the texture mapping binding operation between the target keypoints in the 2D image and the 3D texture image. Finally, the 3D texture image is pasted onto the second 3D face model based on the target keypoints on the second 3D face model to obtain the target 3D face model. The technical solution of this embodiment, by binding the mapping relationship between 2D keypoints and 3D texture images, can improve the accuracy of binding the corresponding points in the 3D face model, making the established 3D face model closer to real-world usage scenarios.

[0098] The 3D texture image is pasted onto the second 3D face model based on the target key points on the second 3D face model to obtain the target 3D face model.

[0099] Based on the above-described method for establishing a 3D model based on 2D key points, the following are various embodiments of the 3D model establishment device, computer, and computer-readable storage medium based on 2D key points of the present invention.

[0100] Reference Figure 7 An embodiment of the present invention also provides a three-dimensional model building apparatus based on two-dimensional key points, comprising:

[0101] The acquisition module 910 is used to acquire the target two-dimensional face image;

[0102] The prediction module 920 is used to perform prediction processing on the target two-dimensional face image through the target two-dimensional face key point recognition model to obtain a first key point image including the target key points, and the coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image.

[0103] The reconstruction module 930 is used to perform three-dimensional reconstruction processing on the target two-dimensional face image to obtain a three-dimensional texture image;

[0104] The conversion module 940 is used to perform coordinate transformation processing on the key point coordinates in the first key point image based on the texture map coordinate data of the three-dimensional texture image to obtain the second key point image based on the texture map coordinate data.

[0105] Module 950 is used to create a target 3D face model based on the second key point image and the 3D texture image.

[0106] In one embodiment, a training module is also included, which is used to acquire a public dataset of key points of a two-dimensional face image; and to train a two-dimensional face key point recognition model based on the public dataset of key points of the target two-dimensional face image to obtain a target two-dimensional face key point recognition model.

[0107] In one embodiment, an extraction module is further included, which is used to acquire a two-dimensional image, the two-dimensional image including several portrait information; and to perform face extraction processing on the two-dimensional image by a face generation module to obtain a target two-dimensional face image.

[0108] In one embodiment, the extraction module is further configured to perform face extraction processing on the two-dimensional image through the face generation module to obtain a two-dimensional face image; and to perform recognition processing on the two-dimensional face image to obtain a target two-dimensional face image with an unobstructed frontal view.

[0109] In one embodiment, the establishment module 950 is further configured to apply the second key point image to the first three-dimensional face model to obtain a second three-dimensional face model; determine target key points from the second three-dimensional face model; and apply a three-dimensional texture image to the second three-dimensional face model based on the target key points on the second three-dimensional face model to obtain a target three-dimensional face model.

[0110] In one embodiment, the prediction module 920 is further configured to input the target two-dimensional face image into the target two-dimensional face key point recognition model to obtain the target key points output by the target two-dimensional face key point recognition model; and to white-encode the target key points in a completely black image with coordinates consistent with the target two-dimensional face image to obtain a first key point image.

[0111] In one embodiment, the module 950 is further configured to attach the second key point image to the first three-dimensional face model to obtain the second three-dimensional face model; determine the target key point by identifying the point with the largest RGB value in the second three-dimensional face model; and attach the three-dimensional texture image to the second three-dimensional face model based on the target key point on the second three-dimensional face model to obtain the target three-dimensional face model.

[0112] It should be noted that the various embodiments of the three-dimensional model building device based on two-dimensional key points described above are consistent with the embodiments of the three-dimensional model building method based on two-dimensional key points in terms of the technical means used, the technical problems solved, and the technical effects achieved. They will not be described in detail here. For details, please refer to the embodiments of the three-dimensional model building method based on two-dimensional key points.

[0113] In addition, one embodiment of the present invention provides a computer, the device including: a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0114] The processor and memory can be connected via a bus or other means.

[0115] It should be noted that the computer in this embodiment may correspond to including, for example, the following: Figure 1 The memory and processor in the illustrated embodiment can constitute Figure 1 The system architecture platform shown in the embodiment is part of the same inventive concept, and therefore has the same implementation principle and beneficial effects, which will not be described in detail here.

[0116] The non-transient software program and instructions required to implement the three-dimensional model building method based on two-dimensional key points in the above embodiments are stored in memory. When executed by a processor, the three-dimensional model building method based on two-dimensional key points in the above embodiments is executed, for example, the method described above is executed. Figure 2 Method steps S100 to S500 Figure 5 Method steps S710 to S720 and Figure 6 Method steps S810 to S830.

[0117] The computer includes components such as radio frequency (RF) circuitry, memory, input unit, display unit, sensors, audio circuitry, wireless fidelity (WiFi) module, processor, and power supply. Those skilled in the art will understand that this embodiment does not uniquely limit the structure of the computer, and may include more or fewer components, combinations of certain components, or different component arrangements.

[0118] RF circuits are used for receiving and transmitting signals during information transmission or calls. Specifically, they receive downlink information from the base station and process it; additionally, they transmit uplink data to the base station. Typically, RF circuits include, but are not limited to, antennas, at least one amplifier, transceivers, couplers, low-noise amplifiers (LNAs), and duplexers. Furthermore, RF circuits can communicate wirelessly with networks and other devices. These wireless communications can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0119] Memory is used to store software programs and modules. The processor executes these stored software programs and modules to perform various computer functions and data processing. Memory primarily includes a program storage area and a data storage area. The program storage area stores the operating system, at least one application program required for a given function (such as sound playback or image playback), etc.; the data storage area stores data created based on computer usage (such as audio data, phonebooks, etc.). Furthermore, memory may include high-speed random access memory (RAM) and non-volatile memory, such as at least one disk storage device, flash memory, or other volatile solid-state storage devices.

[0120] The input unit can be used to receive input numerical or character information, and to generate key signal inputs related to computer settings and function control. Specifically, the input unit may include a touch panel and other input devices. A touch panel, also known as a touchscreen, can collect touch operations (such as operations using fingers, styluses, or any suitable object or accessory on or near the touch panel) and drive corresponding connected devices according to a pre-set program. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the touch location and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor, and can receive and execute commands from the processor. Furthermore, touch panels can be implemented using various types of methods, including resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel, the input unit may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0121] The display unit can be used to display input or provided information, as well as various computer menus. The display unit may include a display panel, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Furthermore, a touch panel may cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event. Although the touch panel and display panel are implemented as two separate components to achieve the computer's input and output functions, in some embodiments, the touch panel and display panel can be integrated to achieve the computer's input and output functions.

[0122] The computer may also include at least one sensor, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel according to the ambient light level, and the proximity sensor can turn off the display panel and / or backlight when the computer is moved close to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that segment the computer's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration segmentation-related functions (such as pedometers, tapping), etc. Other sensors that may be configured in the computer, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be elaborated here.

[0123] Audio circuits, speakers, and microphones provide audio interfaces. The audio circuit converts the received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. On the other hand, the microphone converts the collected sound signals into electrical signals, which are then received by the audio circuit, converted into audio data, and output to the processor for processing. The audio data is then transmitted via RF circuitry to, for example, another computer, or output to memory for further processing.

[0124] WiFi is a short-range wireless transmission technology. Computers using WiFi modules can send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. A WiFi module is not an essential component of a computer and can be omitted as needed without altering the essence of the invention.

[0125] The processor is the control center of a computer. It connects various parts of the computer via various interfaces and lines, and performs various computer functions and processes data by running or executing software programs and / or modules stored in memory, and by accessing data stored in memory, thereby providing overall monitoring of the computer. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor.

[0126] The computer also includes a power supply (such as a battery) that powers the various components. Preferably, the power supply can be connected to the processor logic through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0127] Furthermore, one embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are used to execute the aforementioned terminal-side method for establishing a 3D model based on 2D key points, for example, executing the above-described method... Figure 2 Method steps S100 to S500 Figure 5 Method steps S710 to S720 and Figure 6 Method steps S810 to S830.

[0128] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0129] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for establishing a 3D model based on 2D key points, characterized in that, include: Acquire the target 2D face image; The target two-dimensional face image is predicted by a target two-dimensional face key point recognition model to obtain a first key point image including target key points, and the coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image. The target two-dimensional face image is subjected to three-dimensional reconstruction processing to obtain a three-dimensional texture image; Based on the texture map coordinate data of the three-dimensional texture image, the coordinates of the key points in the first key point image are transformed to obtain the second key point image based on the texture map coordinate data; A target 3D face model is obtained by establishing the second key point image and the 3D texture image; The step of establishing the target 3D face model based on the second key point image and the 3D texture image includes: The second key point image is pasted onto the first 3D face model to obtain the second 3D face model; The point with the largest RGB value in the second 3D face model is determined as the target key point; The three-dimensional texture image is pasted onto the second three-dimensional face model based on the target key points on the second three-dimensional face model to obtain the target three-dimensional face model.

2. The method for establishing a three-dimensional model based on two-dimensional key points according to claim 1, characterized in that, Before performing prediction processing on the target two-dimensional face image using the target two-dimensional face key point recognition model to obtain a first key point image including the target key points, the process includes: Obtain a public dataset of key points for 2D face images; The two-dimensional face key point recognition model is trained using the publicly available dataset of key points of the target two-dimensional face image to obtain the target two-dimensional face key point recognition model.

3. The method for establishing a three-dimensional model based on two-dimensional key points according to claim 1, characterized in that, Before acquiring the target two-dimensional face image, the method includes: Acquire a two-dimensional image, the two-dimensional image including several personal portrait information; The two-dimensional image is processed by a face generation module to extract the face, thereby obtaining the target two-dimensional face image.

4. The method for establishing a three-dimensional model based on two-dimensional key points according to claim 3, characterized in that, The step of performing face extraction processing on the two-dimensional image through a face generation module to obtain a target two-dimensional face image includes: The two-dimensional image is processed by a face generation module to extract faces, thereby obtaining a two-dimensional face image. The two-dimensional face image is processed to obtain a target two-dimensional face image with an unobstructed frontal view.

5. The method for establishing a three-dimensional model based on two-dimensional key points according to claim 1, characterized in that, The step of predicting the target two-dimensional face image using a target two-dimensional face key point recognition model to obtain a first key point image including target key points includes: The target two-dimensional face image is input into the target two-dimensional face key point recognition model to obtain the target key points output by the target two-dimensional face key point recognition model; In a completely black image with coordinates consistent with the target two-dimensional face image, the target key points are painted white to obtain the first key point image.

6. A device for creating a three-dimensional model based on two-dimensional key points, characterized in that, include: The acquisition module is used to acquire the target 2D face image; The prediction module is used to perform prediction processing on the target two-dimensional face image through the target two-dimensional face key point recognition model to obtain a first key point image including the target key points, wherein the coordinate information of the target key points corresponds to the coordinate information of the target key points in the target two-dimensional face image. The reconstruction module is used to perform three-dimensional reconstruction processing on the target two-dimensional face image to obtain a three-dimensional texture image; The conversion module is used to perform coordinate transformation processing on the key point coordinates in the first key point image according to the texture map coordinate data of the three-dimensional texture image, so as to obtain a second key point image based on the texture map coordinate data; A module is established to create a target 3D face model based on the second key point image and the 3D texture image. The step of establishing the target 3D face model based on the second key point image and the 3D texture image includes: The second key point image is pasted onto the first 3D face model to obtain the second 3D face model; The point with the largest RGB value in the second 3D face model is determined as the target key point; The three-dimensional texture image is pasted onto the second three-dimensional face model based on the target key points on the second three-dimensional face model to obtain the target three-dimensional face model.

7. A computer, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the method for establishing a three-dimensional model based on two-dimensional key points as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing computer-executable instructions for performing the three-dimensional model building method based on two-dimensional key points as described in any one of claims 1 to 5.

Images