Method, device, electronic device and readable storage medium for acquiring three-dimensional grid data

Abstract

The present disclosure provides a method, device, electronic device and readable storage medium for acquiring three-dimensional mesh data, which relates to the field of artificial intelligence technology, specifically computer vision, virtual reality, large models and other technical fields, and can be applied to scenes such as artificial intelligence content generation. Among them, the method for acquiring three-dimensional mesh data includes: acquiring a source expression base and initial three-dimensional mesh data corresponding to a target person, the initial three-dimensional mesh data and the source expression base corresponding to different technical specifications; performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, the target expression base and the source expression base corresponding to the same technical specifications; obtaining target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, the target three-dimensional mesh data and the source expression base corresponding to the same technical specifications.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically computer vision, virtual reality, large models and other technical fields, and can be applied to scenes such as artificial intelligence content generation. A method, device, electronic device and readable storage medium for acquiring three-dimensional mesh data are provided. Background Art

[0002] With the rapid development of artificial intelligence technology, more and more technical specifications for producing expression data have emerged. However, before the emergence of new technical specifications, there are still 3D mesh data collected using the original technical specifications. Under the new technical specifications, it is impossible to directly use the 3D mesh data corresponding to the original technical specifications to produce expression data that meets the new technical specifications, resulting in a low utilization rate of these 3D mesh data. Summary of the invention

[0003] According to a first aspect of the present disclosure, a method for acquiring three-dimensional mesh data is provided, comprising: acquiring a source expression base and initial three-dimensional mesh data corresponding to a target character, the initial three-dimensional mesh data and the source expression base corresponding to different technical specifications; performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target character, the target expression base and the source expression base corresponding to the same technical specifications; obtaining target three-dimensional mesh data corresponding to the target character according to the initial three-dimensional mesh data and the target expression base, the target three-dimensional mesh data and the source expression base corresponding to the same technical specifications.

[0004] According to a second aspect of the present disclosure, a device for acquiring three-dimensional mesh data is provided, comprising: an acquisition unit, used to acquire a source expression base and initial three-dimensional mesh data corresponding to a target person, the initial three-dimensional mesh data and the source expression base corresponding to different technical specifications; a migration unit, used to perform deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, the target expression base and the source expression base corresponding to the same technical specifications; a processing unit, used to obtain target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, the target three-dimensional mesh data and the source expression base corresponding to the same technical specifications.

[0005] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0006] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method as described above.

[0007] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method as described above when executed by a processor.

[0008] It can be seen from the above technical solutions that the present invention is able to convert initial three-dimensional mesh data into target three-dimensional mesh data that meets specific technical specifications, thereby achieving the purpose of more effectively utilizing existing initial three-dimensional mesh data that does not meet specific technical specifications, reducing the cost of acquiring target three-dimensional mesh data that meets specific technical specifications, and improving the utilization rate of the initial three-dimensional mesh data and the efficiency of acquiring the target three-dimensional mesh data.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0016] Figure 6 It is a block diagram of an electronic device used to implement the method for acquiring three-dimensional mesh data according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and mechanisms is omitted in the following description.

[0018] Figure 1 Schematic diagram of the first embodiment of the present disclosure. Figure 1 As shown, the method for obtaining three-dimensional grid data in this embodiment specifically includes the following steps:

[0019] S101, obtaining a source expression base and initial three-dimensional mesh data corresponding to a target person, wherein the initial three-dimensional mesh data and the source expression base correspond to different technical specifications;

[0020] S102, performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, wherein the target expression base corresponds to the same technical specification as the source expression base;

[0021] S103, obtaining target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, wherein the target three-dimensional mesh data corresponds to the same technical specification as the source expression base.

[0022] The method for acquiring three-dimensional mesh data of this embodiment, after acquiring a source expression base corresponding to different technical specifications and initial three-dimensional mesh data corresponding to a target person, first performs deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, and then obtains target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base. The obtained target three-dimensional mesh data corresponds to the same technical specifications as the source expression base. This embodiment can convert the initial three-dimensional mesh data into target three-dimensional mesh data that meets specific technical specifications, thereby achieving the purpose of more effectively utilizing the existing initial three-dimensional mesh data that does not meet specific technical specifications, reducing the cost of acquiring target three-dimensional mesh data that meets specific technical specifications, and improving the utilization rate of the initial three-dimensional mesh data and the efficiency of acquiring the target three-dimensional mesh data.

[0023] The source expression base obtained by executing S101 in this embodiment may correspond to an ARKit (Augmented Reality Toolkit) technical specification. The ARKit technical specification is a technical specification used for facial expression capture in the ARKit development platform. Since the technical specification has good expression decoupling characteristics, a large number of facial expression capture technologies use the technical specification. It can be understood that the technical specification corresponding to the source expression base may also be other technical specifications, which is not limited in this embodiment.

[0024] When executing S101, this embodiment can obtain multiple source expression bases. For example, when the technical specification is the ARKit technical specification, 52 expression bases based on ARKit can be obtained as source expression bases; different source expression bases correspond to different facial expressions of the human face. For example, the facial expression of the human face corresponding to source expression base 1 is blinking of the left eye, the facial expression of the human face corresponding to source expression base 2 is looking downward of the left eye, and the facial expression of the human face corresponding to source expression base 3 is pursing the lips, etc.

[0025] The initial three-dimensional mesh data obtained by executing S101 in this embodiment is three-dimensional mesh data of a head or three-dimensional mesh data of a face. The initial three-dimensional mesh data can be an image frame or a video containing multiple image frames. The three-dimensional mesh data is composed of multiple meshes, each of which is a triangle or a quadrilateral. Taking a triangle as an example, each mesh can include three vertices. Therefore, the initial three-dimensional mesh data in this embodiment is composed of different vertices and is a collection of vertices.

[0026] When executing S101 to obtain initial three-dimensional mesh data corresponding to the target person, the present embodiment may adopt the following implementation method: obtaining identification information of the target person, where the identification information obtained may be the person's name, person number, etc.; obtaining three-dimensional mesh data corresponding to the identification information from a three-dimensional mesh data set as the initial three-dimensional mesh data corresponding to the target person, where the three-dimensional mesh data set contains initial three-dimensional mesh data corresponding to different persons.

[0027] Among them, the target person in this embodiment can be one or more; that is, this embodiment can convert the initial three-dimensional mesh data corresponding to a specific person in the three-dimensional mesh data set into technical specifications, or can convert the initial three-dimensional mesh data corresponding to all people in the three-dimensional mesh data set into technical specifications.

[0028] It can be understood that the technical specifications corresponding to the three-dimensional mesh data in the three-dimensional mesh data set of this embodiment are not the same as the technical specifications corresponding to the source expression base, and different initial three-dimensional mesh data corresponding to the target person may also correspond to different technical specifications; for ease of understanding, the technical specifications corresponding to the source expression base can be called target technical specifications, and the technical specifications corresponding to the initial three-dimensional mesh data can be called source technical specifications, and the source technical specifications are not the same as the target technical specifications.

[0029] Preferably, the technical specifications corresponding to the three-dimensional mesh data in this embodiment may be technical specifications that existed before the technical specifications corresponding to the source expression base appeared, such as three-dimensional deformable model (3DMM) technical specifications that existed before ARKit technical specifications.

[0030] After executing S101 to obtain the source expression base and the initial three-dimensional mesh data corresponding to the target person, this embodiment executes S102 to perform deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain the target expression base corresponding to the target person; wherein the obtained target expression base corresponds to the same technical specifications as the source expression base.

[0031] Since the human face corresponding to the source expression base is not the same as the human face corresponding to the initial three-dimensional mesh data, this embodiment needs to convert the human face corresponding to the source expression base into the human face corresponding to the initial three-dimensional mesh data, so as to improve the accuracy of the target three-dimensional mesh data subsequently obtained based on the target expression base.

[0032] In this embodiment, when executing S102 to deform and migrate the source expression base according to the initial three-dimensional mesh data to obtain the target expression base corresponding to the target character, the implementation method that can be adopted is: according to the target topological structure of the initial three-dimensional mesh data, the source topological structure of the source expression base is retopologically processed to obtain the initial expression base, and the topological structure of the obtained initial expression base is the target topological structure; according to the initial three-dimensional mesh data, the initial expression base is deformed and migrated to obtain the target expression base corresponding to the target character.

[0033] In this embodiment, the topological structure is used to characterize the structure of the three-dimensional mesh data or the expression base; retopology processing refers to changing the topological structure of the source expression base so that the topological structure of the initial expression base obtained after the retopology processing is the same as the topological structure of the three-dimensional mesh data.

[0034] That is to say, since the topological structure of the initial three-dimensional mesh data may vary depending on different head or face models, and the topological structure of the initial three-dimensional mesh data may be different from the topological structure of the source expression base, it is necessary to convert the topological structure of the source expression base into the topological structure of the initial three-dimensional mesh data, so as to improve the accuracy of deformation migration.

[0035] When executing S102, this embodiment can use existing retopology tools (such as wrap4d) to convert the topological structure; when executing S102, this embodiment can use existing deformation transfer (DT) algorithm to obtain the target expression base corresponding to the target person.

[0036] When executing S102, this embodiment may also include the following contents: when it is determined that the target topological structure of the initial three-dimensional mesh data is inconsistent with the source topological structure of the source expression base, the source topological structure of the source expression base is retopologically processed according to the target topological structure of the initial three-dimensional mesh data to obtain the initial expression base.

[0037] That is to say, this embodiment can also compare the target topology structure with the source topology structure before the retopology processing, and then perform the retopology processing only when the two are inconsistent, thereby improving the accuracy of the retopology processing and avoiding the waste of computing resources.

[0038] In addition, if the present embodiment performs S102 to determine that the target topological structure of the initial three-dimensional mesh data is consistent with the source topological structure of the source expression base, the source expression base is directly used as the initial expression base, and then the deformation migration operation is performed.

[0039] After executing S102 to obtain the target expression base corresponding to the target person, this embodiment executes S103 to obtain target three-dimensional mesh data corresponding to the target person based on the initial three-dimensional mesh data and the target expression base; wherein the obtained target three-dimensional mesh data corresponds to the same technical specifications as the source expression base.

[0040] Specifically, when the present embodiment executes S103 to obtain the target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, the implementation method that can be adopted is: according to the model size in the initial three-dimensional mesh data, obtain the camera parameters; according to the model posture in the initial three-dimensional mesh data, obtain the model posture parameters, and the model posture reflects whether the model is tilting its head, lowering its head, raising its head, normal, etc.; according to the model expression of the initial three-dimensional mesh data, obtain the expression base weight parameters corresponding to the target expression base. When there are multiple target expression bases, the present embodiment will respectively obtain the expression base weight parameters corresponding to different target expression bases; according to the camera parameters, the model posture parameters, the target expression base, and the expression base parameters corresponding to the target expression base, obtain the target three-dimensional mesh data corresponding to the target person.

[0041] For example, if the target expression base includes expression base 1, expression base 2 and expression base 3, if according to the model expression, the expression base weight parameter corresponding to expression base 1 is determined to be 0.2, the expression base weight parameter corresponding to expression base 2 is 0.3, and the expression base weight parameter corresponding to expression base 3 is 0.5, then when executing S103, this embodiment can obtain the calculation result of (expression base 1×0.2+expression base 2×0.3+expression base 3×0.5) and then obtain the target three-dimensional mesh data according to the calculation result, camera parameters and model pose parameters.

[0042] It can be understood that if the initial three-dimensional mesh data is a video containing multiple image frames, then when executing S103, this embodiment can obtain corresponding camera parameters, model posture parameters and expression-based weight parameters for the three-dimensional mesh data corresponding to each image frame in the initial three-dimensional mesh data, and then obtain the target three-dimensional mesh data corresponding to the image frame, and then compose a video with the target three-dimensional mesh data corresponding to all image frames.

[0043] That is to say, this embodiment obtains camera parameters, model posture parameters and expression base weight parameters according to the initial three-dimensional mesh data, and then combines the target expression base corresponding to the target person to obtain the target three-dimensional mesh data corresponding to the target person, thereby realizing the conversion of technical specifications, so that the target three-dimensional mesh data corresponds to the same technical specifications as the source expression base, thereby improving the application scope of the target three-dimensional mesh data.

[0044] After executing S103 to obtain the target three-dimensional mesh data corresponding to the target person, this embodiment may also include the following contents: obtaining a target texture map. In this embodiment, the texture map corresponding to the target scene may be used as the target texture map; and obtaining a virtual image corresponding to the target person based on the target texture map and the target three-dimensional mesh data corresponding to the target person.

[0045] That is to say, after acquiring the target three-dimensional mesh data, this embodiment can also combine the acquired target texture map to obtain a virtual image corresponding to the target person, so that the obtained virtual image also conforms to the technical specifications corresponding to the source expression base, thereby improving the application scope of the virtual image.

[0046] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure. Figure 2, a flowchart of the present embodiment when acquiring target three-dimensional mesh data is shown in the figure: S201, acquiring a three-dimensional mesh data set and a source expression base based on ARKit, wherein the three-dimensional mesh data set includes initial three-dimensional mesh data corresponding to different characters; S202, retopology processing, i.e. converting the topological structure of the source expression base based on ARKit into the topological structure of the three-dimensional mesh data set, and obtaining an initial expression base based on ARKit that conforms to the topological structure of the three-dimensional mesh data set; S203, deformation migration processing, i.e. performing deformation migration on the initial expression base based on ARKit according to the initial three-dimensional mesh data corresponding to the target character, and obtaining a target expression base based on ARKit corresponding to the target character; S204, fitting solution processing, i.e. obtaining target three-dimensional mesh data based on ARKit corresponding to the target character according to the initial three-dimensional mesh data corresponding to the target character and a target expression base based on ARKit corresponding to the target character.

[0047] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure. Figure 3 , a flowchart of the present embodiment during fitting and solving is shown in the figure: S301, initializing all parameters, including camera parameters, model pose parameters and expression base weight parameters; S302, optimizing the camera parameters and model pose parameters according to the initial three-dimensional mesh data, that is, making the camera parameters and model pose parameters after initialization correspond to the model size and model pose in the initial three-dimensional mesh data; S303, optimizing the expression base weight parameters according to the initial three-dimensional mesh data, so that the expression base weight parameters are aligned with the model expression in the initial three-dimensional mesh data; S304, fine-tuning and optimizing the optimized camera parameters, model pose parameters and expression base weight parameters, and the present embodiment can perform fine-tuning and optimization manually; S305, obtaining the target three-dimensional mesh data corresponding to the target person according to the parameters after fine-tuning and optimization, and the target expression base.

[0048] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure. Figure 4 Schematic diagrams of initial three-dimensional grid data and target three-dimensional grid data are shown in FIG. In order to improve the display effect, the grids in the three-dimensional grid data have been filled; Figure 4 It can be seen that after completing the conversion of the technical specifications of the three-dimensional mesh data, the expression of the target person has not changed significantly, and the conversion effect is good.

[0049] Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure. Figure 5 As shown, the three-dimensional mesh data acquisition device 500 of this embodiment includes:

[0050] An acquisition unit 501 is used to acquire a source expression base and initial three-dimensional mesh data corresponding to a target person, wherein the initial three-dimensional mesh data and the source expression base correspond to different technical specifications;

[0051] A migration unit 502 is used to perform deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, wherein the target expression base corresponds to the same technical specification as the source expression base;

[0052] The processing unit 503 is used to obtain target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, wherein the target three-dimensional mesh data corresponds to the same technical specification as the source expression base.

[0053] The source expression base obtained by the acquisition unit 501 may correspond to an ARKit technical specification. The ARKit technical specification is a technical specification used in the ARKit development platform for capturing facial expressions of human faces. Since the technical specification has a good expression decoupling characteristic, a large number of facial expression capture technologies use the technical specification. It can be understood that the technical specification corresponding to the source expression base may also be other technical specifications, which is not limited in this embodiment.

[0054] The acquisition unit 501 can acquire multiple source expression bases. For example, when the technical specification is the ARKit technical specification, 52 expression bases based on ARKit can be acquired as source expression bases; different source expression bases correspond to different facial expressions of the human face. For example, the facial expression of the human face corresponding to source expression base 1 is blinking of the left eye, the facial expression of the human face corresponding to source expression base 2 is looking downward of the left eye, and the facial expression of the human face corresponding to source expression base 3 is pursing of the lips, etc.

[0055] The initial three-dimensional mesh data acquired by the acquisition unit 501 is three-dimensional mesh data of a head or three-dimensional mesh data of a face. The initial three-dimensional mesh data can be an image frame or a video containing multiple image frames. The three-dimensional mesh data is composed of multiple meshes, each mesh is a triangle or a quadrilateral. Taking a triangle as an example, each mesh can include three vertices. Therefore, the initial three-dimensional mesh data in this embodiment is composed of different vertices and is a collection of vertices.

[0056] When the acquisition unit 501 acquires the initial three-dimensional mesh data corresponding to the target person, the implementation method that can be adopted is: acquiring the identification information of the target person; acquiring the three-dimensional mesh data corresponding to the identification information from the three-dimensional mesh data set as the initial three-dimensional mesh data corresponding to the target person, and the three-dimensional mesh data set contains the initial three-dimensional mesh data corresponding to different persons.

[0057] Among them, the target person in this embodiment can be one or more; that is, the acquisition unit 501 can convert the initial three-dimensional mesh data corresponding to a specific person in the three-dimensional mesh data set into technical specifications, or can convert the initial three-dimensional mesh data corresponding to all people in the three-dimensional mesh data set into technical specifications.

[0058] It can be understood that the technical specifications corresponding to the three-dimensional mesh data in the three-dimensional mesh data set of this embodiment are not the same as the technical specifications corresponding to the source expression base, and different initial three-dimensional mesh data corresponding to the target person may also correspond to different technical specifications; for ease of understanding, the technical specifications corresponding to the source expression base can be called target technical specifications, and the technical specifications corresponding to the initial three-dimensional mesh data can be called source technical specifications, and the source technical specifications are not the same as the target technical specifications.

[0059] Preferably, the technical specifications corresponding to the three-dimensional mesh data in this embodiment may be technical specifications that existed before the technical specifications corresponding to the source expression base appeared, such as three-dimensional deformable model (3DMM) technical specifications that existed before ARKit technical specifications.

[0060] In this embodiment, after the acquisition unit 501 acquires the source expression base and the initial three-dimensional mesh data corresponding to the target person, the migration unit 502 performs deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain the target expression base corresponding to the target person; wherein the obtained target expression base corresponds to the same technical specifications as the source expression base.

[0061] Since the human face corresponding to the source expression base is not the same as the human face corresponding to the initial three-dimensional mesh data, this embodiment needs to convert the human face corresponding to the source expression base into the human face corresponding to the initial three-dimensional mesh data, so as to improve the accuracy of the target three-dimensional mesh data subsequently obtained based on the target expression base.

[0062] When the migration unit 502 performs deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain the target expression base corresponding to the target character, the implementation method that can be adopted is: according to the target topological structure of the initial three-dimensional mesh data, the source topological structure of the source expression base is retopologically processed to obtain the initial expression base, and the topological structure of the obtained initial expression base is the target topological structure; according to the initial three-dimensional mesh data, the initial expression base is deformed and migrated to obtain the target expression base corresponding to the target character.

[0063] In this embodiment, the topological structure is used to characterize the structure of the three-dimensional mesh data or the expression base; retopology processing refers to changing the topological structure of the source expression base so that the topological structure of the initial expression base obtained after the retopology processing is the same as the topological structure of the three-dimensional mesh data.

[0064] That is to say, since the topological structure of the initial three-dimensional mesh data may vary depending on different head or face models, and the topological structure of the initial three-dimensional mesh data may be different from the topological structure of the source expression base, it is necessary to convert the topological structure of the source expression base into the topological structure of the initial three-dimensional mesh data, so as to improve the accuracy of deformation migration.

[0065] The migration unit 502 may use an existing retopology tool (such as wrap4d) to convert the topological structure; the migration unit 502 may use an existing deformation transfer (DT) algorithm to obtain a target expression base corresponding to the target person.

[0066] The migration unit 502 may also include the following contents: when it is determined that the target topology structure of the initial three-dimensional mesh data is inconsistent with the source topology structure of the source expression base, the source topology structure of the source expression base is retopologically processed according to the target topology structure of the initial three-dimensional mesh data to obtain the initial expression base.

[0067] That is, the migration unit 502 may also compare the target topology with the source topology before the retopology processing, and then perform the retopology processing only when the two are inconsistent, thereby improving the accuracy of the retopology processing and avoiding the waste of computing resources.

[0068] In addition, if the migration unit 502 determines that the target topological structure of the initial three-dimensional mesh data is consistent with the source topological structure of the source expression base, the source expression base is directly used as the initial expression base to perform the deformation migration operation.

[0069] In this embodiment, after the migration unit 502 obtains the target expression base corresponding to the target person, the processing unit 503 obtains the target three-dimensional mesh data corresponding to the target person based on the initial three-dimensional mesh data and the target expression base; wherein the obtained target three-dimensional mesh data corresponds to the same technical specifications as the source expression base.

[0070] Specifically, when the processing unit 503 obtains the target three-dimensional mesh data corresponding to the target person based on the initial three-dimensional mesh data and the target expression base, the implementation method that can be adopted is: obtaining camera parameters according to the model size in the initial three-dimensional mesh data; obtaining model pose parameters according to the model pose in the initial three-dimensional mesh data; obtaining expression base weight parameters corresponding to the target expression base according to the model expression of the initial three-dimensional mesh data. When there are multiple target expression bases, this embodiment will respectively obtain expression base weight parameters corresponding to different target expression bases; obtaining the target three-dimensional mesh data corresponding to the target person according to the camera parameters, model pose parameters, target expression base, and expression base parameters corresponding to the target expression base.

[0071] It can be understood that if the initial three-dimensional mesh data is a video containing multiple image frames, the processing unit 503 can obtain the corresponding camera parameters, model pose parameters and expression base weight parameters for the three-dimensional mesh data corresponding to each image frame in the initial three-dimensional mesh data, and then obtain the target three-dimensional mesh data corresponding to the image frame.

[0072] That is to say, the processing unit 503 obtains the camera parameters, model posture parameters and expression base weight parameters according to the initial three-dimensional mesh data, and then combines the target expression base corresponding to the target person to obtain the target three-dimensional mesh data corresponding to the target person, thereby realizing the conversion of technical specifications, so that the target three-dimensional mesh data corresponds to the same technical specifications as the source expression base, thereby improving the application scope of the target three-dimensional mesh data.

[0073] The three-dimensional mesh data acquisition device 500 of this embodiment may further include a generation unit 504 for executing: acquiring a target texture map; and obtaining a virtual image corresponding to the target person based on the target texture map and the target three-dimensional mesh data corresponding to the target person.

[0074] That is to say, after obtaining the target three-dimensional mesh data, the generation unit 504 can combine the obtained target texture map to obtain a virtual image corresponding to the target person, so that the obtained virtual image also conforms to the technical specifications corresponding to the source expression base, thereby improving the application scope of the virtual image.

[0075] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0076] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0077] like Figure 6 , is a block diagram of an electronic device according to a method for acquiring three-dimensional mesh data according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0078] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0079] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0080] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a method for obtaining three-dimensional mesh data. For example, in some embodiments, the method for obtaining three-dimensional mesh data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608.

[0081] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for acquiring three-dimensional mesh data described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method for acquiring three-dimensional mesh data in any other appropriate manner (e.g., by means of firmware).

[0082] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable three-dimensional mesh data acquisition device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0086] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0087] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.

[0088] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0089] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for acquiring three-dimensional grid data, comprising: Acquire a source expression base and initial three-dimensional mesh data corresponding to a target person, wherein the initial three-dimensional mesh data and the source expression base correspond to different technical specifications, and the technical specification corresponding to the source expression base is an ARKit technical specification; Performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, wherein the target expression base corresponds to the same technical specification as the source expression base, and the technical specification corresponding to the target expression base is the ARKit technical specification; According to the initial three-dimensional mesh data and the target expression base, target three-dimensional mesh data corresponding to the target person is obtained, wherein the target three-dimensional mesh data and the source expression base correspond to the same technical specification, and the technical specification corresponding to the target three-dimensional mesh data is an ARKit technical specification; Wherein, obtaining the source expression base includes: Acquire an expression base based on an augmented reality toolkit ARKit as the source expression base; The step of obtaining target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base comprises: Obtaining camera parameters according to the model size in the initial three-dimensional mesh data; Acquiring model pose parameters according to the model pose in the initial three-dimensional mesh data; According to the model expression of the initial three-dimensional mesh data, acquiring an expression base weight parameter corresponding to the target expression base; The target three-dimensional mesh data corresponding to the target person is obtained according to the camera parameters, the model posture parameters, the target expression base, and the expression base parameters corresponding to the target expression base.

2. The method according to claim 1, wherein: The step of performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person comprises: According to the target topological structure of the initial three-dimensional mesh data, retopologically process the source topological structure of the source expression base to obtain an initial expression base, wherein the topological structure of the initial expression base is the target topological structure; The initial expression base is deformed and migrated according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target person.

3. The method according to claim 2, wherein: The retopological processing of the source topological structure of the source expression base according to the target topological structure of the initial three-dimensional mesh data to obtain the initial expression base comprises: When it is determined that the target topology structure is inconsistent with the source topology structure, the source topology structure is retopologically processed according to the target topology structure to obtain the initial expression base.

4. The method according to claim 1, wherein: The obtaining of initial three-dimensional grid data corresponding to the target person includes: Obtaining identification information of the target person; The three-dimensional mesh data corresponding to the identification information is acquired from the three-dimensional mesh data set as the initial three-dimensional mesh data corresponding to the target person.

5. The method according to claim 1, further comprising: Get the target texture map; A virtual image corresponding to the target person is obtained according to the target texture map and the target three-dimensional grid data corresponding to the target person.

6. A device for acquiring three-dimensional grid data, comprising: An acquisition unit, used for acquiring a source expression base and initial three-dimensional mesh data corresponding to a target person, wherein the initial three-dimensional mesh data and the source expression base correspond to different technical specifications, and the technical specification corresponding to the source expression base is an ARKit technical specification; A migration unit, configured to perform deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, wherein the target expression base corresponds to the same technical specification as the source expression base, and the technical specification corresponding to the target expression base is an ARKit technical specification; A processing unit, configured to obtain target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, wherein the target three-dimensional mesh data corresponds to the same technical specification as the source expression base, and the technical specification corresponding to the target three-dimensional mesh data is an ARKit technical specification; Wherein, when acquiring the source expression base, the acquisition unit specifically performs: Acquire an expression base based on an augmented reality toolkit ARKit as the source expression base; When the processing unit obtains the target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression base, the processing unit specifically performs: Obtaining camera parameters according to the model size in the initial three-dimensional mesh data; Acquiring model pose parameters according to the model pose in the initial three-dimensional mesh data; According to the model expression of the initial three-dimensional mesh data, acquiring an expression base weight parameter corresponding to the target expression base; The target three-dimensional mesh data corresponding to the target person is obtained according to the camera parameters, the model posture parameters, the target expression base, and the expression base parameters corresponding to the target expression base.

7. The device according to claim 6, wherein: When the migration unit performs deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, the migration unit specifically performs: According to the target topological structure of the initial three-dimensional mesh data, retopologically process the source topological structure of the source expression base to obtain an initial expression base, wherein the topological structure of the initial expression base is the target topological structure; The initial expression base is deformed and migrated according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target person.

8. The device according to claim 7, wherein: When the migration unit performs a retopology process on the source topology structure of the source expression base according to the target topology structure of the initial three-dimensional mesh data to obtain the initial expression base, the migration unit specifically executes: When it is determined that the target topology structure is inconsistent with the source topology structure, the source topology structure is retopologically processed according to the target topology structure to obtain the initial expression base.

9. The device according to claim 6, wherein: When acquiring the initial three-dimensional mesh data corresponding to the target person, the acquisition unit specifically performs: Obtaining identification information of the target person; The three-dimensional mesh data corresponding to the identification information is acquired from the three-dimensional mesh data set as the initial three-dimensional mesh data corresponding to the target person.

10. The apparatus according to claim 6, further comprising a generating unit, configured to execute: Get the target texture map; A virtual image corresponding to the target person is obtained according to the target texture map and the target three-dimensional grid data corresponding to the target person.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images