Facial capture data processing method and apparatus

Abstract

The embodiment of the present application provides a face capture data processing method, the method comprises the following steps: obtaining face capture data; inputting the face capture data into a preset protocol structure, so as to obtain first mixed deformation data by using the preset protocol structure, wherein the first mixed deformation data at least includes mixed deformation data corresponding to a target model, and the preset protocol structure is used for generating mixed deformation data corresponding to at least one mixed deformation protocol according to the face capture data; and driving the target model according to the mixed deformation data corresponding to the target model. The face capture data processing method provided by the embodiment of the present application can reduce the adaptation cost of different mixed deformation protocols used by users.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of animation, and in particular to a face capture data processing method and device, a computer device, and a storage medium. BACKGROUND

[0002] Face motion capture is a process of converting the motion of a human face into a digital database electronically using a camera or a laser scanner. A camera captures a face picture, and a face capture engine calculates a series of feature points. A corresponding induction algorithm module obtains a series of BS (BlendShape) values that can drive a virtual character, so as to drive the virtual character to generate a face motion corresponding to the BS values. When the blendshape protocol is different, the algorithm used to calculate the BS values and the model driven are different.

[0003] Currently, there are various blendshape protocols, such as the ARkit BlendShape protocol open sourced by Apple and the BlendShape protocol open sourced by the VRM model. However, since these protocols are incompatible, if two or more protocols need to be used in a project, the user needs to do two or more blendshape channels to adapt to different protocols, which increases the adaptation cost of the user. SUMMARY

[0004] The present application aims to provide a face capture data processing method and device, a computer device, and a storage medium, to solve the technical problem of high adaptation cost of different blendshape protocols used by users at present.

[0005] One aspect of an embodiment of the present application provides a face capture data processing method, including: obtaining face capture data; inputting the face capture data into a preset protocol structure to obtain first blendshape data by using the preset protocol structure, wherein the first blendshape data at least includes blendshape data corresponding to a target model, and the preset protocol structure is used to generate blendshape data corresponding to at least one blendshape protocol according to the face capture data; and driving the target model according to the blendshape data corresponding to the target model.

[0006] Optionally, the step of inputting the face capture data into the preset protocol structure to obtain the first blendshape data by using the preset protocol structure includes: determining a target blendshape protocol corresponding to the target model; and inputting the face capture data into the preset protocol structure to obtain target blendshape data corresponding to the target blendshape protocol by using the preset protocol structure.

[0007] Optionally, the inputting the face capture data into the preset protocol structure to obtain first blendshape data comprises: inputting the face capture data into the preset protocol structure to obtain blendshape data corresponding to at least two blendshape protocols by using the preset protocol structure.

[0008] Optionally, the method further comprises: obtaining second blendshape data; and converting the second blendshape data into third blendshape data by using the preset protocol structure, wherein the second blendshape data and the third blendshape data correspond to different blendshape protocols.

[0009] Optionally, the target model comprises a plurality of channels of special expressions, the channels correspond to fourth blendshape data, and the method further comprises: inputting the face capture data into the preset protocol structure to obtain the fourth blendshape data by using the preset protocol structure, the preset protocol structure being further used to generate the fourth blendshape data according to the face capture data; and inputting the fourth blendshape data into the special expression channels to drive the target model to generate the special expressions.

[0010] Optionally, the blendshape protocol comprises an ARkitBlendShape protocol and a VRMBlendShapeProxy protocol.

[0011] An aspect of an embodiment of the present application further provides a face capture data processing apparatus, comprising: an obtaining module configured to obtain face capture data; an inputting module configured to input the face capture data into a preset protocol structure to obtain first blendshape data by using the preset protocol structure, wherein the first blendshape data at least comprises blendshape data corresponding to a target model, and the preset protocol structure is used to generate blendshape data corresponding to at least one blendshape protocol according to the face capture data; and a driving module configured to drive the target model according to the blendshape data corresponding to the target model.

[0012] Optionally, the inputting module is further configured to: determine a target blendshape protocol corresponding to the target model; and input the face capture data into the preset protocol structure to obtain blendshape data corresponding to the target blendshape protocol by using the preset protocol structure.

[0013] An aspect of an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor is configured to implement the steps of the face capture data processing method described above when executing the computer program.

[0014] Another aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by at least one processor to perform the steps of the facial capture data processing method described above.

[0015] The facial capture data processing method, apparatus, computer equipment, and storage medium provided in this application have the following advantages:

[0016] By acquiring facial capture data and inputting it into a preset protocol structure, the system obtains first hybrid deformation data, including hybrid deformation data corresponding to the target model. Then, the system drives the target model based on the hybrid deformation data corresponding to the target model, generating hybrid deformation data corresponding to the target model to drive the target model to make corresponding facial movements or expressions. Since the preset protocol structure can generate hybrid deformation data corresponding to more than one hybrid deformation protocol, users do not need to create more than two sets of hybrid deformation channels to adapt to different hybrid deformation protocols, thereby reducing the adaptation cost for users using different hybrid deformation protocols. Attached Figure Description

[0017] Figure 1 This illustration schematically shows an application environment diagram of an embodiment of this application;

[0018] Figure 2 The flowchart of the facial capture data processing method of Embodiment 1 of this application is illustrated schematically;

[0019] Figure 3 for Figure 2 A flowchart of the sub-steps of step S320;

[0020] Figure 4 for Figure 2 The flowchart for the newly added steps;

[0021] Figure 5 for Figure 2 A flowchart of another newly added step;

[0022] Figure 6 This is a schematic diagram illustrating the principle structure of a preset protocol.

[0023] Figure 7 A block diagram of the facial capture data processing device according to Embodiment 2 of this application is shown schematically;

[0024] Figure 8 The schematic diagram illustrates the hardware architecture of the computer device according to Embodiment 3 of this application. Detailed Implementation

[0025] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0026] It should be noted that the description involving "first", "second" and the like in the embodiments of the present application is only for the purpose of description, and should not be understood as indicating or implying the relative importance of the technical features indicated or implicitly indicating the number of technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of those of ordinary skill in the art, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection claimed by the present application.

[0027] In the description of the present application, it should be understood that the numerical reference number before the step does not indicate the order of execution of the steps before and after the step, but is only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as limiting the present application.

[0028] The following is the explanation of the terms involved in the present application:

[0029] VRMBlendShapeProxy: VRM model open source blend shape protocol, where VRM is a 3D file format focusing on human shape, the biggest difference from 3D formats such as FBX is that VRM unifies the coordinate system, skeleton, expression and other information of each model, making the model more convenient to use, and the interchangeability between models is better.

[0030] ARkitBlendShape: Apple's open source blend shape protocol, where ARkit is a development kit that provides AR experience in applications by integrating device camera image information and device motion sensor information.

[0031] Mediapipe: Google's open source machine learning framework.

[0032] Figure 1 The application environment of the embodiments of the present application is schematically shown, as shown in the figure:

[0033] The face capture device 200 is connected to the client 100. The face capture device 200 can be used to capture facial features of a user to obtain face capture data, and input the face capture data to the client 100. The client 100 has a preset protocol structure and a model including at least a face. The client 100 inputs the face capture data into the preset protocol structure, and obtains first hybrid morphing data including at least hybrid morphing data corresponding to the model, by using the preset protocol structure. The preset protocol structure is used to generate hybrid morphing data corresponding to at least one hybrid morphing protocol according to the face capture data. The client 100 inputs the hybrid morphing data corresponding to the model into a corresponding channel of the model, to drive the model to make a corresponding facial action or expression.

[0034] In an exemplary embodiment, the client 100 can include a mobile device, a tablet device, a laptop computer, a smart device (for example, a smart clothing, a smart watch, smart glasses), a virtual reality headset, a game device, a set-top box, a digital streaming device, a robot, a vehicle terminal, a smart television, a television box, or an electronic book reader. Alternatively, the client 100 can also be replaced by a server, where the server can be implemented by a single server or a cluster of multiple servers.

[0035] The face capture device 200 can include an input source, which can specifically include a real camera input source such as a camera, a video camera, or a laser scanner, and can also include a virtual camera input source. Alternatively, the input source of the face capture device 200 can also be a video source or a picture source, etc. In addition, the face capture device 200 can further include a data storage, transmission and reading module, so as to transmit the face capture data to the client 100.

[0036] In the related art, if a project needs to use two or more hybrid morphing protocols, the user needs to adapt two or more hybrid morphing channels to adapt to different protocols, and the adaptation cost of the user is high.

[0037] The face capture data processing scheme of the embodiment of the present application can reduce the adaptation cost of the user using different protocols.

[0038] The face capture data processing scheme will be described below through several embodiments. For the convenience of understanding, the client 100 in the following embodiment will be described as an execution subject. Figure 1

[0039] Embodiment one

[0040] Figure 2 An exemplary flowchart of a face capture data processing method of the embodiment of the present application is shown, which can include steps S310-S330, and specifically as follows:

[0041] In step S310, face capture data is obtained.​

[0042] Specifically, the client 100 can obtain the face capture data from the face capture device 200 as shown in the figure, or the face capture device 200 can send the face capture data to the client 100, so that the client 100 can obtain the face capture data. Figure 1

[0043] In step S320, the face capture data is input into a preset protocol structure to obtain first hybrid deformation data by using the preset protocol structure, wherein the first hybrid deformation data at least includes hybrid deformation data corresponding to the target model, and the preset protocol structure is used to generate hybrid deformation data corresponding to at least one hybrid deformation protocol according to the face capture data.

[0044] The preset protocol structure can generate hybrid deformation data corresponding to at least two hybrid deformation protocols. In an exemplary embodiment, the hybrid deformation protocol can include ARkitBlendShape protocol and VRMBlendShapeProxy protocol. Optionally, other hybrid deformation protocols can also be included, such as hybrid deformation protocols of Sogou Technology, etc.

[0045] It should be understood that the preset protocol structure can generate hybrid deformation data corresponding to more than two hybrid deformation protocols. Since only one model can be driven in actual application, at least one hybrid deformation protocol corresponding to the hybrid deformation data is generated according to the face capture data in step S320. When generating hybrid deformation data corresponding to one hybrid deformation protocol, hybrid deformation data corresponding to the hybrid deformation protocol of the target model can be generated. For example, if the target model is a VRM model, hybrid deformation data corresponding to the VRMBlendShapeProxy hybrid deformation protocol of the VRM model is generated.

[0046] In the preset protocol structure, a learning module and an induction module can be included, wherein the learning module is used to learn the face capture data (such as texture data output by the camera) to obtain face feature points, and then the induction module induces the face feature points to obtain hybrid deformation data (which can be hybrid deformation values, i.e. BS values) corresponding to the hybrid deformation protocol. The learning module can be a learning framework, such as a Mediapipe learning framework. After obtaining the face feature points, the induction module can use an induction algorithm to induce the face feature points to obtain hybrid deformation data corresponding to the hybrid deformation protocol. Optionally, the preset protocol structure can also include a filtering module for filtering the hybrid deformation data output by the induction module, so as to obtain more stable hybrid deformation data.

[0047] In step S330, the target model is driven according to the hybrid deformation data corresponding to the target model. ​

[0048] Specifically, the client 100 can obtain the mixed morphing data corresponding to the target model from the first mixed morphing data, input the mixed morphing data corresponding to the target model into the target model, so as to drive the target model to generate the facial action or expression corresponding to the mixed morphing data. For example, if the mixed morphing data corresponding to the target model is "close the left eye", after the mixed morphing data is input into the target model, the target model makes the facial action of "close the left eye". Wherein, when the mixed morphing data corresponding to the target model is input into the target model, the mixed morphing value corresponding to the target model can be input into the corresponding channel of the target model.

[0049] The face capture data processing method provided by the embodiment of the present application can obtain the face capture data, input the face capture data into the preset protocol structure, use the preset protocol structure to obtain the first mixed morphing data including the mixed morphing data corresponding to the target model, and then drive the target model according to the mixed morphing data corresponding to the target model. The corresponding facial action or expression of the target model can be generated by driving the target model to make the corresponding facial action or expression corresponding to the mixed morphing data of the target model. Since the preset protocol structure can generate the mixed morphing data corresponding to more than one mixed morphing protocol, the user does not need to make more than two sets of mixed morphing channels to adapt to different mixed morphing protocols, thereby reducing the adaptation cost of the user using different mixed morphing protocols.

[0050] In the exemplary embodiment, in step S320, the face capture data is input into the preset protocol structure to obtain the first mixed morphing data using the preset protocol structure. As shown in Figure 3 The step S401 to step S402 can be included, and the details are as follows:

[0051] Step S401, determine the target mixed morphing protocol corresponding to the target model.

[0052] Optionally, the target mixed morphing protocol corresponding to the target model can be input by the user of the client 100, and the client 100 determines the target mixed morphing protocol corresponding to the target model according to the input content of the user after receiving the input of the user. Optionally, the client 100 can also go to a specified location (such as a location storing target mixed morphing protocol data) to obtain the attribute value of the target mixed morphing protocol, so as to determine the target mixed morphing protocol according to the specific attribute value.

[0053] Step S402, input the face capture data into the preset protocol structure to obtain the target mixed morphing data corresponding to the target mixed morphing protocol using the preset protocol structure.

[0054] Specifically, the client 100 inputs the facial capture data into the preset protocol structure, and generates target blendshape data corresponding to the target blendshape protocol according to the facial capture data by using the preset protocol structure. For example, if the target blendshape protocol is ARkitBlendShape, the preset protocol structure can generate blendshape values corresponding to ARkitBlendShape (i.e., target blendshape data) according to the facial capture data.

[0055] In actual application, the target model can be one, and the target blendshape data is the first blendshape data. Alternatively, the target model can also be multiple, and each target model corresponds to a different blendshape protocol. Then, the client 100 can determine the blendshape protocol corresponding to each target model according to steps S401-S402, and then generate blendshape data corresponding to each blendshape protocol by using the preset protocol structure.

[0056] In the embodiment, by determining the target blendshape protocol corresponding to the target model, inputting the facial capture data into the preset protocol structure, and obtaining the target blendshape data corresponding to the target blendshape protocol by using the preset protocol structure, the corresponding blendshape data can be generated according to the blendshape protocol corresponding to the target model, and the purpose of generating the blendshape data is improved. In addition, since the target model can be different models corresponding to different blendshape protocols in actual application, the blendshape data corresponding to multiple different blendshape protocols can be generated according to needs, so that the adaptation cost of using different blendshape protocols by users is reduced.

[0057] In the exemplary embodiment, in step S320, inputting the facial capture data into the preset protocol structure to obtain the first blendshape data by using the preset protocol structure can include: inputting the facial capture data into the preset protocol structure to obtain blendshape data corresponding to at least two blendshape protocols by using the preset protocol structure. Correspondingly, in step S330, driving the target model according to the blendshape data corresponding to the target model can be that the client 100 obtains the blendshape data corresponding to the target model from the blendshape data obtained from the preset protocol structure, and then inputs the blendshape data into the target model for driving.

[0058] It can be understood that the mixed deformation data corresponding to at least two mixed deformation protocols obtained by using the preset protocol structure at least includes the mixed deformation data of the target model corresponding to the mixed deformation protocol; in addition, it also includes the mixed deformation data of more than one other mixed deformation protocol. Wherein, the obtained other mixed deformation data can be temporarily not used, if the project has a model corresponding to the mixed deformation data, the mixed deformation data is input to the newly accessed model for driving. In actual application, the preset protocol structure can be used to generate mixed deformation data corresponding to multiple mixed deformation protocols, and the user can use the corresponding data from the mixed deformation data generated by the preset protocol structure according to the specific model used. For example, if the preset protocol structure generates mixed deformation data corresponding to A, B and C three mixed deformation protocols, the target model is A model, then the A model can take the mixed deformation data corresponding to the A mixed deformation protocol from the mixed deformation data generated by the preset protocol structure to drive the model; if there is another model-B model, then the B model can take the mixed deformation data corresponding to the B mixed deformation protocol from the mixed deformation data generated by the preset protocol structure to drive the model.

[0059] In the embodiment, by inputting the face capture data into the preset protocol structure, the mixed deformation data corresponding to at least two mixed deformation protocols is obtained by using the preset protocol structure, and the mixed deformation data corresponding to multiple mixed deformation protocols can be generated, so that different models can use the corresponding mixed deformation data according to the corresponding mixed deformation protocol, and the adaptation cost of using different mixed deformation protocols is reduced.

[0060] In the exemplary embodiment, as shown in Figure 4 The face capture data processing method can further include steps S501-S502, as follows:

[0061] Step S501, obtaining second mixed deformation data.

[0062] The second mixed deformation data refers to the mixed deformation data corresponding to a certain mixed deformation protocol, which can be obtained by the preset protocol structure according to the face capture data. It can be understood that the second mixed deformation data is to distinguish from the first mixed deformation data mentioned above, which can be the data corresponding to one of the mixed deformation protocols in the first mixed deformation data.

[0063] Step S502, converting the second mixed deformation data into third mixed deformation data by using the preset protocol structure, wherein the mixed deformation protocols corresponding to the second mixed deformation data and the third mixed deformation data are different.

[0064] That is, the preset protocol structure is also used to convert the mixed deformation data corresponding to one mixed deformation protocol into the mixed deformation data corresponding to another mixed deformation protocol.

[0065] In actual application, the client 100 can obtain the second mixed morphing data according to the preset protocol structure, input the second mixed morphing data into the preset protocol structure, and convert the second mixed morphing data into third mixed morphing data by using the preset protocol structure. Alternatively, the preset protocol structure can directly obtain the second mixed morphing data and convert the second mixed morphing data into the third mixed morphing data.

[0066] The mixed morphing data corresponding to the target model can be the third mixed morphing data, that is, the preset protocol structure generates mixed morphing data corresponding to a certain mixed morphing protocol according to the facial capture data, and then converts the mixed morphing data into mixed morphing data corresponding to the target model. For example, the preset protocol structure generates mixed morphing data corresponding to the B mixed morphing protocol according to the facial capture data, and the target model is the A model, which corresponds to the A mixed morphing protocol. Then, the client 100 converts the mixed morphing data into mixed morphing data corresponding to the A mixed morphing protocol by using the preset protocol structure. Alternatively, the mixed morphing data corresponding to the target model can also be the second mixed morphing data, that is, the preset protocol structure generates mixed morphing data corresponding to the target model according to the facial capture data, and when the model is switched to another model, the preset protocol structure can be used to convert the mixed morphing data corresponding to the target model into mixed morphing data corresponding to the other model. For example, the target model is the A model, which corresponds to the A mixed morphing protocol. The preset protocol structure generates mixed morphing data corresponding to the A mixed morphing protocol according to the facial capture data. If the model needs to be switched to the model B, which corresponds to the B mixed morphing protocol, the preset protocol structure can be used to convert the mixed morphing data corresponding to the A mixed morphing protocol into mixed morphing data corresponding to the B mixed morphing protocol.

[0067] In the specific implementation of the preset protocol structure, the conversion relationship between the second mixed morphing data and the third mixed morphing data can be determined in advance. After obtaining the second mixed morphing data, the second mixed morphing data can be converted into the third mixed morphing data according to the conversion relationship. For example, the conversion relationship between the second mixed morphing data and the third mixed morphing data can be determined by determining y=f(x), where y corresponds to the third mixed morphing data, and x corresponds to the second mixed morphing data. After determining y=f(x), the second mixed morphing data can be converted into the third mixed morphing data according to y=f(x).

[0068] In this embodiment, by obtaining the second mixed morphing data and converting the second mixed morphing data into third mixed morphing data with different mixed morphing protocols by using the preset protocol structure, the conversion of mixed morphing data corresponding to different mixed morphing protocols can be performed as needed, and the adaptation cost of using different mixed morphing protocols can be reduced.

[0069] In an exemplary embodiment, the target model includes channels of several special expressions, and the channels correspond to the fourth blendshape data, as shown in Figure 5 As shown in the figure, the face capture data processing method can further include steps S601-S602, and the details are as follows:

[0070] In step S601, the face capture data is input into the preset protocol structure to obtain the fourth blendshape data by using the preset protocol structure, and the preset protocol structure is also used to generate the fourth blendshape data according to the face capture data.

[0071] That is, the client 100 inputs the face capture data into the preset protocol structure to generate the blendshape data corresponding to the special expression channel according to the face capture data by using the preset protocol structure. The special expression can include expressions such as smiling, anger, sadness, surprise, and laughter. Specifically, a value in the interval [0, 1] can be set for each special expression, for example, 0.1-0.2 is defined as a smiling expression. In a specific implementation, the weight of each part of the face can be determined according to the data of the eyebrows, eyes, and mouth in the face capture data, and then a value in the interval [0, 1] can be induced to obtain the blendshape data corresponding to the special expression.

[0072] In step S602, the fourth blendshape data is input into the special expression channel to drive the target model to generate the special expression.

[0073] In actual application, if the target model supports the expansion of the channel, a channel corresponding to the special expression can be added, and the blendshape data corresponding to the special expression can be obtained by using the preset protocol structure. Finally, the client 100 inputs the blendshape data corresponding to the special expression into the special expression channel of the target model to drive the target model to make the special expression. In addition, since the channel of the special expression is added separately, there can be a conflict with the blendshape data of the target model itself, and therefore the blendshape data of the target model itself can be set to zero to solve the conflict problem.

[0074] Since the current model only has basic expressions (such as ARkitBlendShape having only 52 basic expressions), there is no special expression, and therefore the expression form is not rich enough. In this embodiment, the face capture data is input into the preset protocol structure to obtain the fourth blendshape data corresponding to the special expression channel by using the preset protocol structure, and the fourth blendshape data is input into the special expression channel to drive the target model to generate the special expression. Therefore, a plurality of custom special expressions can be realized according to the needs, and the expressions that can be realized by the model are effectively enriched.

[0075] Please refer to Figure 6which is a principle structure example diagram of the preset protocol structure in the face capture data processing method of the embodiment of the present application. As shown in the figure, the preset protocol structure (corresponding to CompatibleFaceData in the figure) includes three layers of structures, one layer is ARkitBlendShape, one layer is VRMBlendShapeProxy, and the other layer is Effect BlendShape, wherein Effect BlendShape is used to obtain the blendshape data corresponding to special expressions. It can be understood that, Figure 6 The structure in the figure is only an example, and actually other protocols can be added as needed, such as the blendshape protocol of Sutong Technology, so that the preset protocol structure is compatible with more blendshape protocols, and the adaptation cost of users using different protocols is reduced.

[0076] Embodiment Two

[0077] Figure 7 A block diagram of a face capture data processing apparatus 700 according to Embodiment Two of the present application is schematically shown, which can be divided into one or more program modules stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program module referred to in the embodiments of the present application refers to a series of computer program instruction segments capable of completing a specific function, and the functions of the program modules in the embodiments will be specifically described below.

[0078] As Figure 7 shown, the face capture data processing apparatus 700 can include an acquisition module 710, an input module 720, and a driving module 730.

[0079] The acquisition module 710 is configured to acquire face capture data.

[0080] The input module 720 is configured to input the face capture data into a preset protocol structure to obtain first blendshape data by using the preset protocol structure, wherein the first blendshape data at least includes blendshape data corresponding to a target model, and the preset protocol structure is used to generate blendshape data corresponding to at least one blendshape protocol according to the face capture data.

[0081] The driving module 730 is configured to drive the target model according to the blendshape data corresponding to the target model.

[0082] In an exemplary embodiment, the input module 720 is further configured to: determine a target blendshape protocol corresponding to the target model; and input the face capture data into the preset protocol structure to obtain blendshape data corresponding to the target blendshape protocol by using the preset protocol structure.

[0083] In the example embodiment, the input module 720 is further configured to input the face capture data into the preset protocol structure to obtain mixed morphing data corresponding to at least two mixed morphing protocols by using the preset protocol structure.

[0084] In the example embodiment, the face capture data processing apparatus 700 can further comprise a conversion module (not shown in the figure), wherein the conversion module is configured to: obtain second mixed morphing data; and convert the second mixed morphing data into third mixed morphing data by using the preset protocol structure, wherein the mixed morphing protocols corresponding to the second mixed morphing data and the third mixed morphing data are different.

[0085] In the example embodiment, the target model comprises channels of several special expressions, and the channels correspond to the fourth mixed morphing data. The face capture data processing apparatus 700 can further comprise a special expression driving module (not shown in the figure), wherein the special expression driving module is configured to: input the face capture data into the preset protocol structure to obtain the fourth mixed morphing data by using the preset protocol structure, and the preset protocol structure is further configured to generate the fourth mixed morphing data according to the face capture data; and input the fourth mixed morphing data into the special expression channel to drive the target model to generate a special expression.

[0086] In the example embodiment, the mixed morphing protocol comprises an ARkitBlendShape protocol and a VRMBlendShapeProxy protocol.

[0087] Embodiment Three

[0088] Figure 8 A hardware architecture diagram of a computer device 800 suitable for a face capture data processing method according to Embodiment Three of the present application is schematically shown. The computer device 800 can be a device capable of automatically performing numerical calculation and / or data processing according to pre-set or stored instructions. For example, it can be a rack-mounted server, a blade server, a tower server or a cabinet server (including a standalone server or a server cluster composed of multiple servers), a gateway, etc. As shown, the computer device 800 at least includes but is not limited to a memory 810, a processor 820, a network interface 830 which can be communicatively connected through a system bus. Among them: Figure 8

[0089] ​The memory 810 includes at least one type of computer-readable storage media, such as a flash memory, a hard disk, a multimedia card (e.g., SD or DX memory, and the like), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 810 can be an internal memory module of the computer device 800, such as a hard disk or a memory of the computer device 800. In other embodiments, the memory 810 can also be an external memory device of the computer device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Of course, the memory 810 can include both an internal memory module and an external memory device of the computer device 800. In this embodiment, the memory 810 is generally used to store an operating system and various application programs installed in the computer device 800, such as program codes of the face capture data processing method, and the like. In addition, the memory 810 can also be used to temporarily store various data that have been output or will be output.

[0090] The processor 820 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 820 is generally used to control the overall operation of the computer device 800, such as performing control and processing related to data interaction or communication of the computer device 800, and the like. In this embodiment, the processor 820 is used to run program codes or process data stored in the memory 810.

[0091] The network interface 830 can include a wireless network interface or a wired network interface, which is generally used to establish a communication link between the computer device 800 and other computer devices. For example, the network interface 830 is used to connect the computer device 800 with an external terminal through a network, to establish a data transmission channel and a communication link between the computer device 800 and the external terminal, and the like. The network can be an Intranet, the Internet, a Global System for Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, and the like wireless or wired network.

[0092] It should be noted that, Figure 8 Only the computer device with the components 810-830 is shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0093] In this embodiment, the face capture data processing method stored in the memory 810 can also be divided into one or more program modules, and executed by one or more processors (in this embodiment, the processor 820) to complete the face capture data processing method in the embodiments of the present application.

[0094] Embodiment Four

[0095] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the face capture data processing method in the embodiments.

[0096] In this embodiment, the computer readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the computer readable storage medium can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the computer readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer readable storage medium is usually used to store an operating system and various application software installed on the computer device, such as program codes of the face capture data processing method in the embodiments, etc. In addition, the computer readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0097] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiments of the present application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and optionally, each module or each step can be realized by program codes executable by a computing device, so that each module or each step can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in different order, or each module or each step can be manufactured into an individual integrated circuit module, or multiple modules or steps can be manufactured into a single integrated circuit module. Therefore, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0098] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing facial capture data, characterized in that, include: Acquire facial capture data; The facial capture data is input into a preset protocol structure to obtain first hybrid deformation data using the preset protocol structure. The first hybrid deformation data includes at least hybrid deformation data corresponding to the target model. The preset protocol structure is used to generate hybrid deformation data corresponding to at least one hybrid deformation protocol based on the facial capture data. The target model is driven by the hybrid deformation data corresponding to the target model.

2. The facial capture data processing method according to claim 1, characterized in that, The step of inputting the facial capture data into a preset protocol structure to obtain first hybrid deformation data using the preset protocol structure includes: Determine the target hybrid deformation protocol corresponding to the target model; The facial capture data is input into a preset protocol structure to obtain target hybrid deformation data corresponding to the target hybrid deformation protocol using the preset protocol structure.

3. The facial capture data processing method according to claim 1, characterized in that, The step of inputting the facial capture data into a preset protocol structure to obtain first hybrid deformation data using the preset protocol structure includes: Facial capture data is input into a preset protocol structure to obtain hybrid deformation data corresponding to at least two hybrid deformation protocols.

4. The facial capture data processing method according to any one of claims 1-3, characterized in that, Also includes: Obtain the second hybrid deformation data; The second hybrid deformation data is converted into third hybrid deformation data using the preset protocol structure, wherein the hybrid deformation protocols corresponding to the second hybrid deformation data and the third hybrid deformation data are different.

5. The facial capture data processing method according to any one of claims 1-3, characterized in that, The target model includes channels for several special facial expressions, each channel corresponding to a fourth hybrid deformation data. The method further includes: The facial capture data is input into the preset protocol structure to obtain the fourth hybrid deformation data using the preset protocol structure. The preset protocol structure is also used to generate the fourth hybrid deformation data based on the facial capture data. The fourth hybrid deformation data is input into the special expression channel to drive the target model to generate the special expression.

6. The facial capture data processing method according to any one of claims 1-3, characterized in that, The hybrid deformation protocols include the ARkitBlendShape protocol and the VRMBlendShapeProxy protocol.

7. A facial capture data processing device, characterized in that, include: The acquisition module is used to acquire facial capture data; An input module is used to input the facial capture data into a preset protocol structure to obtain first hybrid deformation data using the preset protocol structure. The first hybrid deformation data includes at least hybrid deformation data corresponding to the target model. The preset protocol structure is used to generate hybrid deformation data corresponding to at least one hybrid deformation protocol based on the facial capture data. The driving module is used to drive the target model according to the hybrid deformation data corresponding to the target model.

8. The facial capture data processing device according to claim 7, characterized in that, The input module is also used for: Determine the target hybrid deformation protocol corresponding to the target model; The facial capture data is input into a preset protocol structure to obtain hybrid deformation data corresponding to the target hybrid deformation protocol using the preset protocol structure.

9. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it is used to implement the steps of the facial capture data processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the facial capture data processing method according to any one of claims 1 to 6.

Images