Data processing method and apparatus therefor

Abstract

The application discloses a data processing method and device, and belongs to the technical field of three-dimensional technology. The data processing method comprises the following steps: determining second point cloud data according to a target three-dimensional face model and first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of a plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; superimposing different expressions on the second point cloud data respectively by adjusting first expression parameters to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; and performing data fusion on the first point cloud data and the third point cloud data to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of three dimensions, and particularly relates to a data processing method and device thereof. BACKGROUND

[0002] In recent years, with the development of face recognition technology, three dimensions (3D) animation, augmented reality (AR) technology, and virtual reality (VR) technology, three-dimensional face modeling has attracted widespread attention, and 3D face reconstruction technology based on deep learning is applied in more and more scenes. High-quality 3D face reconstruction deep learning algorithm depends on a large amount of training data, so in order to obtain sufficient training data, various algorithms and technologies need to be explored to collect 3D real data by using high-precision equipment.

[0003] In the related art, the scheme of collecting 3D real data based on high-precision equipment has a long data collection period, low data collection efficiency, and great limitations. SUMMARY

[0004] Embodiments of the present application provide a data processing method and device, which can improve the problem of low data efficiency and great limitations of the scheme of collecting 3D real data in the related art.

[0005] In a first aspect, the embodiments of the present application provide a data processing method, which comprises: determining second point cloud data according to a target three-dimensional face model and first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of a plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; performing superposition processing of different expressions on the second point cloud data respectively by adjusting first expression parameters to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; and performing data fusion on the first point cloud data and the third point cloud data to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states.

[0006] In a second aspect, an embodiment of the present application provides a data processing apparatus, comprising: a determination module configured to determine second point cloud data according to a target three-dimensional face model and first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of a plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; an expression superimposition module configured to perform superimposition processing of different expressions on the second point cloud data respectively by adjusting first expression parameters to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; and a data fusion module configured to perform data fusion on the first point cloud data and the third point cloud data to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in the different expression states.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored on the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the data processing method of the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, which stores a program or instruction, and the program or instruction is executed by a processor to implement the steps of the data processing method of the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, which comprises a processor and a communication interface, the communication interface and the processor are coupled, and the processor is configured to run a program or instruction to implement the steps of the data processing method of the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps of the data processing method of the first aspect.

[0011] In the embodiment of the present application, the first point cloud data is three-dimensional point cloud data of the head of the plurality of target objects in an expressionless state, and the corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state, i.e., the second point cloud data, is determined according to the target three-dimensional face model and the first point cloud data. Thus, the expressionless 3D point cloud face data of the plurality of target objects can be obtained, and the second point cloud data is subjected to superposition processing of different expressions by adjusting the first expression parameter, so as to obtain the corresponding face three-dimensional point cloud data of the plurality of target objects in different expressions, i.e., the third point cloud data. Since the third point cloud data fuses different expressions, the data fusion of the third point cloud data with the original expressionless first point cloud data can quickly obtain the corresponding head three-dimensional point cloud data (fourth point cloud data) of the plurality of target objects in different expressions, i.e., quickly generate a large amount of real 3D head data with expressions, the data generation period is shorter, and the data acquisition efficiency of the 3D real data can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 FIG. 1 is a flow diagram of a data processing method according to an embodiment of the present application;

[0013] Figure 2 FIG. 2 is a flow diagram of a data processing method according to another embodiment of the present application;

[0014] Figure 3 FIG. 3 is a flow diagram of a data processing method according to still another embodiment of the present application;

[0015] Figure 4 FIG. 4 is a flow diagram of a data processing method according to still another embodiment of the present application;

[0016] Figure 5 FIG. 5 is a structural diagram of a data processing device according to an embodiment of the present application;

[0017] Figure 6 FIG. 6 is a structural diagram of an electronic device according to an embodiment of the present application;

[0018] Figure 7 FIG. 7 is a hardware structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0020] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in an "or" relationship.

[0021] As in the background, the scheme based on high-precision equipment to collect 3D real data has a long data collection period, low data collection efficiency, and great limitations.

[0022] To solve the problems in the related art, the embodiments of the present application provide a data processing method. The first point cloud data is three-dimensional point cloud data of the head of a plurality of target objects in an expressionless state. According to a target three-dimensional face model and the first point cloud data, corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state, i.e., second point cloud data, is determined. Thus, expressionless 3D point cloud face data of the plurality of target objects can be obtained. By adjusting first expression parameters, the second point cloud data is subjected to different expression superposition processing, respectively, to obtain corresponding face three-dimensional point cloud data of the plurality of target objects in different expressions, i.e., third point cloud data. Since different expressions are fused in the third point cloud data, data fusion of the third point cloud data with expressions and the original expressionless first point cloud data can quickly obtain corresponding head three-dimensional point cloud data (fourth point cloud data) of the plurality of target objects in different expressions, i.e., quickly generate a large amount of real 3D head data with expressions. The data generation period is short, the data acquisition efficiency of 3D real data can be effectively improved, and the problem of low data efficiency and great limitations of the scheme for collecting 3D real data in the related art is solved.

[0023] The data processing method provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios.

[0024] Figure 1 is a flowchart of the data processing method provided by an embodiment of the present application. The execution subject of the data processing method can be an electronic device. It should be noted that the above execution subject does not constitute a limitation on the present application.

[0025] As Figure 1 indicated, the data processing method provided by the embodiments of the present application can include steps 110-130.

[0026] In step 110, second point cloud data is determined according to the target three-dimensional face model and the first point cloud data.

[0027] The first point cloud data is head three-dimensional point cloud data of the plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state.

[0028] In step 120, the second point cloud data is subjected to superposition processing of different expressions respectively by adjusting a first expression parameter, to obtain third point cloud data.

[0029] The third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states.

[0030] In step 130, data fusion is performed on the first point cloud data and the third point cloud data, to obtain fourth point cloud data.

[0031] The fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states.

[0032] The data processing method provided in the embodiments of the present application is as follows: the first point cloud data is head three-dimensional point cloud data of the plurality of target objects in an expressionless state, and second point cloud data, which is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state, is determined according to the target three-dimensional face model and the first point cloud data. Thus, expressionless 3D point cloud face data of the plurality of target objects can be obtained. The second point cloud data is subjected to superposition processing of different expressions respectively by adjusting a first expression parameter, to obtain third point cloud data, which is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states. Since different expressions are fused in the third point cloud data, data fusion is performed on the third point cloud data with expressions and the original first point cloud data without expressions, to quickly obtain fourth point cloud data, which is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states, that is, to quickly generate a large amount of real 3D head data with expressions, so that the data generation period is short, and the data acquisition efficiency of 3D real data is effectively improved.

[0033] The steps 110-130 are described in detail below in combination with specific embodiments.

[0034] The step 110 is related to determining second point cloud data according to the target three-dimensional face model and the first point cloud data.

[0035] In some embodiments of the present application, in order to obtain the first point cloud data, before the step 110, the method can further include: obtaining head three-dimensional point cloud data of the plurality of target objects in an expressionless state, to obtain the first point cloud data.

[0036] Specifically, the plurality of target objects can include acquisition personnel of different genders and different age groups, and the electronic device can acquire three-dimensional point cloud data of heads of all the acquisition personnel in an expressionless state by using a 3D head scanning system, to obtain first point cloud data.

[0037] In one embodiment, the acquisition process can specifically include: fixing the camera position of the 3D head scanning system, and the acquisition personnel sitting in a fixed position in front of the camera position (the position is determined to ensure that the heads of all the acquisition personnel are in the central position of the camera imaging).

[0038] Specifically, during data acquisition, the acquisition personnel fix their bodies, and then slowly rotate their heads in turn up, down, left and right, and the angle of rotation is as large as possible.

[0039] In one embodiment, during the 3D point cloud data acquisition process, in order to eliminate the influence of facial expression differences between individuals on the shape of the face, all the acquisition personnel are required to keep their eyes normally open and their mouths slightly closed during data acquisition, and to use a hair band to tie up their hair to ensure that their ears are exposed, and their faces do not have any other expressions, that is, the acquisition personnel are in an expressionless state.

[0040] In some embodiments of the present application, in order to obtain the target three-dimensional face model, before step 110, the method can further include: constructing the target three-dimensional face model based on a three-dimensional deformable face model 3DMM.

[0041] The target three-dimensional face model can be a deformable face model (DFM) composed of face three-dimensional point coordinate information and patch mesh information; the three-dimensional deformable face model (3D Morphable Models, 3DMM) is a general three-dimensional face model that uses a fixed number of points to represent a face, and its core idea is that a face can be one-to-one matched in three-dimensional space and can be linearly added by many face orthogonal bases.

[0042] Specifically, the electronic device can use professional 3D modeling software to standardize the modeling of biologically defined face structures in the international system of units, to obtain a target three-dimensional face model that meets the 3DMM standard.

[0043] In some embodiments of the present application, constructing the target three-dimensional face model based on the three-dimensional deformable face model 3DMM can specifically include: densely modeling a first preset face area and sparsely modeling a second preset face area based on the 3DMM to obtain the target three-dimensional face model, the first preset face area being different from the second preset face area.

[0044] The first preset face region is a key region, and the second preset face region is a non-key region. The first preset face region and the second preset face region can be set according to specific requirements, and the application does not make specific limitations.

[0045] For example, the key regions such as eyes, mouth, nose, etc. are taken as the first preset regions for dense modeling, and the non-key regions such as ears, eyebrows, etc. are taken as the second preset regions for appropriate sparse modeling.

[0046] In the embodiments of the application, considering that the business application requirements focus on different face organ regions, when the target three-dimensional face model is constructed based on 3DMM, different regions of the face can be distinguished by keyness. By densely modeling the first preset face region which is relatively key in the face region and densely modeling the second preset face region which is not key, the obtained target three-dimensional face model can allocate more point cloud quantities to the key regions of the face, and the point cloud density is relatively large. Correspondingly, for the non-key regions of the face, the point cloud density is relatively small. Therefore, the target three-dimensional face model can more comprehensively and in detail reflect the details of the key regions of the face, which meets the actual needs of users.

[0047] In one embodiment, step 110 can specifically include: obtaining face region point cloud data in the first point cloud data based on a nearest neighbor algorithm; and mapping the face region point cloud data to a topological space corresponding to the target three-dimensional face model based on a non-rigid registration algorithm to obtain second point cloud data.

[0048] The second point cloud data is 3D real face data consistent with the topology and the DFM, and the second point cloud data can correspond to a deformable face model based on three-dimensional scanning.

[0049] In the embodiments of the application, since the face region point cloud data is only a part of the head three-dimensional point cloud data (i.e. the first point cloud data), the point cloud quantities corresponding to the face region point cloud data and the first point cloud data are different. Based on this, mapping the face region point cloud data to the topological space corresponding to the target three-dimensional face model based on the non-rigid registration algorithm can ensure that the obtained second point cloud data is uniform in topology with the target three-dimensional face model.

[0050] In one embodiment, to improve the registration accuracy, before mapping the face region point cloud data to the corresponding topological space of the target three-dimensional face model based on the non-rigid registration algorithm, the method can further include: obtaining first texture map data of the head of the plurality of target objects under the condition that the plurality of target objects are in an expressionless state; constructing a mesh structure based on the first point cloud data to obtain first mesh data; performing texture map fusion and UV expansion processing on the first texture map data to obtain a UV texture map and a first mapping relationship between the UV texture map and the first texture map data; detecting the 2D face key point positions in the UV texture map using a face key point detection algorithm; determining the 3D face key point positions in the first mapping relationship using a nearest neighbor algorithm; and taking the Euclidean distance between the 2D face key point positions and the 3D face key point positions as a loss function constraint parameter of the non-rigid registration algorithm.

[0051] In the embodiments of the present application, by adding a face key point registration loss constraint in the non-rigid registration algorithm, the registration accuracy of the non-rigid registration algorithm can be improved.

[0052] Step 120 involves adjusting the first expression parameter to perform different expression superposition processing on the second point cloud data respectively to obtain third point cloud data.

[0053] In some embodiments of the present application, Figure 2 is a flowchart of a data processing method provided by another embodiment of the present application, and step 120 can include Figure 2 steps 210-230 shown in the figure.

[0054] Step 210 adjusts the first expression parameter.

[0055] Specifically, the first expression parameter can include at least one sub-expression parameter, and different sub-expression parameters are used to superimpose different types of second expression parameters; adjusting the first expression parameter can be adjusting the value of the first expression parameter, or adjusting the value of each sub-expression parameter in the first expression parameter.

[0056] For example, the first expression parameter a can include sub-expression parameters a1 and a2, a1 can be used to superimpose a "smile" second expression parameter, the value of a1 can be 0 to 1, and the closer the value of a1 is to 1, the greater the amplitude of the "smile" expression; a2 can be used to superimpose a "cry" second expression parameter, the value of a1 can be 0 to 1, and the closer the value of a2 is to 1, the greater the amplitude of the "cry" expression.

[0057] Step 220 determines the second expression parameter based on the expression basis vector and the adjusted first expression parameter.

[0058] The first expression parameter can be a coefficient of an expression basis vector, and the second expression parameter can be a product of the expression basis vector and a preset expression coefficient.

[0059] Specifically, by adjusting the value of the first expression parameter or the value of each sub-expression parameter in the first expression parameter, the product of the first expression parameter and the expression basis vector (i.e., the second expression parameter) can represent different expression types.

[0060] In step 230, the second expression parameter is linearly added to the second point cloud data to obtain third point cloud data.

[0061] In one embodiment, the third point cloud data can be determined based on formula (1).

[0062] DFM_WE = DFM_SCAN + a * DFM_EB (1)

[0063] wherein a is the first expression parameter, DFM_SCAN is the second point cloud data, DFM_EB is an expression basis vector (Deformable Face Model Expression Base), and DFM_WE is the third point cloud data.

[0064] In the embodiments of the present application, when the second expression parameter is determined based on the expression basis vector and the first expression parameter, the second expression parameter obtained by adjusting the first expression parameter can represent different expression types. Therefore, based on the second expression parameter, the second point cloud data can be subjected to different expression superposition processing, and the expression is fused based on the non-expression face three-dimensional point cloud data, so that the third point cloud data obtained can present multiple expressions, thereby generating a large amount of real 3D face data with expressions, and the data acquisition method is simple and effective, and has low cost and small limitation compared with using high-precision equipment to collect 3D real data.

[0065] In some embodiments of the present application, in order to obtain the expression basis vector, Figure 3 is a flowchart of a data processing method provided by another embodiment of the present application, and before step 220, the method can further include steps 310-330.

[0066] In step 310, a principal component analysis (PCA) algorithm is used to reduce the dimensionality of the second point cloud data to obtain a first model.

[0067] The first model can be a DFM mean face model (Deformable Face Model Mean Shape, DFM_MS).

[0068] In one embodiment, the second point cloud data is dimensionally reduced by using a principal component analysis algorithm, and a DFM shape base vector (Deformable Face Model Shape Base, DFM_SB) can also be obtained.

[0069] In step 320, different expression deformations of the first model are modeled to obtain a second model.

[0070] Specifically, the electronic device can use a 3D modeling software to model different expression deformations of the DFM_MS model to obtain a second model, which can be a DFM_MS Add Expression model (Deformable Face Model Mean Shape Add Expression, DFM_MSAE).

[0071] In step 330, the first model and the second model are linearly subtracted to obtain an expression base vector.

[0072] In the embodiments of the present application, after dimensionally reducing the expressionless face three-dimensional point cloud data by using a principal component analysis algorithm, the required DFM mean face model can be obtained. By modeling different expression deformations of the DFM mean face model, a DFM_MS Add Expression model is obtained. Based on this, by linearly subtracting the DFM mean face model from the DFM_MS Add Expression model, an expression base vector is finally obtained. Thus, the second point cloud data is processed by superimposing different expressions based on the expression base vector, and expressions are fused based on the expressionless face three-dimensional point cloud data, so that a large amount of real 3D face data with expressions is quickly generated, the data acquisition period of 3D real data is shortened, and the data acquisition cost is effectively reduced compared to the scheme of acquiring 3D real data based on high-precision equipment.

[0073] Involving step 130, the first point cloud data and the third point cloud data are data fused to obtain fourth point cloud data.

[0074] In some embodiments of the present application, in order to obtain multi-expression head 3D texture map data, Figure 4 is a flowchart of a data processing method provided by another embodiment of the present application, as shown in the figure, the method can further include steps 410-460. Figure 4

[0075] In step 410, first texture map data of the heads of a plurality of target objects is obtained under the condition that the plurality of target objects are in an expressionless state.

[0076] The first texture map data can be texture map data collected when scanning 3D point cloud data of the head under different head rotation postures.

[0077] ​At step 420, a mesh structure is constructed based on the first point cloud data, to obtain first mesh data.

[0078] Specifically, the electronic device can use a mesh construction tool in the 3D head scanning system to triangulate the first point cloud data, connect every two of the three 3D point cloud vertices to form a head patch, and thus construct the entire 3D head mesh structure, to obtain the first mesh data.

[0079] At step 430, the first texture mapping data is subjected to texture mapping fusion and UV unwrapping processing, to obtain a UV texture map and a first mapping relationship between the UV texture map and the first texture mapping data.

[0080] Specifically, UV refers to accurately mapping each point on an image to the surface of a model object, and the gaps between the points are subjected to image smoothing interpolation processing by software. In order to reasonably distribute the UV texture of the model on the canvas, the three-dimensional surface is reasonably tiled on the two-dimensional canvas, which is referred to as UV unwrapping processing. The first mapping relationship can be a one-to-one texture mapping table between the UV texture map and the first texture mapping data.

[0081] At step 440, a mesh structure is constructed based on the fourth point cloud data, to obtain second mesh data.

[0082] Specifically, the electronic device can use a mesh construction tool in the 3D head scanning system to triangulate the fourth point cloud data, connect every two of the three 3D point cloud vertices to form a head patch, and thus construct the entire 3D head mesh structure, to obtain the second mesh data.

[0083] It should be noted that step 440 is performed after the fourth point cloud data is generated at step 150.

[0084] At step 450, the first mapping relationship is traversed based on the second mesh data, to obtain second mapping relationship.

[0085] Specifically, the first mapping relationship is traversed based on the information saved by the 3D point cloud fusion, i.e., the second mesh data, to generate a new UV texture mapping table, i.e., the second mapping relationship.

[0086] At step 460, target three-dimensional texture mapping data is generated based on the second mapping relationship and the UV texture map.

[0087] Specifically, based on the second mapping relationship and the real-scanned UV texture map generated at step 430, the target three-dimensional texture mapping data is constructed and generated, which is the multi-expression head 3D texture mapping data.

[0088] In this embodiment, when multiple target objects are in a blank state, in addition to acquiring the 3D point cloud data of the heads of the multiple target objects, the first texture map data of the heads of the multiple target objects can also be acquired. A mesh structure is constructed based on the first point cloud data to obtain the first mesh data. Texture map fusion and UV unwrapping are then performed on the first texture map data to obtain UV texture maps, and a first mapping relationship between the UV texture maps and the first texture map data. After obtaining the realistic 3D head data with expressions, i.e., the fourth point cloud data, the mesh structure can be reconstructed based on the fourth point cloud data to achieve mesh regeneration and obtain the second mesh data. The first texture map data in the first mapping relationship is blank. Therefore, after obtaining the fourth point cloud data with multiple expressions, the first mapping relationship is traversed based on the second mesh data corresponding to the fourth point cloud data to obtain the second mapping relationship. Based on the second mapping relationship and the UV texture map, target 3D texture map data for generating a head with multiple expressions is constructed. This target 3D texture map data can be used to express the display details of facial expressions, thus facilitating 3D face reconstruction.

[0089] It should be noted that the data processing method provided in this application embodiment can be executed by a data processing device or a control module within that data processing device for executing the data processing method. This application embodiment uses the execution of the data processing method by a data processing device as an example to illustrate the data processing device provided in this application embodiment. The data processing device will now be described in detail.

[0090] Figure 5 This is a schematic diagram of the structure of a data processing device provided in this application.

[0091] like Figure 5 As shown in the figure, this application embodiment provides a data processing device 500, which includes: a determination module 510, an expression overlay module 520, and a data fusion module 530.

[0092] The system includes a determination module 510, which determines second point cloud data based on the target 3D face model and the first point cloud data. The first point cloud data consists of 3D head point cloud data of multiple target objects in an expressionless state, and the second point cloud data consists of 3D face point cloud data of multiple target objects in an expressionless state. The expression overlay module 520 is used to overlay different expressions onto the second point cloud data by adjusting the first expression parameters to obtain third point cloud data. The third point cloud data consists of 3D face point cloud data of multiple target objects in different expression states. The data fusion module 530 is used to fuse the first point cloud data and the third point cloud data to obtain fourth point cloud data. The fourth point cloud data consists of 3D head point cloud data of multiple target objects in different expression states.

[0093] In some embodiments of the present application, the determining module 510 comprises: an obtaining unit, configured to obtain the face region point cloud data in the first point cloud data based on a nearest neighbor algorithm; and a mapping unit, configured to map the face region point cloud data to a topological space corresponding to the target three-dimensional face model based on a non-rigid registration algorithm to obtain the second point cloud data.

[0094] In some embodiments of the present application, the expression superimposition module 520 comprises: an adjusting unit, configured to adjust the first expression parameter; a determining unit, configured to determine the second expression parameter based on the expression basis vector and the adjusted first expression parameter; and a linear processing unit, configured to linearly add the second expression parameter and the second point cloud data to obtain the third point cloud data.

[0095] In some embodiments of the present application, the apparatus further comprises: a dimension reduction module, configured to reduce the dimension of the second point cloud data by using a principal component analysis algorithm to obtain the first model; a model construction module, configured to model the first model for different expression deformations to obtain the second model; and the linear processing unit, further configured to linearly subtract the first model and the second model to obtain the expression basis vector.

[0096] In some embodiments of the present application, the apparatus further comprises: an obtaining module, configured to obtain first texture map data of the head of the plurality of target objects in a no-expression state; a mesh construction module, configured to construct a mesh structure based on the first point cloud data to obtain first mesh data; a texture map processing module, configured to perform texture map fusion and UV expansion processing on the first texture map data to obtain UV texture map and a first mapping relationship between the UV texture map and the first texture map data; the mesh construction module, further configured to construct a mesh structure based on the fourth point cloud data to obtain second mesh data; a traversal module, configured to traverse the first mapping relationship based on the second mesh data to obtain a second mapping relationship; and a generating module, configured to generate target three-dimensional texture map data based on the second mapping relationship and the UV texture map.

[0097] The data processing apparatus provided in the embodiments of the present application can obtain the 3D point cloud face data of the multiple target objects in the expressionless state, perform superposition processing of different expressions on the second point cloud data respectively by adjusting the first expression parameter, and obtain the corresponding face 3D point cloud data of the multiple target objects in different expressions, i.e., third point cloud data. Since the third point cloud data fuses different expressions, the data fusion of the third point cloud data with the original expressionless first point cloud data can quickly obtain the corresponding head 3D point cloud data of the multiple target objects in different expressions (fourth point cloud data), i.e., quickly generate a large amount of real 3D head data with expressions, the data generation period is relatively short, and the data acquisition efficiency of the 3D real data can be effectively improved.

[0098] The data processing apparatus provided in the embodiments of the present application can realize Figures 1-4 The processes realized by the electronic device in the method embodiments are not repeated here to avoid repetition.

[0099] The data processing apparatus in the embodiments of the present application can be an electronic device, or a component, an integrated circuit, or a chip in the electronic device. The electronic device can be a terminal or other devices except the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the like. The electronic device can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like. The embodiments of the present application are not limited in this regard.

[0100] The data processing apparatus in the embodiments of the present application can be an apparatus with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0101] Optionally, as shown in Figure 6 The electronic device 600 provided by the embodiment of the present application includes a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, the processes of the above-mentioned data processing method embodiments are implemented, and the same technical effects are achieved. To avoid repetition, details are not described here.

[0102] It should be noted that the electronic device in the embodiment of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0103] Figure 7 The hardware structure of the electronic device in the embodiment of the present application is shown in the figure.

[0104] The electronic device 700 includes, but is not limited to, a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710, etc.

[0105] Those skilled in the art can understand that the electronic device 700 can also include a power supply (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 710 through a power management system, so as to realize management of charging, discharging, and power consumption management, etc. through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0106] The processor 710 is configured to: determine second point cloud data according to the target three-dimensional face model and the first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of the plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; perform superposition processing of different expressions on the second point cloud data respectively by adjusting first expression parameters to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; and perform data fusion on the first point cloud data and the third point cloud data to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states.

[0107] In the embodiments of the present application, the first point cloud data is three-dimensional point cloud data of the head of the plurality of target objects in an expressionless state, and the corresponding facial three-dimensional point cloud data of the plurality of target objects in the expressionless state, i.e., the second point cloud data, is determined according to the target three-dimensional face model and the first point cloud data. Thus, the expressionless 3D point cloud face data of the plurality of target objects can be obtained, and the second point cloud data is subjected to superposition processing of different expressions by adjusting the first expression parameter, so as to obtain the corresponding facial three-dimensional point cloud data of the plurality of target objects in different expressions, i.e., the third point cloud data. Since the third point cloud data fuses different expressions, the data fusion of the third point cloud data with the original expressionless first point cloud data can quickly obtain the corresponding head three-dimensional point cloud data of the plurality of target objects in different expressions (fourth point cloud data), i.e., quickly generate a large number of real 3D head data with expressions, the data generation period is shorter, and the data acquisition efficiency of the 3D real data can be effectively improved.

[0108] It should be understood that in the embodiments of the present application, the input unit 704 can include a graphics processing unit (GPU) 7041 and a microphone 7042. The graphics processing unit 7041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 can include a display panel 7061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 can include two parts of a touch detection device and a touch controller. The other input devices 7072 can include, but are not limited to, a physical keyboard, a key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like, which will not be described here.

[0109] The memory 709 can be used to store software programs and various data. The memory 709 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, N required application programs or instructions (such as sound playing, image playing, etc.), and the like. In addition, the memory 709 can include a volatile memory or a non-volatile memory, or the memory 709 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synchlink DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 709 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0110] The processor 710 can include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 710.

[0111] The embodiments of the present application also provide a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned data processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0112] The processor is a processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, examples of the computer readable storage medium include non-transitory computer readable storage medium, such as computer readable only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0113] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the above data processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0114] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0115] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and the program product is executed by at least one processor to realize the processes of the above data processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0116] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order shown or discussed, and can also include the execution according to the order involved in the basic simultaneous manner or in the opposite order, for example, the described method can be executed in different order from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disc, an optical disc), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0118] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A data processing method, characterized by, The method comprises the following steps: determining second point cloud data according to a target three-dimensional face model and first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of a plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; performing superposition processing of different expressions on the second point cloud data respectively by adjusting first expression parameters to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; performing data fusion on the first point cloud data and the third point cloud data to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states; the method further comprises the following steps: obtaining face region point cloud data in the first point cloud data based on a nearest neighbor algorithm; mapping the face region point cloud data to a topological space corresponding to the target three-dimensional face model based on a non-rigid registration algorithm to obtain the second point cloud data; before the step of mapping the face region point cloud data to the topological space corresponding to the target three-dimensional face model based on the non-rigid registration algorithm to obtain the second point cloud data, the method further comprises the following steps: obtaining first texture map data of the heads of the plurality of target objects in the expressionless state; constructing a mesh structure based on the first point cloud data to obtain first mesh data; performing texture map fusion and UV expansion processing on the first texture map data to obtain UV texture maps and a first mapping relationship between the UV texture maps and the first texture map data; detecting two-dimensional face key point positions in the UV texture maps by using a face key point detection algorithm; determining three-dimensional face key point positions in the first mapping relationship by using a nearest neighbor algorithm; taking the Euclidean distances between the two-dimensional face key point positions and the three-dimensional face key point positions as loss function constraint parameters of the non-rigid registration algorithm.

2. The method of claim 1, wherein, the method further comprises the following steps: adjusting the first expression parameters; determining second expression parameters based on expression basis vectors and the adjusted first expression parameters; performing linear addition on the second expression parameters and the second point cloud data to obtain the third point cloud data.

3. The method of claim 2, wherein, before the step of determining the second expression parameters based on the expression basis vectors and the adjusted first expression parameters, the method further comprises the following steps: performing dimension reduction on the second point cloud data by using a principal component analysis algorithm to obtain a first model; performing different expression deformation modeling on the first model to obtain a second model; performing linear subtraction on the first model and the second model to obtain the expression basis vectors.

4. The method of claim 1, wherein, the method further comprises the following steps: constructing a mesh structure based on the fourth point cloud data to obtain second mesh data; Traversal is performed on the first mapping relationship based on the second mesh data, and a second mapping relationship is obtained. A target three-dimensional texture map data is generated based on the second mapping relationship and the UV texture map.

5. A data processing apparatus, characterized by, Comprise: The determination module is configured to determine second point cloud data according to a target three-dimensional face model and first point cloud data, wherein the first point cloud data is head three-dimensional point cloud data of a plurality of target objects in an expressionless state, and the second point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in the expressionless state; The expression superposition module is configured to perform superposition processing of different expressions on the second point cloud data respectively by adjusting first expression parameters, to obtain third point cloud data, wherein the third point cloud data is corresponding face three-dimensional point cloud data of the plurality of target objects in different expression states; The data fusion module is configured to perform data fusion on the first point cloud data and the third point cloud data, to obtain fourth point cloud data, wherein the fourth point cloud data is corresponding head three-dimensional point cloud data of the plurality of target objects in different expression states; The determination module comprises: The acquisition unit is configured to acquire face region point cloud data in the first point cloud data based on a nearest neighbor algorithm; The mapping unit is configured to map the face region point cloud data to a topological space corresponding to the target three-dimensional face model based on a non-rigid registration algorithm, to obtain the second point cloud data; The device further comprises: The acquisition module is configured to acquire first texture map data of heads of the plurality of target objects in an expressionless state; The mesh construction module is configured to construct a mesh structure based on the first point cloud data, to obtain first mesh data; The texture map processing module is configured to perform texture map fusion and UV expansion processing on the first texture map data, to obtain a UV texture map and a first mapping relationship between the UV texture map and the first texture map data; The detection module is configured to detect two-dimensional face key point positions in the UV texture map by using a face key point detection algorithm; The determination module is further configured to determine three-dimensional face key point positions in the first mapping relationship by using a nearest neighbor algorithm, and use Euclidean distances between the two-dimensional face key point positions and the three-dimensional face key point positions as loss function constraint parameters of the non-rigid registration algorithm.

6. The apparatus of claim 5, wherein, The expression superposition module comprises: The adjustment unit is configured to adjust the first expression parameters; The determination unit is configured to determine second expression parameters based on an expression basis vector and the adjusted first expression parameters; The linear processing unit is configured to perform linear addition on the second expression parameters and the second point cloud data, to obtain the third point cloud data.

7. The apparatus of claim 6, wherein, The device further comprises: The dimension reduction module is configured to perform dimension reduction on the second point cloud data by using a principal component analysis algorithm, to obtain a first model; The model construction module is configured to model different expression deformations on the first model, to obtain a second model; The linear processing unit is further configured to perform linear subtraction on the first model and the second model, to obtain the expression basis vector.

8. The apparatus of claim 5, wherein, The mesh construction module is further configured to construct a mesh structure based on the fourth point cloud data to obtain second mesh data. The device further includes: A traversal module configured to traverse the first mapping relationship based on the second mesh data to obtain a second mapping relationship. A generation module configured to generate target three-dimensional texture map data based on the second mapping relationship and the UV texture map.

Images