Face data optimization method and device

By collecting and optimizing facial point data in real time, the problems of facial animation beating and stiffness in virtual images are solved, and the continuity and accuracy of facial expression capture are achieved, ensuring the smooth and natural animation.

CN118918225APending Publication Date: 2024-11-08HUNAN HAPPLY SUNSHINE INTERACTIVE ENTERTAINMENT MEDIA CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410748999.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-11
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

During the expression animation, virtual image facial animation is prone to beating and too stiff. The reason is that the facial features of the facial features change in small amplitude and fast frequency, which makes the camera unable to continuously match the actor's expression data in real time, and there are facial points. Acquisition error.

Method used

By collecting facial point data in real time, including eye and mouth point data, if the facial optimization conditions are met, the error point is determined and its coordinate data is optimized, and corrected to the preset trajectory interval range to eliminate errors.

Benefits of technology

It eliminates the jumping and stiffness of facial animation during the expression animation display process, ensures that the facial animation of virtual images is smooth and natural, and improves the accuracy and continuity of facial expression capture.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918225A_ABST
    Figure CN118918225A_ABST
Patent Text Reader

Abstract

The invention discloses a face data optimization method and device, and the method comprises the steps: collecting the face point location data of a to-be-collected person in real time, enabling the face point location data to at least comprise eye point location data and mouth point location data, and if the eye point location data and / or the mouth point location data meet the face optimization condition, carrying out the face optimization of the to-be-collected person. The face optimization condition is determined by a standard numerical value and a fault-tolerant interval, error point locations in the facial expression capturing process are determined, coordinate data of the error point locations are obtained, facial point location optimization is carried out on the coordinate data of the error point locations, and the facial point location optimization is used for correcting the coordinate data of the error point locations to a preset track interval range.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and more specifically, to a method and device for optimizing face data. Background Art

[0002] With the development of facial capture technology, facial capture technology is used to capture the actor's facial data in real time to drive the virtual image in the rendering engine to display facial expression animation.

[0003] In the process of displaying facial animation, the facial animation of the captured virtual image may show jumpy movements and overly stiff expressions. The main reason is that the changes in the facial features are small in amplitude and fast in frequency, and the coordinate displacement of the expression is relatively small. This causes the sampling camera to be unable to continuously and real-time match the actor's expression data while capturing the actor's facial expressions, resulting in errors in facial point collection.

[0004] Therefore, how to avoid the situation in which the facial animation of the captured virtual image has jerking movements and overly stiff expressions during the display of facial expression animation is a problem that needs to be urgently solved in this application. Summary of the invention

[0005] In view of this, the present application discloses a method and device for optimizing facial data, aiming to eliminate errors in facial point acquisition, thereby avoiding the situation in which the facial animation of the captured virtual image has jerking movements and overly stiff expressions during the display of facial animation.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] In a first aspect, the present application discloses a method for optimizing face data, the method comprising:

[0008] Collecting facial point data of the person to be collected in real time; the facial point data at least includes eye point data and mouth point data;

[0009] If the eye point data and / or the mouth point data meet the facial optimization condition, determine the error point in the facial expression capture process; wherein the facial optimization condition is determined by a standard value and a tolerance interval;

[0010] The coordinate data of the error point is obtained, and the coordinate data of the error point is optimized; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range.

[0011] Preferably, the real-time collection of facial point data of the person to be collected includes:

[0012] In response to the eye movement of the person to be collected according to the system prompt, the eye point data corresponding to the eye movement is collected according to the preset collection standard;

[0013] In response to the mouth movement of the person to be collected according to the system prompt, the mouth point data corresponding to the mouth movement is collected according to the preset collection standard.

[0014] Preferably, if the eye point data and / or the mouth point data meet the facial optimization condition, determining the error point in the process of facial expression capture includes:

[0015] Generate corresponding key points according to the eye point data and the mouth point data;

[0016] Locate each of the key points according to a preset positioning method to obtain corresponding motion coordinates of each point;

[0017] Calculate the displacement velocity and acceleration of the motion coordinates of each point;

[0018] Comparing the displacement velocity and the acceleration with standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and the acceleration and the standard values;

[0019] If the comparison result is not within the error tolerance interval, determining that the eye point data and the mouth point data meet the facial optimization condition;

[0020] According to the eye point data and mouth point data that meet the facial optimization conditions, the error points in the process of facial expression capture are determined.

[0021] Preferably, it also includes:

[0022] If the comparison result is within the error tolerance interval, it is determined that the eye point data and the mouth point data do not meet the facial optimization conditions, and the optimization is not performed.

[0023] Preferably, the step of obtaining the coordinate data of the error point and optimizing the coordinate data of the error point includes:

[0024] Determine the movement rate and direction of the error point;

[0025] According to the movement speed and movement direction, coordinate data of the error point is obtained;

[0026] The coordinate data of the error point is corrected to be within a preset trajectory interval, thereby completing the process of optimizing the coordinate data of the error point.

[0027] Preferably, it also includes:

[0028] The coordinate data of the corrected error points are converted into facial point data for driving the virtual image.

[0029] Preferably, the process of determining the standard value and the error tolerance interval includes:

[0030] Obtain historical facial point motion data of personnel;

[0031] Generate each normal point according to the historical facial point motion data of the personnel; the normal point is a point that drives the facial expression of the virtual image in accordance with the preset driving requirements;

[0032] Positioning each normal point according to a preset positioning method to obtain corresponding point motion coordinates of each normal point;

[0033] Calculate the displacement velocity and acceleration of the motion coordinates of each normal point;

[0034] Determine the displacement speed and acceleration of the motion coordinates of each normal point as standard values;

[0035] Determine the maximum acceleration and minimum acceleration through the acceleration of the motion coordinates of each normal point;

[0036] An error tolerance interval is determined according to the maximum acceleration and the minimum acceleration.

[0037] The second aspect of the present application discloses a device for optimizing face data, the device comprising:

[0038] A real-time acquisition unit, used for acquiring facial point data of the person to be acquired in real time; the facial point data at least includes eye point data and mouth point data;

[0039] A first determination unit is used to determine the error point in the process of facial expression capture if the eye point data and / or the mouth point data meet the facial optimization condition; wherein the facial optimization condition is determined by a standard value and a tolerance interval;

[0040] The acquisition optimization unit is used to acquire the coordinate data of the error point and optimize the coordinate data of the error point; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range.

[0041] Preferably, the real-time acquisition unit comprises:

[0042] The first response collection module is used to respond to the eye movements of the person to be collected according to the system prompt, and collect the eye point data corresponding to the eye movement according to the preset collection standard;

[0043] The second response collection module is used to respond to the mouth movement of the person to be collected according to the system prompt, and collect the mouth point data corresponding to the mouth movement according to the preset collection standard.

[0044] Preferably, the first determining unit includes:

[0045] A first generating module, used for generating corresponding key points according to the eye point data and the mouth point data;

[0046] A first positioning module, used to locate each key point according to a preset positioning method to obtain corresponding motion coordinates of each point;

[0047] The first calculation module is used to calculate the displacement speed and acceleration of the motion coordinates of each point;

[0048] A comparison module, used for comparing the displacement velocity and the acceleration with standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and the acceleration and the standard values;

[0049] A first determination module, configured to determine, if the comparison result is not within the error tolerance interval, that the eye point data and the mouth point data meet the facial optimization condition;

[0050] The second determination module is used to determine the error points in the process of facial expression capture according to the eye point data and mouth point data that meet the facial optimization conditions.

[0051] It can be known from the above technical scheme that the present application discloses a method and device for optimizing facial data, which collects facial point data of the person to be collected in real time, and the facial point data at least includes eye point data and mouth point data. If the eye point data and / or mouth point data meet the facial optimization conditions, the error points in the process of facial expression capture are determined, wherein the facial optimization conditions are determined by standard values ​​and fault tolerance intervals, the coordinate data of the error points are obtained, and the coordinate data of the error points are optimized for facial points, and the facial point optimization is used to correct the coordinate data of the error points to a preset trajectory interval range. Through the above scheme, since there are many points on the face and the movements are complex, and the expression animations of the eyes and mouth are the richest and most recognizable, there is no need to capture all the points on the face. It is only necessary to obtain the eye point data and the mouth point data during the process of facial expression capture. Under normal circumstances, the point speed and acceleration of facial movement will change within a reasonable range, that is, the fault tolerance range. It is determined in real time whether the eye point data and the mouth point data meet the facial optimization conditions determined by the standard value and the fault tolerance range, so as to achieve continuous and real-time matching of the facial expression data in the process of capturing facial expressions. If they meet the requirements, the point coordinates of the error points of the erroneous movement in the facial coordinate points can be extracted in real time, and the coordinate data of the error points can be optimized to adjust them within the reasonable trajectory range of the fault tolerance range, thereby eliminating the errors in facial point acquisition, thereby avoiding the situation in which the facial animation of the captured virtual image has jumping movements and overly stiff expressions during the display of expression animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 A flowchart of a method for optimizing facial data disclosed in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of facial expression data feature collection disclosed in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the process of collecting standard values ​​disclosed in the embodiment of the present application;

[0056] Figure 4 A schematic diagram of a virtual image and key points disclosed in an embodiment of the present application;

[0057] Figure 5A schematic diagram of the range of collection points disclosed in the embodiments of the present application;

[0058] Figure 6 A schematic diagram of the specific point distribution for driving the facial expression animation disclosed in the embodiment of the present application;

[0059] Figure 7 A schematic diagram of a motion path of an expression action disclosed in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of facial coordinate points disclosed in an embodiment of the present application;

[0061] Fig. 9 This is a workflow diagram of the facial expression optimization system disclosed in the embodiment of the present application;

[0062] Fig.10 A flowchart of another method for optimizing facial data disclosed in an embodiment of the present application;

[0063] Fig.11 A schematic diagram of the structure of a facial data optimization device disclosed in an embodiment of the present application;

[0064] Fig.12 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0067] As can be seen from the background technology, in the process of displaying facial animation, the facial animation of the captured virtual image may have jumping movements and too stiff expressions. The main reason is that the changes in the movements of the facial features are small and the frequency is fast, and the coordinate displacement of the expression is relatively small. This causes the sampling camera to be unable to continuously match the actor's expression data in real time during the process of capturing the actor's facial expression, and there are errors in facial point collection. Therefore, how to avoid the facial animation of the captured virtual image from having jumping movements and too stiff expressions during the process of displaying facial animation is a problem that needs to be solved urgently in this application.

[0068] In order to solve the above problems, the present application discloses a method and device for optimizing facial data, which collects facial point data of a person to be collected in real time, wherein the facial point data at least includes eye point data and mouth point data. If the eye point data and / or mouth point data meet the facial optimization conditions, the error points in the process of facial expression capture are determined, wherein the facial optimization conditions are determined by a standard value and a tolerance interval, the coordinate data of the error points are obtained, and the coordinate data of the error points are optimized for facial points, and the facial point optimization is used to correct the coordinate data of the error points to a preset trajectory interval range. Through the above scheme, since there are many points on the face and the movements are complex, and the expression animations of the eyes and mouth are the richest and most recognizable, it is not necessary to capture all points on the face. It is only necessary to obtain the eye point data and the mouth point data during the process of facial expression capture. Under normal circumstances, the point speed and acceleration of facial movement will change within a reasonable range, that is, the fault tolerance range. It is determined in real time whether the eye point data and the mouth point data meet the facial optimization conditions determined by the standard value and the fault tolerance range, so as to achieve continuous and real-time matching of the facial expression data in the process of capturing facial expressions. If they meet, the point coordinates of the error points of the wrong movement in the facial coordinate points can be extracted in real time, and the coordinate data of the error points can be optimized to adjust them within the reasonable trajectory range of the fault tolerance range, eliminate the error of facial point acquisition, and avoid the situation that the facial animation of the captured virtual image has jumping movements and the expression is too stiff during the display of the expression animation. The specific implementation method is specifically described by the following embodiments.

[0069] refer to Figure 1 FIG. 1 is a flow chart of a method for optimizing face data disclosed in an example of the present application. The method for optimizing face data mainly includes the following steps:

[0070] S101: Collecting facial point data of the person to be collected in real time; the facial point data at least includes eye point data and mouth point data.

[0071] In S101, facial expression information of the person to be collected is collected based on the facial capture device to obtain eye point data and mouth point data, and the point movement trajectory can be represented by the eye point data and mouth point data.

[0072] In the process of collecting facial point data of the person to be collected, the person to be collected performs relevant facial expressions according to the prompts of the face data optimization device. The relevant facial expressions at least include eye movements and mouth movements.

[0073] Since the frequency and amplitude of facial expression movements vary from person to person, the facial expression optimization system will require the person to be collected to collect facial point movement data before the real-time facial capture optimization process is carried out. The main requirement is to require the person to be collected to perform targeted eye movements (such as turning eyes to the left, turning eyes to the right, closing eyes, blinking, raising eyebrows, frowning, squinting, etc.) and mouth movements (such as opening mouth, closing mouth, grinning, laughing, etc.). Figure 2 As shown, Figure 2 A schematic diagram of facial expression data feature collection is shown.

[0074] In response to the eye movements of the person to be collected according to the prompt of the facial expression optimization system, the eye point data corresponding to the eye movement is collected according to the preset collection standard (such as the collection standard of 60fps). The preset collection standard of this solution includes but is not limited to the collection standard of 60fps.

[0075] In response to the mouth movements of the person to be collected according to the system prompts, the mouth point data corresponding to the mouth movements are collected according to the preset collection standards.

[0076] S102: If the eye point data and the mouth point data meet the facial optimization conditions, determine the error points in the facial expression capture process; wherein the facial optimization conditions are determined by a standard value and a tolerance interval.

[0077] The specific process of collecting standard values, such as Figure 3 As shown, Figure 3 A schematic diagram of the process of collecting standard values ​​is shown.

[0078] Figure 3 In A1: determine whether all facial expressions of the actor (person to be collected) are collected. If not, execute A2. If so, execute A7.

[0079] A2: The facial expression optimization system collects eye movements of actors.

[0080] A3: Prompt the actors to perform the prescribed actions one by one to obtain the eye point data.

[0081] A4: Determine whether all eye point data have been collected. If so, execute A5. If not, return to execute A1.

[0082] A5: The facial expression optimization system collects the mouth movements of the actors.

[0083] A6: The facial expression optimization system prompts the actor to perform the prescribed actions one by one to obtain the mouth point data, and returns to execute A1 to determine whether all the actor's facial expression data has been collected. If so, execute A7.

[0084] A7: Confirm that facial data collection is complete.

[0085] A8: Calculate the displacement speed and acceleration of each point on the motion path of the facial expression.

[0086] A9: The displacement velocity and acceleration of each point are determined as standard values ​​of each point, and the maximum acceleration and minimum acceleration are determined according to the acceleration of each point.

[0087] A10: Determine the tolerance interval based on the maximum acceleration and the minimum acceleration, and store the standard value and tolerance interval of each point in the system database.

[0088] The real-time facial expression optimization system collects the movement path of the facial features of the person to be collected at a standard of 60fps and calculates its key points, locates each key point to obtain the corresponding motion coordinates of each point, and calculates the displacement speed and acceleration of the motion coordinates of each point.

[0089] Specifically, if the eye point data and / or mouth point data meet the facial optimization condition, the process of determining the error points in the facial expression capture process is shown in B1-B6.

[0090] B1: Generate corresponding key points based on the eye point data and mouth point data.

[0091] The specific point distribution that drives facial expression animation. The process of facial capture is to generate key points from the collected facial data. The displacement movement of the key points drives the facial expression animation of the virtual image.

[0092] Specific virtual images and key points are as follows: Figure 4 As shown, Figure 4 A schematic diagram of a virtual image and key points is shown.

[0093] Figure 4 In the figure, the left side is the virtual image to be driven, and the right side GUI interface is the specific point distribution for driving the facial expression animation. The facial capture process is to generate key points from the collected facial data. The purpose of pre-collecting the facial point data of the person is to identify the displacement speed and acceleration of their normal points, so as to calculate the standard value.

[0094] It should be noted that the normal point is the point corresponding to the point data that does not exceed the fault tolerance interval, and the point path that meets the standard trajectory can be reached through the normal point.

[0095] The range of key points generated is as follows Figure 5 As shown, Figure 5 A schematic diagram showing the range of acquisition points.

[0096] The range of key points collected by the real-time facial expression optimization system is Figure 5 The red frame part is mainly concentrated on the eyes and mouth. The purpose of focusing on the eyes and mouth is to maximize the computing efficiency to accurately proofread the most recognizable eye and mouth animations, while ensuring the fastest transmission of data results.

[0097] Figure 5 The 1 marked in the middle corresponds to the left eye, 2 corresponds to both eyes, 3 corresponds to the right eye, and 4 corresponds to the lips. The range of the left eye, both eyes, right eye, and lip point movement (values ​​are -1 to 1). An area can be understood as a plane rectangular coordinate system, and the points controlling the eyes and mouth move in it. Therefore, the coordinate system can be used to locate the motion coordinates of each point, thereby defining the position of the point at different time periods and calculating its displacement speed and acceleration.

[0098] B2: Locate each key point according to the preset positioning method to obtain the corresponding motion coordinates of each point.

[0099] It should be noted that the preset positioning method includes but is not limited to the positioning method of the coordinate system. The preset positioning method of the present application is preferably the positioning method of the coordinate system. The specific positioning method of the coordinate system is as follows Figure 6 shown. Figure 6 A schematic diagram showing the specific point distribution that drives facial expression animation.

[0100] B3: Calculate the displacement velocity and acceleration of the motion coordinates of each point.

[0101] In B3, the displacement speed and acceleration of the motion coordinates of each point are calculated for the motion path of the facial expression. Figure 7 shown.

[0102] The facial expression optimization system obtains the speed and acceleration of each point through the actor's facial expression movement trajectory, calculates the law of point movement of its value, and obtains its standard value and tolerance range.

[0103] Figure 7 X1, X2, X3, and X4 are the displacement distances of the points at the same time t. dot To express it, we can get the displacement velocity and acceleration Calculation formula.

[0104] In order to facilitate the calculation of the displacement velocity and acceleration of the motion coordinates of each point, combined with the above Figure 6 Provide explanation.

[0105] by Figure 6 Taking the coordinate system midpoint M as an example, the derived calculation formulas for its displacement velocity are shown in formulas (1), (2) and (3).

[0106]

[0107] in, is the displacement velocity of coordinate point M; For the direction; and are all unit vectors; x and y are rectangular coordinates; v x is the x-axis displacement velocity; v y is the displacement speed of the y-axis; through this expression formula, the displacement speed of point M can be obtained. The value of can be used to find its acceleration The specific values ​​of are shown in formula (4), formula (5) and formula (6).

[0108]

[0109] Among them, a x is the x-axis acceleration; a y is the y-axis acceleration.

[0110] The formulas for acceleration and displacement velocity (i.e., formula (1) to formula (6)) are input into the facial expression optimization system as the basic algorithm, which can detect the data of the facial animation point movement of the person in real time. For the collected data, the facial expression optimization system will automatically calculate it and obtain its maximum value, minimum value and average value, the average acceleration a bar The derivative formula is shown in formula (7):

[0111]

[0112] Among them, v t is the final velocity of the point displacement, v0 is the initial velocity of the point displacement; t is the time.

[0113] In addition, in addition to the coordinate system motion parameters described above, the system will identify the other linear motion points that also need to be collected and identified, such as Figure 8 shown. Figure 8 A schematic diagram showing facial coordinate points.

[0114] Figure 8In the figure, the red framed part is the facial coordinate point that moves in a straight line. In the process of facial expression capture, the facial points move in a non-uniformly accelerated straight line, so the movement pattern of the points will be as follows: Figure 7 The facial expression optimization system will obtain the speed and acceleration of each point through the actor's facial expression movement trajectory, calculate the law of point movement of its value, and obtain its standard value and tolerance range.

[0115] Figure 7 X1, X2, X3, and X4 are the displacement distances of the points at the same time t. dot To express it, we can get the displacement velocity and acceleration The calculation formulas are shown in formula (8) and formula (9).

[0116]

[0117] B4: Compare the displacement velocity and acceleration with the standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and acceleration and the standard values.

[0118] B5: If the comparison result is not within (i.e. exceeds) the error tolerance interval, it is determined that the eye point data and the mouth point data meet the facial optimization conditions.

[0119] In order to facilitate the understanding of the process of comparing displacement velocity and acceleration with standard values, an explanation is given in conjunction with Table 1 and Table 2. Table 1 shows the standard values. Table 2 shows the point data parameters collected in real-time facial capture.

[0120]

[0121] Table 1

[0122]

[0123] Table 2

[0124] For example, the point number in the point data collected from the actual face is X1, the displacement speed is 65.4 mm / s, and the displacement acceleration is 1659 mm / s 2 , the displacement speed is 65.4mm / s within the speed tolerance range of 54.6mm / s-82.3mm / s, and the displacement acceleration is 1659mm / s 2 In the acceleration tolerance range 1457mm / s 2 -2146mm / s 2 Therefore, the comparison result is that the displacement velocity and displacement acceleration of point number X1 are within the tolerance interval, and it is determined that it does not meet the facial optimization conditions.

[0125] For example, the point number is X3, the displacement speed is 51.2mm / s, and the displacement acceleration is 1423mm / s 2 , the displacement speed of 51.2mm / s is not within the speed tolerance range of 54.6mm / s-82.3mm / s, and the displacement acceleration is 1423mm / s 2 Not in the acceleration tolerance range 1457mm / s 2 -2146mm / s 2 Therefore, the comparison result is that the displacement velocity and displacement acceleration of point number X3 are not within the tolerance interval, and it is determined that the facial optimization conditions are met.

[0126] After data collection is completed, real-time facial capture begins. During the process of facial driving the virtual image, the data will be collected in real time by the real-time facial expression optimization system. The real-time displacement speed and acceleration of each point will be calculated through the movement of each point. At the same time, the real-time facial expression optimization system will judge these data. When the difference with the standard value exceeds the tolerance range, the facial expression optimization system will adjust the point coordinate information to achieve a point path that meets the standard trajectory. It is preferred to collect and calculate the point data at a standard of 60fps.

[0127] The point calculation of this solution is based on 60fps. The identification of points requires the system to collect, calculate and identify data. This process cannot be done in advance. Therefore, in the process of real-time synchronization of the point data captured by the person's face, the entire facial expression optimization system sets a delay of 5-10fps. This has two benefits:

[0128] First, the system needs to perform large calculations on the facial capture data. On the basis of saving the amount of calculation, a certain delay is still required, which can better ensure the accuracy of the system operation and ensure that the animation driven by the facial capture binding is smoother and more natural;

[0129] Second, the delay of 5-10fps will not cause a large delay in the real-time facial capture process. The perception and impact of facial capture work are within a controllable range and are acceptable. The delay of this solution includes but is not limited to 5-10fps.

[0130] B6: Determine the error points in the process of facial expression capture based on the eye point data and mouth point data that meet the facial optimization conditions.

[0131] In B6, the eye point data that meets the facial optimization conditions, that is, the displacement speed and acceleration values ​​of the eye point exceed the standard value or fault tolerance interval in the acquisition process, and the facial expression optimization system will automatically determine that the eye point is an error point.

[0132] When the mouth point data that meets the facial optimization conditions, that is, when the displacement velocity and acceleration values ​​of the mouth point exceed the standard values ​​or the tolerance interval during the acquisition process, the facial expression optimization system will automatically determine that the mouth point is an error point.

[0133] All the above derived formulas are written into the real-time facial expression optimization system. Through calculation, the point data of the facial capture point when driving the virtual image can be obtained in real time. The displacement speed and acceleration calculated by the point data of the virtual image can be used as the standard value for system judgment. At the same time, its maximum acceleration and minimum acceleration will become the fault tolerance interval of the system judgment, that is, the reference interval. The eye point data and mouth point data are judged to meet the facial optimization conditions through the fault tolerance interval.

[0134] During normal facial capture, the facial expression optimization system will calculate the point data of the eyes and mouth according to the above formula to obtain the displacement speed and acceleration values. When the comparison results of the displacement speed and acceleration with the standard values ​​are in the normal range, that is, they do not exceed the fault tolerance range, the point data will drive the facial expression of the virtual image normally. On the contrary, when the speed and acceleration values ​​exceed or even far exceed the standard values ​​or normal ranges during the acquisition process, the facial expression optimization system will automatically determine that the point is an error point, and adjust the position data of the error point to correct it to the path interval that conforms to the normal speed trajectory, so as to achieve the effect of optimizing the expression data.

[0135] The process of determining the standard value and the tolerance interval is shown in C1-C7.

[0136] C1: Obtain historical facial point motion data of people.

[0137] C2: Generate normal points based on historical facial point motion data of personnel; normal points are points that drive the facial expressions of the virtual image in accordance with preset driving requirements.

[0138] C3: Position each normal point according to a preset positioning method (i.e., the positioning method of the coordinate system) to obtain the corresponding point motion coordinates of each normal point.

[0139] C4: Calculate the displacement velocity and acceleration of the motion coordinates of each normal point.

[0140] C5: Determine the displacement velocity and acceleration of the motion coordinates of each normal point as standard values.

[0141] C6: Determine the maximum acceleration and minimum acceleration through the acceleration of the motion coordinates of each normal point.

[0142] C7: Determine the tolerance interval based on the maximum acceleration and the minimum acceleration.

[0143] If the comparison result is within the tolerance range, it is determined that the eye point data and the mouth point data do not meet the facial optimization conditions, and the optimization operation of the facial point data is not performed.

[0144] S103: Acquire the coordinate data of the error point and optimize the coordinate data of the error point; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range. The preset trajectory interval range is determined according to the fault tolerance interval. Specifically, the preset trajectory interval range is a path interval obtained by converting the displacement velocity and acceleration in the fault tolerance interval.

[0145] The preset trajectory interval range is the path interval of the normal speed trajectory. The virtual image can only be driven when the coordinate data of the error point is corrected to the path interval of the normal speed trajectory.

[0146] The coordinate data of the error point is corrected to the preset trajectory range through optimization, that is, within the normal range, to achieve the point path that meets the standard trajectory. The coordinate data of the error point is optimized to adjust it to the reasonable trajectory range of the fault tolerance interval, so as to eliminate the error of facial point collection and improve the accuracy of facial point collection.

[0147] The specific process of obtaining the coordinate data of the error point and optimizing the coordinate data of the error point is shown in D1-D3.

[0148] D1: Determine the movement speed and direction of the error point.

[0149] D2: Obtain the coordinate data of the error point based on the movement speed and direction.

[0150] D3: Correcting the coordinate data of the error point to correct the coordinate data of the error point to a preset trajectory interval range, thereby completing the process of optimizing the coordinate data of the error point.

[0151] The coordinate data of the corrected error points are converted into facial point data for driving the virtual image, so as to realize real-time driving of the virtual image.

[0152] In order to understand the specific implementation process of the facial expression optimization system, combined with Fig. 9 To explain, Fig. 9 A workflow diagram of the facial expression optimization system is shown.

[0153] Fig. 9 In the middle, E1: perform facial data detection, camera capture and extract face area.

[0154] E2: The facial expression optimization system locates facial feature points and collects actor’s facial data.

[0155] E3: The facial expression optimization system performs facial feature point calculations, and the system obtains standard values ​​of the facial feature points.

[0156] E4: The facial expression optimization system performs real-time facial capture, and the facial points drive the virtual image's expression animation in real time.

[0157] E5: The facial expression optimization system optimizes facial feature points. The system calculates and identifies point data in real time and optimizes point coordinate information based on standard values ​​and tolerance intervals.

[0158] E6: The facial expression optimization system saves the facial animation data, and the system will capture the points for further optimization and correction.

[0159] The facial point data proofread by the facial expression optimization system is transmitted to the driving points of the virtual image. The point data will be transmitted with a delay of 5-10fps, and the expression animation begins to be performed in real time.

[0160] When facial capture is finished, the facial expression optimization system will prompt whether to save the collected facial expression animation. During the saving process, the facial expression optimization system will connect the captured points into a data link for further calibration and analysis to ensure the highest quality facial animation as much as possible.

[0161] When the animation output in real time by facial capture needs to be saved and reused, the facial expression optimization system will connect the collected point information to form a data trajectory. Given the displacement speed and acceleration formula, the motion arc of the line will be re-determined, the tolerance value between the points will be further optimized, and it will be reduced as much as possible within the tolerance range. This will make the movement trajectory of the expression points smoother, providing effective help for the later use of the collected animation.

[0162] In order to facilitate the understanding of the optimization process of face data, combined with Fig.10 Provide explanation. Fig.10 Another optimization method for face data is shown.

[0163] Fig.10 In the middle, F1: real-time collection of actor’s facial point data;

[0164] F2: Real-time monitoring of the actor's facial eye and mouth point data;

[0165] F3: Determine whether the point data is a standard value or within the normal range (tolerance range);

[0166] F4: If no, confirm the coordinate data and position information of the error point in real time; if yes, execute F8.

[0167] F5: Generate point trajectory based on acceleration and velocity data in real time;

[0168] F6: Correct the error point data to the normal trajectory range in real time;

[0169] F7: Real-time optimization of point data;

[0170] F8: Continue moving along the normal trajectory without making any changes or adjustments to the optimization system.

[0171] In the embodiment of the present application, since there are many points on the face and the movements are complex, and the expression animations of the eyes and mouth are the richest and most recognizable, there is no need to capture all points on the face. It is only necessary to obtain the eye point data and the mouth point data during the process of facial expression capture. Under normal circumstances, the point speed and acceleration of facial movement will change within a reasonable range, that is, the fault tolerance range. It is determined in real time whether the eye point data and the mouth point data meet the facial optimization conditions determined by the standard value and the fault tolerance range, so as to achieve continuous and real-time matching of the facial expression data in the process of capturing facial expressions. If they meet the requirements, the point coordinates of the error points of the erroneous movement in the facial coordinate points can be extracted in real time, and the coordinate data of the error points can be optimized to adjust them within the reasonable trajectory range of the fault tolerance range, thereby eliminating the errors in facial point acquisition, thereby avoiding the situation in which the facial animation of the captured virtual image has jumping movements and overly stiff expressions during the display of expression animation.

[0172] Based on the above embodiments Figure 1 A method for optimizing face data is disclosed, and an embodiment of the present application also discloses a device for optimizing face data, such as Fig.11 As shown, the face data optimization device includes:

[0173] The real-time acquisition unit 1101 is used to acquire facial point data of the person to be acquired in real time; the facial point data at least includes eye point data and mouth point data;

[0174] The first determining unit 1102 is used to determine the error point in the process of facial expression capture if the eye point data and the mouth point data meet the facial optimization condition; wherein the facial optimization condition is determined by a standard value and a tolerance interval;

[0175] The acquisition optimization unit 1103 is used to acquire the coordinate data of the error point and optimize the coordinate data of the error point; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range.

[0176] Furthermore, the real-time acquisition unit 1101 includes:

[0177] The first response collection module is used to respond to the eye movements of the person to be collected according to the system prompts, and collect eye point data corresponding to the eye movements according to the preset collection standards;

[0178] The second response collection module is used to respond to the mouth movements of the person to be collected according to the system prompts, and collect the mouth point data corresponding to the mouth movements according to the preset collection standards.

[0179] Furthermore, the first determining unit 1102 includes:

[0180] The first generating module is used to generate corresponding key points according to the eye point data and the mouth point data;

[0181] The first positioning module is used to locate each key point according to a preset positioning method to obtain the corresponding motion coordinates of each point;

[0182] The first calculation module is used to calculate the displacement speed and acceleration of the motion coordinates of each point;

[0183] A comparison module is used to compare the displacement velocity and acceleration with the standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and acceleration and the standard values;

[0184] A first determination module, configured to determine, if the comparison result is not within the error tolerance interval, that the eye point data and the mouth point data meet the facial optimization condition;

[0185] The second determination module is used to determine the error points in the process of facial expression capture according to the eye point data and mouth point data that meet the facial optimization conditions.

[0186] Furthermore, it also includes:

[0187] The second determination unit is used to determine that the eye point data and the mouth point data do not meet the facial optimization conditions and do not perform the optimization if the comparison result is within the error tolerance interval.

[0188] Further, the optimization unit 1103 is obtained, including:

[0189] The third determination module is used to determine the movement rate and movement direction of the error point;

[0190] A first acquisition module is used to acquire coordinate data of the error point according to the movement speed and movement direction;

[0191] The correction module is used to correct the coordinate data of the error point to correct the coordinate data of the error point to a preset trajectory interval range, thereby completing the process of optimizing the coordinate data of the error point.

[0192] Furthermore, it also includes:

[0193] The conversion module is used to convert the coordinate data of the corrected error points into facial point data for driving the virtual image.

[0194] Furthermore, the first determination unit 902 for determining the standard value and the error tolerance interval includes:

[0195] The second acquisition module is used to obtain historical facial point motion data of personnel;

[0196] A generation module, used to generate normal points according to historical facial point motion data of personnel; normal points are points that drive the facial expressions of the virtual image in accordance with preset driving requirements;

[0197] The third acquisition module is used to locate each normal point according to a preset positioning method to obtain the corresponding point motion coordinates of each normal point;

[0198] The second calculation module is used to calculate the displacement speed and acceleration of the motion coordinates of each normal point;

[0199] A fourth determination module is used to determine the displacement speed and acceleration of the motion coordinates of each normal point as standard values;

[0200] A fifth determination module, used to determine the maximum acceleration and the minimum acceleration through the acceleration of the motion coordinates of each normal point;

[0201] The sixth determination module is used to determine the error tolerance interval according to the maximum acceleration and the minimum acceleration.

[0202] In the embodiment of the present application, since there are many points on the face and the movements are complex, and the expression animations of the eyes and mouth are the richest and most recognizable, there is no need to capture all points on the face. It is only necessary to obtain the eye point data and the mouth point data during the process of facial expression capture. Under normal circumstances, the point speed and acceleration of facial movement will change within a reasonable range, that is, the fault tolerance range. It is determined in real time whether the eye point data and the mouth point data meet the facial optimization conditions determined by the standard value and the fault tolerance range, so as to achieve continuous and real-time matching of the facial expression data in the process of capturing facial expressions. If they meet the requirements, the point coordinates of the error points of the erroneous movement in the facial coordinate points can be extracted in real time, and the coordinate data of the error points can be optimized to adjust them within the reasonable trajectory range of the fault tolerance range, thereby eliminating the errors in facial point acquisition, thereby avoiding the situation in which the facial animation of the captured virtual image has jumping movements and overly stiff expressions during the display of expression animation.

[0203] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned method for optimizing facial data.

[0204] The present application also provides an electronic device, whose structural diagram is as follows: Fig.12 As shown, it specifically includes a memory 1201 and one or more instructions 1202, wherein the one or more instructions 1202 are stored in the memory 1201 and are configured to be executed by one or more processors 1203 to execute the one or more instructions 1202 to perform the above-mentioned face data optimization method.

[0205] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0206] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0207] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0208] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0209] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0210] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for optimizing face data, characterized in that: The method comprises: Collecting facial point data of the person to be collected in real time; the facial point data at least includes eye point data and mouth point data; If the eye point data and / or the mouth point data meet the facial optimization condition, determine the error point in the facial expression capture process; wherein the facial optimization condition is determined by a standard value and a tolerance interval; The coordinate data of the error point is obtained, and the coordinate data of the error point is optimized; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range.

2. The method according to claim 1, characterized in that The real-time collection of facial point data of the person to be collected includes: In response to the eye movement of the person to be collected according to the system prompt, the eye point data corresponding to the eye movement is collected according to the preset collection standard; In response to the mouth movement of the person to be collected according to the system prompt, the mouth point data corresponding to the mouth movement is collected according to the preset collection standard.

3. The method according to claim 1, characterized in that If the eye point data and / or the mouth point data meet the facial optimization condition, determining the error point in the facial expression capture process includes: Generate corresponding key points according to the eye point data and the mouth point data; Locate each of the key points according to a preset positioning method to obtain corresponding motion coordinates of each point; Calculate the displacement velocity and acceleration of the motion coordinates of each point; Comparing the displacement velocity and the acceleration with standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and the acceleration and the standard values; If the comparison result is not within the error tolerance interval, determining that the eye point data and the mouth point data meet the facial optimization condition; According to the eye point data and mouth point data that meet the facial optimization conditions, the error points in the process of facial expression capture are determined.

4. The method according to claim 3, characterized in that Also includes: If the comparison result is within the error tolerance interval, it is determined that the eye point data and the mouth point data do not meet the facial optimization conditions, and the optimization is not performed.

5. The method according to claim 1, characterized in that The obtaining of coordinate data of the error point and optimizing the coordinate data of the error point includes: Determine the movement rate and direction of the error point; According to the movement speed and movement direction, coordinate data of the error point is obtained; The coordinate data of the error point is corrected to be within a preset trajectory interval, thereby completing the process of optimizing the coordinate data of the error point.

6. The method according to claim 1, characterized in that Also includes: The coordinate data of the corrected error points are converted into facial point data for driving the virtual image.

7. The method according to claim 1, characterized in that The process of determining standard values ​​and tolerance intervals includes: Obtain historical facial point motion data of personnel; Generate each normal point according to the historical facial point motion data of the personnel; the normal point is a point that drives the facial expression of the virtual image in accordance with the preset driving requirements; Positioning each normal point according to a preset positioning method to obtain corresponding point motion coordinates of each normal point; Calculate the displacement velocity and acceleration of the motion coordinates of each normal point; Determine the displacement speed and acceleration of the motion coordinates of each normal point as standard values; Determine the maximum acceleration and minimum acceleration through the acceleration of the motion coordinates of each normal point; An error tolerance interval is determined according to the maximum acceleration and the minimum acceleration.

8. A facial data optimization device, characterized in that: The device comprises: A real-time acquisition unit, used for acquiring facial point data of the person to be acquired in real time; the facial point data at least includes eye point data and mouth point data; A first determination unit is used to determine the error point in the process of facial expression capture if the eye point data and / or the mouth point data meet the facial optimization condition; wherein the facial optimization condition is determined by a standard value and a tolerance interval; The acquisition optimization unit is used to acquire the coordinate data of the error point and optimize the coordinate data of the error point; the optimization is used to correct the coordinate data of the error point to a preset trajectory interval range.

9. The device according to claim 8, characterized in that The real-time acquisition unit comprises: The first response collection module is used to respond to the eye movements of the person to be collected according to the system prompt, and collect the eye point data corresponding to the eye movement according to the preset collection standard; The second response collection module is used to respond to the mouth movement of the person to be collected according to the system prompt, and collect the mouth point data corresponding to the mouth movement according to the preset collection standard.

10. The device according to claim 8, characterized in that The first determining unit includes: A first generating module, used for generating corresponding key points according to the eye point data and the mouth point data; A first positioning module, used to locate each key point according to a preset positioning method to obtain corresponding motion coordinates of each point; The first calculation module is used to calculate the displacement speed and acceleration of the motion coordinates of each point; A comparison module, used for comparing the displacement velocity and the acceleration with standard values ​​to obtain a comparison result; wherein the comparison result represents the difference between the displacement velocity and the acceleration and the standard values; A first determination module, configured to determine, if the comparison result is not within the error tolerance interval, that the eye point data and the mouth point data meet the facial optimization condition; The second determination module is used to determine the error points in the process of facial expression capture according to the eye point data and mouth point data that meet the facial optimization conditions.