A facial motion capture system and method based on data analysis

Abstract

The invention relates to the technical field of facial motion capture, and discloses a facial motion capture system and method based on data analysis; the facial motion capture system comprises a 3D recording module, a storage module, a processing module, a recognition module and an analysis module; the processing module divides a standard lip shape video stored in the storage module and a follow-up lip shape video recorded by the 3D recording module into i basic units according to word syllables; the recognition module performs frame-by-frame recognition on standard image data to obtain a standard lip area, a standard tooth area and a standard tongue area; the follow-up image data is recognized frame-by-frame to obtain a follow-up lip area, a follow-up tooth area and a follow-up tongue area; finally, the analysis module obtains the lip shape matching degree and the sound similarity of the basic unit; the follow-up score is calculated according to the lip shape matching degree and the sound similarity of each basic unit according to a score calculation rule, and the accuracy of pronunciation can be more accurately evaluated by comparing the sound and the lip shape.

Description

Technical Field

[0001] The present invention relates to the technical field of facial motion capture, and in particular to a facial motion capture system and method based on data analysis. Background Art

[0002] Facial movements are generally achieved through the contraction and relaxation of facial muscles to achieve various facial expressions and movements. The coordinated movement and position of facial muscles affect the accuracy and clarity of pronunciation when speaking. In language learning, the accuracy and clarity of pronunciation directly affect the effectiveness of information transmission.

[0003] With the rapid development of mobile Internet technology, more and more people are learning languages ​​through APP. When learning the pronunciation of words, learners look at the phonetic symbols of the words, listen to the correct pronunciation of the words, and then practice following the words. They record the words while following the words, compare the recordings with the correct pronunciation, and give scores based on the comparison results. Learners understand their own pronunciation based on the scores.

[0004] However, there are many words that require facial muscles to make corresponding facial movements in order to be pronounced accurately. Based on the learner's follow-up recording and the correct pronunciation, we can only judge the similarity of pronunciation, but cannot accurately judge the accuracy of pronunciation. Summary of the invention

[0005] The purpose of the present invention is to provide a facial motion capture system and method based on data analysis to solve the following technical problems:

[0006] How to more accurately assess pronunciation accuracy.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A facial motion capture system based on data analysis, the facial motion capture system comprising:

[0009] 3D recording module, used to record lip-syncing videos;

[0010] A storage module is used to store the standard lip shape video corresponding to each word;

[0011] A processing module, connected to the storage module and the 3D recording module, for dividing the standard lip-sync video and the follow-up lip-sync video into i basic units according to word syllables;

[0012] The basic unit includes a standard pronunciation unit and a follow-up pronunciation unit;

[0013] The standard pronunciation unit includes standard image data and standard sound data;

[0014] The follow-up pronunciation unit includes follow-up image data and follow-up sound data;

[0015] The recognition module is connected to the processing module and is used to perform frame-by-frame recognition on the standard image data to obtain the standard lip area, the standard tooth area and the standard tongue area; and perform frame-by-frame recognition on the follow-up image data to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area;

[0016] The analysis module is connected to the recognition module and is used to analyze the standard lip area, the standard teeth area, the standard tongue area, the follow-up lip area, the follow-up teeth area and the follow-up tongue area to obtain the lip shape matching degree of the basic unit; compare the standard sound data and the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the scoring calculation rules.

[0017] As a further solution of the present invention: the scoring calculation rule is through the formula:

[0018]

[0019] Calculate the reading-along score P;

[0020] Among them, P0 is the full score; S i is the sound similarity of the ith basic unit; S0 is the ith preset sound similarity; K i is the lip shape matching degree of the i-th basic unit; f(X) is the judgment function; when X≥0, f(X)=1; when X<0, f(X)=0; δ1 is the first weight coefficient; δ2 is the second weight coefficient.

[0021] As a further solution of the present invention: the process of obtaining the lip shape matching degree of the basic unit includes the following steps:

[0022] S10: acquiring the three-dimensional coordinates of each point in the standard lip region, the standard tooth region, and the standard tongue region of the standard image data frame by frame;

[0023] S20: analyzing the three-dimensional coordinates of each point in the standard lip region, the standard tooth region, and the standard tongue region of each frame to obtain a standard lip change parameter curve, a standard tooth exposure change curve, and a standard tongue exposure change curve;

[0024] S30: Acquire the three-dimensional coordinates of each point in the lip region, the teeth region, and the tongue region of the follow-up image data frame by frame;

[0025] S40: Analyze the three-dimensional coordinates of each point in the lip area, the teeth area and the tongue area of ​​each frame to obtain a lip change parameter curve, a teeth exposure change curve and a tongue exposure change curve;

[0026] S50: Compare the standard lip change parameter curve with the follow-up lip change parameter curve to obtain the lip change similarity; compare the standard teeth exposure change curve with the follow-up teeth exposure change curve to obtain the teeth change similarity; compare the standard tongue exposure change curve with the follow-up tongue exposure change curve to obtain the tongue change similarity;

[0027] S60: Calculate the lip change similarity, the teeth change similarity and the tongue change similarity according to a preset formula to obtain the lip shape matching degree.

[0028] As a further solution of the present invention: the preset formula is:

[0029] K i =γ1X C +γ2f(X C -X1)X Y +γ3f(X Y -X2)X S

[0030] Among them, X C is the lip change similarity; X1 is the preset lip change similarity; X Y is the tooth change similarity; X2 is the preset tooth change similarity; X S is the similarity of tongue change; γ1 is the first proportional coefficient; γ2 is the second proportional coefficient; γ3 is the third proportional coefficient.

[0031] As a further solution of the present invention: the standard lip change parameter curve includes a standard outer lip angle width change curve, a standard inner lip angle width change curve and a standard lip protrusion change curve.

[0032] As a further solution of the present invention: the lip change parameter curve includes a lip angle width change curve, an inner lip angle width change curve and a lip protrusion change curve.

[0033] As a further solution of the present invention: the lip change similarity X C By public notice:

[0034]

[0035] Among them, X W is the similarity of the change of the width of the outer lip angle; X N is the similarity of the change of inner lip angle width; X Tis the similarity of lip protrusion change; Z W is the learner's outer lip angle width in the preset state; Z WS is the preset outer lip angle width in the preset state; Z N is the learner's inner lip angle width in the preset state; Z NS is the preset inner lip angle width in the preset state; Z T is the learner's lip protrusion in the preset state; Z TS It is the preset lip protrusion in the preset state.

[0036] A facial motion capture method based on data analysis, the facial motion capture method comprising the following steps:

[0037] S1: Record lip-reading video through 3D recording module;

[0038] S2: Segment the standard lip-sync video and the follow-up lip-sync video stored in the storage module into i basic units according to word syllables through the processing module;

[0039] S3: The standard image data of the basic unit is recognized frame by frame by the recognition module to obtain the standard lip area, the standard tooth area and the standard tongue area; the follow-up image data is recognized frame by frame to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area;

[0040] S4: Analyze the standard lip area, standard teeth area, standard tongue area, follow-up lip area, follow-up teeth area and follow-up tongue area through the analysis module to obtain the lip shape matching degree of the basic unit; compare the standard sound data and the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the scoring calculation rules.

[0041] Beneficial effects of the present invention:

[0042] The present invention records a follow-up lip shape video through a 3D recording module; stores a standard lip shape video corresponding to each word through a storage module; divides the standard lip shape video and the follow-up lip shape video into i basic units according to the word syllables through a processing module; then, the standard image data of the basic unit is recognized frame by frame through a recognition module to obtain a standard lip area, a standard tooth area and a standard tongue area; the follow-up image data is recognized frame by frame to obtain a follow-up lip area, a follow-up tooth area and a follow-up tongue area; finally, the standard lip area, the standard tooth area, the standard tongue area, the follow-up lip area, the follow-up tooth area and the follow-up tongue area are analyzed through an analysis module to obtain the lip shape matching degree of the basic unit; the standard sound data and the follow-up sound data are compared to obtain the sound similarity of the basic unit; then, the follow-up score is calculated according to the lip shape matching degree and the sound similarity of each basic unit according to a score calculation rule, and the accuracy of pronunciation can be more accurately evaluated through the sound and lip shape comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below in conjunction with the accompanying drawings.

[0044] Figure 1 A schematic diagram of a system module framework of an embodiment of the present invention;

[0045] Figure 2 The present invention is a method flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 As shown, in one embodiment, a facial motion capture system based on data analysis is provided, comprising:

[0048] 3D recording module, used to record lip-syncing videos;

[0049] A storage module is used to store the standard lip shape video corresponding to each word;

[0050] A processing module, connected to the storage module and the 3D recording module, for dividing the standard lip-sync video and the follow-up lip-sync video into i basic units according to word syllables;

[0051] The basic unit includes a standard pronunciation unit and a follow-up pronunciation unit;

[0052] The standard pronunciation unit includes standard image data and standard sound data;

[0053] The follow-up pronunciation unit includes follow-up image data and follow-up sound data;

[0054] The recognition module is connected to the processing module and is used to perform frame-by-frame recognition on the standard image data to obtain the standard lip area, the standard tooth area and the standard tongue area; and perform frame-by-frame recognition on the follow-up image data to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area;

[0055] The analysis module is connected to the recognition module and is used to analyze the standard lip area, the standard teeth area, the standard tongue area, the follow-up lip area, the follow-up teeth area and the follow-up tongue area to obtain the lip shape matching degree of the basic unit; compare the standard sound data with the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the score calculation rules;

[0056] Through the above technical scheme, this embodiment records the lip shape video through the 3D recording module; stores the standard lip shape video corresponding to each word through the storage module; divides the standard lip shape video and the lip shape video into i basic units according to the syllables of the word through the processing module; then, the standard image data of the basic unit is recognized frame by frame by the recognition module to obtain the standard lip area, standard tooth area and standard tongue area; the follow-up image data is recognized frame by frame to obtain the follow-up lip area, follow-up tooth area and follow-up tongue area; finally, the standard lip area, standard tooth area, standard tongue area, follow-up lip area, follow-up tooth area and follow-up tongue area are analyzed through the analysis module to obtain the lip shape matching degree of the basic unit; the standard sound data and the follow-up sound data are compared to obtain the sound similarity of the basic unit; then, the follow-up score is calculated according to the lip shape matching degree and sound similarity of each basic unit according to the scoring calculation rules, and the accuracy of pronunciation can be more accurately evaluated by comparing the sound and lip shape.

[0057] As an implementation mode of the present invention, the scoring calculation rule is through the formula:

[0058]

[0059] Calculate the reading-along score P;

[0060] Among them, P0 is the full score; S i is the sound similarity of the ith basic unit; S0 is the preset sound similarity; K i is the lip shape matching degree of the i-th basic unit; f(X) is the judgment function; when X≥0, f(X)=1; when X<0, f(X)=0; δ1 is the first weight coefficient; δ2 is the second weight coefficient;

[0061] Through the above technical solution, in this embodiment, the follow-up reading score P is calculated through the formula Calculate the follow-up reading score P; δ1S i +δ2f(S i -S0)K i is the comprehensive similarity of the i-th basic unit; is the average similarity of the i basic units; when the average similarity of the i basic units is higher, the follow-up reading score P is closer to P0; when S i <S0, the comprehensive similarity of the i-th basic unit is γ1X C ; There is no need to calculate the lip matching degree, reducing the calculation amount; when S i ≥S0, the comprehensive similarity of the i-th basic unit is δ1S i +δ2K i , more accurately evaluate the pronunciation accuracy.

[0062] It should be noted that, therefore, the voice similarity S of the i-th basic unit i The acquisition method is the prior art; the preset voice similarity S0, the first weight coefficient δ1, and the second weight coefficient δ2 are preset values, obtained according to experience, and will not be elaborated here.

[0063] As an implementation manner of the present invention, the obtaining process of the lip matching degree of the basic unit includes the following steps:

[0064] S10: Frame by frame, obtain the three-dimensional coordinates of each point in the standard lip area, standard tooth area, and standard tongue area within the standard image data;

[0065] S20: Analyze the three-dimensional coordinates of each point in each frame of the standard lip area, standard tooth area, and standard tongue area to obtain the standard lip change parameter curve, standard tooth exposure change curve, and standard tongue exposure change curve;

[0066] S30; Frame by frame, obtain the three-dimensional coordinates of each point in the follow-up reading lip area, follow-up reading tooth area, and follow-up reading tongue area within the follow-up reading image data;

[0067] S40: Analyze the three-dimensional coordinates of each point in each frame of the follow-up reading lip area, follow-up reading tooth area, and follow-up reading tongue area to obtain the follow-up reading lip change parameter curve, follow-up reading tooth exposure change curve, and follow-up reading tongue exposure change curve;

[0068] S50: Compare the standard lip change parameter curve with the follow-up lip change parameter curve to obtain the lip change similarity; compare the standard teeth exposure change curve with the follow-up teeth exposure change curve to obtain the teeth change similarity; compare the standard tongue exposure change curve with the follow-up tongue exposure change curve to obtain the tongue change similarity;

[0069] S60: Calculating the lip change similarity, the teeth change similarity and the tongue change similarity according to a preset formula to obtain a lip shape matching degree;

[0070] Through the above technical scheme, this embodiment obtains the three-dimensional coordinates of each point in the standard lip area, the standard tooth area and the standard tongue area of ​​the standard image data frame by frame; then analyzes the three-dimensional coordinates of each point in the standard lip area, the standard tooth area and the standard tongue area of ​​each frame to obtain the standard lip change parameter curve, the standard tooth exposure change curve and the standard tongue exposure change curve; then obtains the three-dimensional coordinates of each point in the follow-up lip area, the follow-up tooth area and the follow-up tongue area of ​​the follow-up image data frame by frame; then analyzes the three-dimensional coordinates of each point in the follow-up lip area, the follow-up tooth area and the follow-up tongue area of ​​each frame The following lip change parameter curve, the following tooth exposure change curve and the following tongue exposure change curve are obtained by analysis; then the standard lip change parameter curve is compared with the following lip change parameter curve to obtain the lip change similarity; the standard tooth exposure change curve is compared with the following tooth exposure change curve to obtain the tooth change similarity; the standard tongue exposure change curve is compared with the following tongue exposure change curve to obtain the tongue change similarity; finally, the lip change similarity, the tooth change similarity and the tongue change similarity are calculated according to a preset formula to obtain the lip shape matching degree; the accuracy of the lip shape matching degree is improved;

[0071] It should be noted that since the standard image data is obtained through a 3D recording module, specifically, the 3D recording module can use a 3D camera, so the three-dimensional coordinates of each point in the standard image data can be obtained; and the similarity of the changes of two curved edges to obtain curves is an existing technology and will not be described in detail here.

[0072] As an implementation manner of the present invention, the preset formula is:

[0073] K i =γ1X C +γ2f(X C -X1)X Y +γ3f(X Y -X2)X S

[0074] Among them, X C is the lip change similarity; X1 is the preset lip change similarity; XY is the similarity of dental changes; X2 is the preset similarity of dental changes; X S is the similarity of tongue changes; γ1 is the first proportionality coefficient; γ2 is the second proportionality coefficient; γ3 is the third proportionality coefficient;

[0075] Through the above technical solution, in this embodiment, the formula K i = γ1X C + γ2f(X C - X1)X Y + γ3f(X Y - X2)X S is used to calculate the mouth shape matching degree of the i-th basic unit; when X C < X1, K i = γ1X C ; there is no need to calculate the similarity of dental changes and the similarity of tongue changes; when X C ≥ X1, X Y < X2, K i = γ1X C + γ2X Y ; there is no need to calculate the similarity of tongue changes; when X C ≥ X1, X Y ≥ X2, K i = γ1X C + γ2X Y + γ3X S ; the mouth shape matching degree of the i-th basic unit is calculated more accurately;

[0076] It should be noted that the preset lip change similarity X1, the preset dental change similarity X2, the first proportionality coefficient γ1, the second proportionality coefficient γ2, and the third proportionality coefficient γ3 are preset values, which are obtained according to experience and will not be elaborated here.

[0077] As an implementation manner of the present invention, the standard lip change parameter curve includes a standard outer lip angle width change curve, a standard inner lip angle width change curve, and a standard lip protrusion change curve;

[0078] The follow-up lip change parameter curve includes a follow-up outer lip angle width change curve, a follow-up inner lip angle width change curve, and a follow-up lip protrusion change curve;

[0079] The similarity X C of lip changes is calculated more accurately from the standard outer lip angle width change curve and the follow-up outer lip angle width change curve, the standard inner lip angle width change curve and the follow-up inner lip angle width change curve, and the standard lip protrusion change curve and the follow-up lip protrusion change curve;

[0080] As an implementation manner of the present invention, the lip change similarity X CBy public notice:

[0081]

[0082] Among them, X W is the similarity of the change of the width of the outer lip angle; X N is the similarity of the change of inner lip angle width; X T is the similarity of lip protrusion change; Z W is the learner's outer lip angle width in the preset state; Z WS is the preset outer lip angle width in the preset state; Z N is the learner's inner lip angle width in the preset state; Z NS is the preset inner lip angle width in the preset state; Z T is the learner's lip protrusion in the preset state; Z TS is the preset lip protrusion in the preset state;

[0083] Through the above technical solution, this embodiment is the ratio of the absolute value of the difference between the learner's inner lip angle width in the preset state and the preset inner lip angle width to the preset inner lip angle width; is the ratio of the absolute value of the difference between the learner's inner lip angle width in the preset state and the preset inner lip angle width to the preset inner lip angle width; is the ratio of the absolute value of the difference between the learner's lip protrusion in the preset state and the preset lip protrusion to the preset lip protrusion; is the similarity between the learner and the preset lip shape in the preset state; then through the formula Calculate the lip change similarity X C ; Prevent the lip shape change similarity X caused by the large difference between the learner's lip shape and the preset lip shape C Produce influence and improve the similarity of lip changes X C The accuracy of the outer lip angle width change similarity X W The larger the value, the more similar the lip changes are. C The larger the inner lip angle width, the similarity of change X N The larger the value, the more similar the lip changes are. C The larger the size, the similarity of lip protrusion change X T The larger the value, the more similar the lip changes are. C The bigger;

[0084] It should be noted that the default state is that the mouth muscles are relaxed when not speaking.

[0085] See also Figure 2 As shown, a facial motion capture method based on data analysis, the facial motion capture method comprises the following steps:

[0086] S1: Record lip-reading video through 3D recording module;

[0087] S2: Segment the standard lip-sync video and the follow-up lip-sync video stored in the storage module into i basic units according to word syllables through the processing module;

[0088] S3: The standard image data of the basic unit is recognized frame by frame by the recognition module to obtain the standard lip area, the standard tooth area and the standard tongue area; the follow-up image data is recognized frame by frame to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area;

[0089] S4: Analyze the standard lip area, standard teeth area, standard tongue area, follow-up lip area, follow-up teeth area and follow-up tongue area through the analysis module to obtain the lip shape matching degree of the basic unit; compare the standard sound data and the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the scoring calculation rules.

[0090] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A facial motion capture system based on data analysis, characterized in that: The facial motion capture system includes: 3D recording module, used to record lip-syncing videos; A storage module is used to store the standard lip shape video corresponding to each word; A processing module, connected to the storage module and the 3D recording module, for dividing the standard lip-sync video and the follow-up lip-sync video into i basic units according to word syllables; The basic unit includes a standard pronunciation unit and a follow-up pronunciation unit; The standard pronunciation unit includes standard image data and standard sound data; The follow-up pronunciation unit includes follow-up image data and follow-up sound data; The recognition module is connected to the processing module and is used to perform frame-by-frame recognition on the standard image data to obtain the standard lip area, the standard tooth area and the standard tongue area; and perform frame-by-frame recognition on the follow-up image data to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area; The analysis module is connected to the recognition module and is used to analyze the standard lip area, the standard teeth area, the standard tongue area, the follow-up lip area, the follow-up teeth area and the follow-up tongue area to obtain the lip shape matching degree of the basic unit; compare the standard sound data with the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the score calculation rules; The process of obtaining the lip shape matching degree of the basic unit comprises the following steps: S10: acquiring the three-dimensional coordinates of each point in the standard lip region, the standard tooth region, and the standard tongue region of the standard image data frame by frame; S20: analyzing the three-dimensional coordinates of each point in the standard lip region, the standard tooth region, and the standard tongue region of each frame to obtain a standard lip change parameter curve, a standard tooth exposure change curve, and a standard tongue exposure change curve; S30: Acquire the three-dimensional coordinates of each point in the lip region, the teeth region, and the tongue region of the follow-up image data frame by frame; S40: Analyze the three-dimensional coordinates of each point in the lip area, the teeth area and the tongue area of ​​each frame to obtain a lip change parameter curve, a teeth exposure change curve and a tongue exposure change curve; S50: Compare the standard lip change parameter curve with the follow-up lip change parameter curve to obtain the lip change similarity; compare the standard teeth exposure change curve with the follow-up teeth exposure change curve to obtain the teeth change similarity; compare the standard tongue exposure change curve with the follow-up tongue exposure change curve to obtain the tongue change similarity; S60: Calculate the lip change similarity, the teeth change similarity and the tongue change similarity according to a preset formula to obtain the lip shape matching degree.

2. A facial motion capture system based on data analysis according to claim 1, characterized in that: The scoring calculation rule is as follows: Calculate the reading-along score P; Among them, P0 is the full score; S i is the sound similarity of the ith basic unit; S0 is the ith preset sound similarity; K i is the lip shape matching degree of the i-th basic unit; f(X) is the judgment function; when X≥0, f(X)=1; when X<0, f(X)=0; δ1 is the first weight coefficient; δ2 is the second weight coefficient.

3. A facial motion capture system based on data analysis according to claim 1, characterized in that: The preset formula is: K i =γ1X C +γ2f(X C -X1)X Y +γ3f(X Y -X2)X S Among them, X C is the lip change similarity; X1 is the preset lip change similarity; X Y is the tooth change similarity; X2 is the preset tooth change similarity; X S is the similarity of tongue change; γ1 is the first proportional coefficient; γ2 is the second proportional coefficient; γ3 is the third proportional coefficient.

4. A facial motion capture system based on data analysis according to claim 3, characterized in that: The standard lip change parameter curve includes a standard outer lip angle width change curve, a standard inner lip angle width change curve and a standard lip protrusion change curve.

5. A facial motion capture system based on data analysis according to claim 4, characterized in that: The lip change parameter curve includes a lip angle width change curve, a lip angle width change curve and a lip protrusion change curve.

6. A facial motion capture system based on data analysis according to claim 5, characterized in that: The lip change similarity X C By formula: Among them, X W is the similarity of the change of the width of the outer lip angle; X N is the similarity of the change of inner lip angle width; X T is the similarity of lip protrusion change; Z W is the learner's outer lip angle width in the preset state; Z WS is the preset outer lip angle width in the preset state; Z N is the learner's inner lip angle width in the preset state; Z NS is the preset inner lip angle width in the preset state; Z T is the learner's lip protrusion in the preset state; Z TS It is the preset lip protrusion in the preset state.

7. A facial motion capture method based on data analysis, applicable to a facial motion capture system based on data analysis according to any one of claims 1 to 6, characterized in that: The facial motion capture method comprises the following steps: S1: Record lip-reading video through 3D recording module; S2: Segment the standard lip-sync video and the follow-up lip-sync video stored in the storage module into i basic units according to word syllables through the processing module; S3: The standard image data of the basic unit is recognized frame by frame by the recognition module to obtain the standard lip area, the standard tooth area and the standard tongue area; the follow-up image data is recognized frame by frame to obtain the follow-up lip area, the follow-up tooth area and the follow-up tongue area; S4: Analyze the standard lip area, standard teeth area, standard tongue area, follow-up lip area, follow-up teeth area and follow-up tongue area through the analysis module to obtain the lip shape matching degree of the basic unit; compare the standard sound data and the follow-up sound data to obtain the sound similarity of the basic unit; and then calculate the follow-up score according to the lip shape matching degree and sound similarity of each basic unit according to the scoring calculation rules.

Images