Method, device, computer equipment and storage medium for acquiring facial feature point coordinates

Through feature point detection and mapping technology, the face rotation angle and feature point coordinates are calculated, which solves the problem of inaccurate detection of traditional face feature point and achieves high-accurate feature point acquisition.

CN114155565BActive Publication Date: 2024-11-19SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010824213.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-17
Publication Date
2024-11-19
Estimated Expiration
2040-08-17

AI Technical Summary

Technical Problem

Traditional face feature point detection methods cannot accurately obtain face feature point coordinates, resulting in inaccurate detection.

Method used

By obtaining the image to be processed, feature point detection is performed to obtain the average 3D face parameters and 3D face parameters, feature point mapping is performed to obtain the 2D face parameters, calculate the face rotation angle, determine the feature point detection method and feature point to be detected , and then obtain the coordinates of the characteristic point of the face.

Benefits of technology

Accurate face feature point detection is achieved, and the accuracy of detection is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114155565B_ABST
    Figure CN114155565B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device, computer equipment and storage medium for obtaining the coordinates of facial feature points. The method comprises: obtaining an image to be processed; performing feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters; performing feature point mapping on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters; calculating the face rotation angle corresponding to the image to be processed according to the 2D face parameters; determining the feature point detection method according to the face rotation angle, determining the corresponding feature points to be detected according to the feature point detection method and the correspondence between the preset feature point detection method and the feature points to be detected; obtaining the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected. The present method can accurately detect facial feature points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for acquiring coordinates of facial feature points. Background Art

[0002] With the development of computer technology, facial feature point detection technology has emerged. Facial feature point detection is a key step in the field of face recognition and analysis. It is the premise and breakthrough for other face-related problems such as automatic face recognition, expression analysis, 3D face reconstruction and 3D animation.

[0003] In traditional technology, deep learning methods are used to detect facial feature points. A detection model for facial feature point detection is first trained using the deep learning method, and then the detection model is used to detect facial feature points.

[0004] However, the conventional technology has the problem of inaccurate facial feature point detection due to the inability to accurately obtain the coordinates of facial feature points. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for obtaining facial feature point coordinates that can improve the accuracy of facial feature point detection in response to the above technical problems.

[0006] A method for acquiring coordinates of facial feature points, the method comprising:

[0007] Get the image to be processed;

[0008] Perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0009] Perform feature point mapping on 3D face parameters according to the average 3D face parameters to obtain 2D face parameters;

[0010] According to the 2D face parameters, calculate the face rotation angle corresponding to the image to be processed;

[0011] Determine a feature point detection method according to the face rotation angle, and determine the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship;

[0012] According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0013] In one embodiment, feature point detection is performed on the image to be processed to obtain the 3D face average parameter and the 3D face parameter, including:

[0014] Feature point detection is performed on the processed image according to the trained face average model to obtain 3D face average parameters, and feature point detection is performed on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model.

[0015] In one embodiment, feature point mapping is performed on 3D face parameters according to the 3D face average parameters to obtain 2D face parameters including:

[0016] According to the 3D face parameters, the 3D face average parameters are screened to obtain the target 3D face average parameters, and the target 3D face average parameters and the 3D face parameters represent the same facial features;

[0017] Calculate the motion vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, wherein the first expression is an equation describing the corresponding relationship between the 3D face parameters, the target 3D face average parameters, and the face rotation angle, and the second expression is an equation describing the corresponding relationship between the 3D face parameters, the motion vector, and the 2D face parameters;

[0018] According to the 3D face parameters, the movement vector, the first expression and the second expression, the 2D face parameters are obtained.

[0019] In one embodiment, calculating the face rotation angle corresponding to the image to be processed according to the 2D face parameters includes:

[0020] Get the current 2D feature parameters corresponding to the 2D face parameters;

[0021] Calculate the feature difference between the current 2D feature parameter and each corresponding feature point in the 2D face parameter, and obtain a third expression, where the third expression is an equation describing the corresponding relationship between the feature difference and the rotation angle change;

[0022] According to the feature difference and the third expression, the change in the rotation angle is calculated, and the rotation angle of the face at the previous moment corresponding to the image to be processed is obtained;

[0023] According to the rotation angle change and the face rotation angle at the previous moment, the face rotation angle corresponding to the image to be processed is obtained.

[0024] In one embodiment, determining a feature point detection method according to a face rotation angle includes:

[0025] Determine head posture based on face rotation angle;

[0026] Determine the feature point detection method based on the head posture comparison preset posture-feature point detection method correspondence.

[0027] In one embodiment, obtaining the coordinates of facial feature points corresponding to the image to be processed according to the feature points to be detected includes:

[0028] Perform feature data query based on the feature points to be detected;

[0029] When the feature data corresponding to the feature point to be detected is queried, the feature point is tracked according to the feature data to obtain the coordinates of the facial feature point corresponding to the image to be processed;

[0030] When feature data corresponding to the feature points to be detected are not found, feature point detection is performed based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0031] In one embodiment, tracking feature points according to feature data to obtain the coordinates of facial feature points corresponding to the image to be processed includes:

[0032] Track the feature points according to the feature data and the preset tracking algorithm to obtain the parameter increment corresponding to the feature point to be detected;

[0033] Calculate the feature point value corresponding to the feature point to be detected according to the feature data and parameter increment;

[0034] According to the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0035] A device for acquiring coordinates of facial feature points, the device comprising:

[0036] An acquisition module, used for acquiring an image to be processed;

[0037] A detection module is used to detect feature points of the image to be processed and obtain 3D face average parameters and 3D face parameters;

[0038] A mapping module is used to map the 3D face parameters to feature points according to the 3D face average parameters to obtain 2D face parameters;

[0039] A calculation module, used to calculate the face rotation angle corresponding to the image to be processed according to the 2D face parameters;

[0040] The first processing module is used to determine a feature point detection method according to the face rotation angle, and determine corresponding feature points to be detected according to the feature point detection method and a preset feature point detection method-feature point to be detected correspondence relationship;

[0041] The second processing module is used to obtain the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected.

[0042] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] Get the image to be processed;

[0044] Perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0045] Perform feature point mapping on 3D face parameters according to the average 3D face parameters to obtain 2D face parameters;

[0046] According to the 2D face parameters, calculate the face rotation angle corresponding to the image to be processed;

[0047] Determine a feature point detection method according to the face rotation angle, and determine the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship;

[0048] According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0050] Get the image to be processed;

[0051] Perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0052] Perform feature point mapping on 3D face parameters according to the average 3D face parameters to obtain 2D face parameters;

[0053] According to the 2D face parameters, calculate the face rotation angle corresponding to the image to be processed;

[0054] Determine a feature point detection method according to the face rotation angle, and determine the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship;

[0055] According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0056] The above-mentioned method, device, computer equipment and storage medium for acquiring the coordinates of facial feature points perform feature point detection on the image to be processed to obtain the 3D face average parameters and 3D face parameters, perform feature point mapping on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters, calculate the face rotation angle corresponding to the image to be processed according to the 2D face parameters, determine the feature point detection method according to the face rotation angle, determine the corresponding feature points to be detected according to the feature point detection method and the preset feature point detection method-the correspondence relationship between the feature points to be detected, and obtain the facial feature point coordinates corresponding to the image to be processed according to the feature points to be detected. The whole process can estimate the head posture by analyzing the image to be processed, obtain the accurate face rotation angle, and then use the face rotation angle to determine the feature point detection method, so that the corresponding feature points to be detected can be obtained on the basis of determining the feature point detection method, and the facial feature point coordinates can be obtained according to the feature points to be detected, so as to achieve accurate facial feature point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of a flow chart of a method for acquiring facial feature point coordinates in one embodiment;

[0058] Figure 2 Schematic diagram of a method for acquiring facial feature point coordinates in one embodiment;

[0059] Figure 3 Schematic diagram of a method for acquiring facial feature point coordinates in one embodiment;

[0060] Figure 4 Schematic diagram of a method for acquiring facial feature point coordinates in one embodiment;

[0061] Figure 5 Schematic diagram of a method for acquiring facial feature point coordinates in one embodiment;

[0062] Figure 6 is a schematic flow chart of a method for acquiring facial feature point coordinates in another embodiment;

[0063] Figure 7 It is a structural block diagram of a device for acquiring coordinates of facial feature points in one embodiment;

[0064] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] In one embodiment, Figure 1 As shown, a method for obtaining the coordinates of facial feature points is provided. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0067] Step 102: Obtain an image to be processed.

[0068] The image to be processed refers to a facial image to be processed.

[0069] Specifically, the server may obtain the image to be processed from a preset image database or a user terminal. The preset image database refers to a database that pre-stores the image to be processed, and the image to be processed in the database may be obtained in advance. Obtaining the image to be processed from the user terminal refers to the user terminal uploading the image to be processed to the server.

[0070] Step 104 , performing feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters.

[0071] Among them, feature point detection refers to detecting facial feature points in the image to be processed and obtaining the position information of the facial feature points in the image to be processed. Facial feature points refer to points used to characterize facial features. For example, facial feature points may specifically refer to points used to characterize features such as nose, eyes, eyebrows, mouth, ears, etc. 3D face average parameters refer to parameters obtained by performing feature point detection using a trained face average model, which are used to characterize the positions of feature points on the image to be processed. For example, 3D face average parameters may specifically refer to 3D face average feature points. 3D face parameters refer to parameters obtained by performing feature point detection using a trained face feature point detection model, which are used to characterize the positions of feature points on the image to be processed. For example, 3D face parameters may specifically refer to 3D face feature points.

[0072] Specifically, the server can perform feature point detection on the processed image according to the trained average face model to obtain 3D average face parameters, and perform feature point detection on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained average face model can be obtained by training the initial active shape model. The active shape model is a relatively mature method for locating facial feature points. It uses a local texture model to perform local searches around feature points and uses a global statistical model to constrain the shape composed of feature point sets. The two are iterated repeatedly and finally converge to the optimal shape. The trained facial feature point detection model can be a common 13-point facial feature point detection model, an 11-point facial feature point detection model, etc., which is not specifically limited here.

[0073] Step 106 , mapping the 3D face parameters to feature points according to the 3D face average parameters to obtain 2D face parameters.

[0074] The 2D face parameters refer to parameters obtained by mapping the 3D face parameters to feature points, that is, performing dimensionality reduction processing on the 3D face parameters. For example, the 2D face parameters may specifically refer to 2D face feature points.

[0075] Specifically, the server will select the corresponding target 3D face average parameters from the 3D face average parameters according to the 3D face parameters. The target 3D face average parameters and the 3D face parameters represent the same facial features, that is, the feature points of the target 3D face average parameters correspond to the feature points of the 3D face parameters. For example, the target 3D face average parameters and the 3D face parameters represent the same facial features, which can specifically refer to facial features such as the nose, eyes, and lips, that is, the nose, eyes, lips, and other facial features can be represented by the target 3D face average parameters, or by the 3D face parameters. After obtaining the target 3D face average parameters, the server will calculate the movement vector between the 3D face parameters and the target 3D face average parameters, and obtain the first expression. The 3D face parameters are mapped to feature points through the movement vector and the first expression to obtain 2D face parameters. Among them, the first expression describes the equation of the corresponding relationship between 3D face parameters, target 3D face average parameters and face rotation angle. Through feature point mapping, 2D face parameters can be described by movement vectors and face rotation angles.

[0076] Step 108: Calculate the face rotation angle corresponding to the image to be processed according to the 2D face parameters.

[0077] Among them, the face rotation angle refers to the posture angle of a person's head, which is similar to the flight of an airplane, namely the three Euler angles of pitch, yaw and roll, which are scientifically known as pitch angle, yaw angle and roll angle respectively. In layman's terms, it means raising your head, shaking your head and turning your head.

[0078] Specifically, the server obtains the current 2D feature parameters corresponding to the 2D face parameters, calculates the feature difference between the current 2D feature parameters and each corresponding feature point in the 2D face parameters, obtains the third expression, calculates the rotation angle change according to the feature difference and the third expression, and obtains the face rotation angle according to the rotation angle change. Among them, the current 2D feature parameters can be obtained by any feature point detection method, such as the supervised descent method, etc., which is not specifically limited here. The third expression is an equation that describes the corresponding relationship between the feature difference and the rotation angle change.

[0079] Step 110, determining a feature point detection method according to the face rotation angle, and determining corresponding feature points to be detected according to the feature point detection method and a preset feature point detection method-feature point to be detected correspondence relationship.

[0080] Among them, the feature point detection method refers to the method of performing feature point detection. For example, the feature point detection method can specifically be front face detection. For another example, the feature point detection method can specifically be left face detection. For another example, the feature point detection method can specifically be right face detection. The preset feature point detection method-the correspondence relationship between the feature point detection method and the feature points to be detected is used to characterize the correspondence between the feature point detection method and the feature points to be detected, and the correspondence includes the quantity correspondence, etc. For example, the feature point detection method-the correspondence relationship between the feature points to be detected can specifically be 68 feature points corresponding to the front face detection. For another example, the feature point detection method-the correspondence relationship between the feature points to be detected can specifically be 24 feature points corresponding to the left face detection. For another example, the feature point detection method-the correspondence relationship between the feature points to be detected can specifically be 24 feature points corresponding to the right face detection. The corresponding feature points to be detected correspond to the feature point detection method.

[0081] Specifically, the server determines the head posture according to the face rotation angle, determines the feature point detection method according to the head posture, and determines the corresponding feature points to be detected according to the feature point detection method and the correspondence between the preset feature point detection method and the feature points to be detected.

[0082] Step 112, obtaining the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected.

[0083] The facial feature point coordinates refer to the coordinates of the facial feature points on the image to be processed.

[0084] Specifically, the server will query feature data based on the feature points to be detected and determine the method of obtaining the coordinates of the facial feature points. When the feature data corresponding to the feature points to be detected can be queried, the coordinates of the facial feature points are obtained by feature point tracking. When the feature data corresponding to the feature points to be detected cannot be queried, the coordinates of the facial feature points are obtained directly by feature point detection.

[0085] The above-mentioned method for acquiring the coordinates of facial feature points performs feature point detection on the image to be processed to obtain the 3D face average parameters and 3D face parameters, performs feature point mapping on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters, calculates the face rotation angle corresponding to the image to be processed according to the 2D face parameters, determines the feature point detection method according to the face rotation angle, determines the corresponding feature points to be detected according to the feature point detection method and the preset feature point detection method-the correspondence between the feature points to be detected, and obtains the facial feature point coordinates corresponding to the image to be processed according to the feature points to be detected. The whole process can estimate the head posture by analyzing the image to be processed, obtain an accurate face rotation angle, and then use the face rotation angle to determine the feature point detection method, so that the corresponding feature points to be detected can be obtained on the basis of determining the feature point detection method, and the facial feature point coordinates can be obtained according to the feature points to be detected, so as to achieve accurate facial feature point detection.

[0086] In one embodiment, feature point detection is performed on the image to be processed to obtain the 3D face average parameter and the 3D face parameter, including:

[0087] Feature point detection is performed on the processed image according to the trained face average model to obtain 3D face average parameters, and feature point detection is performed on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model.

[0088] Specifically, the server performs feature point detection on the processed image according to the trained 3D face average model to obtain 3D face average parameters, and performs feature point detection on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model. Figure 2 As shown, the method of obtaining the average face model by training the initial active shape model can be: first obtain the training set, describe it in matrix form, then complete the description of the training set through principal component analysis (PCA), and shape a priori model that can reflect the average contour of the training set and the key deformation mode samples, and finally complete the search of the priori model through the grayscale matching method. When performing iterative search, the parameters of the priori model need to be adjusted so that the model and the actual contour of the target object gradually match to achieve accurate positioning. Among them, the training set refers to the sample obtained by using the device to collect the boundary point set of the target contour.

[0089] In this embodiment, by performing feature point detection on the processed image according to the trained face average model to obtain 3D face average parameters, and by performing feature point detection on the processed image according to the trained face feature point detection model to obtain 3D face parameters, the acquisition of 3D face average parameters and 3D face parameters can be achieved.

[0090] In one embodiment, feature point mapping is performed on 3D face parameters according to the 3D face average parameters to obtain 2D face parameters including:

[0091] According to the 3D face parameters, the 3D face average parameters are screened to obtain the target 3D face average parameters, and the target 3D face average parameters and the 3D face parameters represent the same facial features;

[0092] Calculate the motion vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, wherein the first expression is an equation describing the corresponding relationship between the 3D face parameters, the target 3D face average parameters, and the face rotation angle, and the second expression is an equation describing the corresponding relationship between the 3D face parameters, the motion vector, and the 2D face parameters;

[0093] According to the 3D face parameters, the movement vector, the first expression and the second expression, the 2D face parameters are obtained.

[0094] Specifically, the server will screen the 3D face average parameters according to the 3D face parameters to obtain the target 3D face average parameters corresponding to the 3D face parameters, where the correspondence refers to the correspondence between the feature points of the target 3D face average parameters and the feature points of the 3D face parameters. After obtaining the target 3D face average parameters, the server will calculate the movement vector between the 3D face parameters and the target 3D face average parameters, and obtain the first expression and the second expression, and obtain the 2D face parameters according to the 3D face parameters, the movement vector, the first expression and the second expression. Among them, the first expression is an equation describing the correspondence between the 3D face parameters, the target 3D face average parameters and the face rotation angle, and the second expression is an equation describing the correspondence between the 3D face parameters, the movement vector and the 2D face parameters.

[0095] Furthermore, the first expression in this embodiment may specifically be:

[0096]

[0097] Among them, xc, yc, zc refer to the coordinates of the feature points in the 3D face parameters, xm, ym, zm refer to the coordinates of the feature points in the corresponding target 3D face average parameters, and θx, θy, θz refer to the face rotation angles. The second expression can be specifically: where x2D ,y 2D refers to the coordinates of the feature points in the 2D face parameters, s refers to the preset scaling factor, mx and my refer to the movement vectors. Through the first and second expressions, the 2D face parameters can be obtained as follows:

[0098]

[0099] In this embodiment, by screening the 3D face average parameters according to the 3D face parameters, the target 3D face average parameters are obtained, the movement vector between the 3D face parameters and the target 3D face average parameters is calculated, and the first expression and the second expression are obtained. According to the 3D face parameters, the movement vector, the first expression and the second expression, the 2D face parameters are obtained, so that the acquisition of the 2D face parameters can be realized.

[0100] In one embodiment, calculating the face rotation angle corresponding to the image to be processed according to the 2D face parameters includes:

[0101] Get the current 2D feature parameters corresponding to the 2D face parameters;

[0102] Calculate the feature difference between the current 2D feature parameter and each corresponding feature point in the 2D face parameter, and obtain a third expression, where the third expression is an equation describing the corresponding relationship between the feature difference and the rotation angle change;

[0103] According to the feature difference and the third expression, the change in the rotation angle is calculated, and the rotation angle of the face at the previous moment corresponding to the image to be processed is obtained;

[0104] According to the rotation angle change and the face rotation angle at the previous moment, the face rotation angle corresponding to the image to be processed is obtained.

[0105] Among them, the current 2D feature parameter refers to the parameter obtained by detecting the feature point of the image to be processed, which is used to characterize the two-dimensional position of the feature point on the image to be processed. For example, the current 2D feature parameter can specifically refer to the current 2D facial feature point. The feature difference refers to the difference between the feature point in the current 2D feature parameter and the corresponding feature point in the 2D facial parameter. The rotation angle change refers to the change of the face rotation angle at the last moment. The face rotation angle at the last moment refers to the face rotation angle calculated at the last moment, which will be pre-stored in the preset database. Further, each time a new face rotation angle is calculated, the face rotation angle stored in the preset database will be updated to obtain the latest face rotation angle at the last moment. For example, in this embodiment, after the face rotation angle corresponding to the image to be processed is calculated, the face rotation angle will be updated to the latest face rotation angle at the last moment and stored in the preset database.

[0106] Specifically, the server first uses feature point detection to obtain the current 2D feature parameters corresponding to the 2D face parameters, then calculates the feature difference between the current 2D feature parameters and each corresponding feature point in the 2D face parameters, and obtains the third expression, calculates the rotation angle change according to the feature difference and the third expression, and obtains the face rotation angle at the last moment corresponding to the image to be processed, and finally obtains the face rotation angle corresponding to the image to be processed according to the rotation angle change and the face rotation angle at the last moment. The third expression is an equation that describes the corresponding relationship between the feature difference and the rotation angle change.

[0107] Furthermore, the formula for calculating the feature difference can be: where x SDM ,y SDM Refers to the coordinates of the feature points in the current 2D feature parameters, x 2D ,y 2D refers to the coordinates of the corresponding feature points in the 2D face parameters. The formula of the third expression is as follows Figure 3 As shown, hx1, hy1...hx13, hy13 refer to the corresponding feature points in the 2D face parameters, and the corresponding feature points in the 2D face parameters can be represented by the movement vector and the face rotation angle, Δx1, Δy1,...Δx13, Δy13 refer to the feature difference, θx, θy, θz refer to the face rotation angle, s refers to the preset scaling factor, mx, my refer to the movement vector. Δθ refers to the change in the rotation angle, and θ refers to the face rotation angle. Figure 3 It can be seen that according to the feature difference and the third expression, the method for calculating the change in the rotation angle can be: calculating the differential of each corresponding feature point in the 2D face parameters on the 3D transformation parameters according to the third expression, and calculating the change in the rotation angle according to the differential matrix and the feature difference.

[0108] In this embodiment, by obtaining the current 2D feature parameters corresponding to the 2D face parameters, the feature differences between the current 2D feature parameters and the corresponding feature points in the 2D face parameters are calculated, and a third expression is obtained. According to the feature differences and the third expression, the change in the rotation angle is calculated, and the face rotation angle at the previous moment corresponding to the image to be processed is obtained. According to the change in the rotation angle and the face rotation angle at the previous moment, the face rotation angle corresponding to the image to be processed is obtained, thereby realizing the acquisition of the face rotation angle.

[0109] In one embodiment, determining a feature point detection method according to a face rotation angle includes:

[0110] Determine head posture based on face rotation angle;

[0111] Determine the feature point detection method based on the head posture comparison preset posture-feature point detection method correspondence.

[0112] Among them, head posture is used to characterize facial correspondence, including frontal correspondence, left face correspondence, and right face correspondence.

[0113] Specifically, the server will compare the preset face rotation angle-head posture correspondence relationship based on the face rotation angle. The preset face rotation angle-head posture correspondence relationship defines the correspondence between each face rotation angle and head posture. The head posture corresponding to the face rotation angle can be determined by comparison. For example, the face rotation angle-head posture correspondence relationship can be described as follows: the pitch angle is within the range of x degrees, the yaw angle is within the range of x degrees, and the roll angle is within the range of x degrees for the front face, the pitch angle is within the range of y degrees, the yaw angle is within the range of y degrees, and the roll angle is within the range of y degrees for the left face, and the pitch angle is within the range of z degrees, the yaw angle is within the range of z degrees, and the roll angle is within the range of z degrees for the right face. The specific angle values ​​can be set as needed. After obtaining the head posture, the server will compare the preset posture-feature point detection method correspondence relationship based on the head posture to determine the feature point detection method. Among them, the head posture and the feature point detection method are corresponding. For example, when the head posture corresponds to the front face, the front face detection is adopted, when the head posture corresponds to the left face, the left face detection is adopted, and when the head posture corresponds to the right face, the right face detection is adopted.

[0114] In this embodiment, the feature point detection method is determined by determining the head posture according to the rotation angle of the face, comparing the head posture with the preset posture-feature point detection method correspondence, and determining the feature point detection method.

[0115] In one embodiment, obtaining the coordinates of facial feature points corresponding to the image to be processed according to the feature points to be detected includes:

[0116] Perform feature data query based on the feature points to be detected;

[0117] When the feature data corresponding to the feature point to be detected is queried, the feature point is tracked according to the feature data to obtain the coordinates of the facial feature point corresponding to the image to be processed;

[0118] When feature data corresponding to the feature points to be detected are not found, feature point detection is performed based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0119] Among them, the feature data refers to the facial image at the previous moment corresponding to the feature point to be detected, and the time amplitude can be set as needed. For example, the time amplitude can be 1 second, then the feature data refers to the facial image 1 second ago. Feature point tracking refers to tracking the feature points at the previous moment when they exist. Feature point detection refers to detecting feature points to obtain feature points when there are no feature points at the previous moment.

[0120] Specifically, the server will query the feature data based on the feature points to be detected, and determine the method of obtaining the coordinates of the facial feature points through the query results. When the feature data corresponding to the feature points to be detected is queried, it means that there are feature points at the previous moment corresponding to the feature points to be detected (which can be obtained through the feature data). The server will track the feature points based on the feature data to obtain the coordinates of the facial feature points corresponding to the image to be processed. When the feature data corresponding to the feature points to be detected is not queried, it means that there are no feature points at the previous moment corresponding to the feature points to be detected. The server will perform feature point detection based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0121] In this embodiment, by querying feature data based on the feature points to be detected, determining the method for obtaining the coordinates of the facial feature points based on the query results, and determining the coordinates of the facial feature points based on the method for obtaining the coordinates of the facial feature points, the coordinates of the facial feature points can be obtained.

[0122] In one embodiment, tracking feature points according to feature data to obtain the coordinates of facial feature points corresponding to the image to be processed includes:

[0123] Track the feature points according to the feature data and the preset tracking algorithm to obtain the parameter increment corresponding to the feature point to be detected;

[0124] Calculate the feature point value corresponding to the feature point to be detected according to the feature data and parameter increment;

[0125] According to the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0126] The preset tracking algorithm refers to an algorithm used for tracking feature points. For example, the preset tracking algorithm may be a KLT (Kanade-Lucas-Tomasi Tracking Method) algorithm. The parameter increment refers to the increase in the feature point value of the feature point to be detected relative to the feature point value of the corresponding feature point at the previous moment.

[0127] Specifically, the server will track feature points based on the feature data and a preset tracking algorithm to obtain parameter increments corresponding to the feature points to be detected, obtain the feature point values ​​of each feature point at the previous moment based on the feature data, calculate the feature point values ​​corresponding to the feature points to be detected based on the feature point values ​​and parameter increments of each feature point at the previous moment, query the image to be processed based on the feature point values, and obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0128] For example, the preset tracking algorithm may be a KLT algorithm. The process of tracking feature points according to the feature data and the preset tracking algorithm to obtain the parameter increment corresponding to the feature point to be detected may be as follows: Figure 4 As shown, Pre-compute includes: step: (1) calculating the gradient map of the template image (feature data), step: (2) calculating the Jacobian matrix, step: (3) calculating the steepest descent image, step: (4) calculating the Hessian inverse matrix. Iterate includes: step: (5) initializing the warped image parameters, step: (6) calculating the image difference, step: (7) calculating the parameter increment using the least squares method, and step: (8) updating the parameters until the parameters converge.

[0129] In this embodiment, feature points are tracked according to feature data and a preset tracking algorithm to obtain parameter increments corresponding to the feature points to be detected. Feature point values ​​corresponding to the feature points to be detected are calculated based on the feature data and the parameter increments. Based on the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained, thereby realizing the acquisition of the coordinates of the facial feature points.

[0130] In one embodiment, Figure 5 As shown, the facial feature point coordinate acquisition method of the present application can be applied to facial feature point detection and tracking of multiple people in multiple postures. The application of the facial feature point coordinate acquisition in facial feature point detection and tracking of multiple people in multiple postures is as follows: the server obtains the image to be processed (the facial image obtained after multi-person facial detection), performs feature point detection (key point detection) on the image to be processed, obtains 3D face average parameters and 3D face parameters, performs feature point mapping on the 3D face parameters according to the 3D face average parameters, obtains 2D face parameters, calculates the face rotation angle corresponding to the image to be processed according to the 2D face parameters (key point tracking and head posture estimation), determines the feature point detection method according to the face rotation angle, determines the corresponding feature points to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship (obtains the structure and selects the most relevant model to detect the key points), and obtains the facial feature point coordinates corresponding to the image to be processed according to the feature points to be detected (key point tracking).

[0131] In one embodiment, Figure 6 As shown, a flow chart is used to illustrate the facial feature point coordinate acquisition method of the present application, which includes: a server obtains an image to be processed (input image), performs feature point detection on the image to be processed, obtains 3D face average parameters and 3D face parameters (wherein the 3D face parameters are 13 key points, when there are 13 key points at the previous moment, directly performs feature point tracking, and when there are no 13 key points, performs feature point detection. Specifically, the front face has 13 key points, and the left face and the right face both have 11 key points), performs feature point mapping on the 3D face parameters according to the 3D face average parameters, obtains 2D face parameters, calculates the face rotation angle (head posture estimation) corresponding to the image to be processed according to the 2D face parameters, and determines the face rotation angle according to the face rotation angle A feature point detection method is determined (68 key points for the front face, and 24 key points for the left and right faces), and the corresponding feature points to be detected are determined according to the feature point detection method and the preset feature point detection method-the correspondence between the feature points to be detected. According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained, wherein the 68 key points refer to the feature points at the previous moment corresponding to the feature points to be detected. When the feature points at the previous moment exist, 68 feature point tracking (68 key point tracking) is performed to obtain the facial feature point coordinates (facial key point coordinates) corresponding to the image to be processed; when the feature points at the previous moment do not exist, 68 feature point detection (68 key point detection) is performed to obtain the facial feature point coordinates (facial key point coordinates) corresponding to the image to be processed.

[0132] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0133] In one embodiment, Figure 7 As shown, a device for acquiring coordinates of facial feature points is provided, including: an acquisition module 702, a detection module 704, a mapping module 706, a calculation module 708, a first processing module 710 and a second processing module 712, wherein:

[0134] An acquisition module 702 is used to acquire an image to be processed;

[0135] A detection module 704 is used to perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0136] A mapping module 706 is used to perform feature point mapping on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters;

[0137] A calculation module 708, used to calculate the face rotation angle corresponding to the image to be processed according to the 2D face parameters;

[0138] The first processing module 710 is used to determine a feature point detection method according to the face rotation angle, and determine corresponding feature points to be detected according to the feature point detection method and a preset feature point detection method-feature point to be detected correspondence relationship;

[0139] The second processing module 712 is used to obtain the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected.

[0140] The above-mentioned facial feature point coordinate acquisition device performs feature point detection on the image to be processed to obtain the 3D face average parameter and the 3D face parameter, performs feature point mapping on the 3D face parameter according to the 3D face average parameter to obtain the 2D face parameter, calculates the face rotation angle corresponding to the image to be processed according to the 2D face parameter, determines the feature point detection method according to the face rotation angle, determines the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-the feature point to be detected correspondence relationship, and obtains the facial feature point coordinates corresponding to the image to be processed according to the feature point to be detected. The whole process can estimate the head posture by analyzing the image to be processed, obtain the accurate face rotation angle, and then use the face rotation angle to determine the feature point detection method, so that the corresponding feature point to be detected can be obtained on the basis of determining the feature point detection method, and the facial feature point coordinates can be obtained according to the feature point to be detected, so as to achieve accurate facial feature point detection.

[0141] In one embodiment, the detection module is also used to perform feature point detection on the processed image according to the trained face average model to obtain 3D face average parameters, and to perform feature point detection on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model.

[0142] In one embodiment, the mapping module is also used to screen the 3D face average parameters according to the 3D face parameters to obtain the target 3D face average parameters, the target 3D face average parameters and the 3D face parameters represent the same facial features, calculate the movement vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, the first expression is an equation describing the correspondence between the 3D face parameters, the target 3D face average parameters and the face rotation angle, the second expression is an equation describing the correspondence between the 3D face parameters, the movement vector and the 2D face parameters, and the 2D face parameters are obtained according to the 3D face parameters, the movement vector, the first expression and the second expression.

[0143] In one embodiment, the calculation module is also used to obtain current 2D feature parameters corresponding to the 2D face parameters, calculate the feature difference between the current 2D feature parameters and each corresponding feature point in the 2D face parameters, and obtain a third expression, the third expression is an equation that describes the corresponding relationship between the feature difference and the change in rotation angle, calculate the change in rotation angle based on the feature difference and the third expression, and obtain the face rotation angle at the previous moment corresponding to the image to be processed, and obtain the face rotation angle corresponding to the image to be processed based on the change in rotation angle and the face rotation angle at the previous moment.

[0144] In one embodiment, the first processing module is further used to determine the head posture according to the rotation angle of the face, and determine the feature point detection method according to the head posture comparison with the preset posture-feature point detection method correspondence relationship.

[0145] In one embodiment, the second processing module is also used to query feature data based on the feature points to be detected. When feature data corresponding to the feature points to be detected are queried, feature point tracking is performed based on the feature data to obtain the coordinates of the facial feature points corresponding to the image to be processed. When feature data corresponding to the feature points to be detected are not queried, feature point detection is performed based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0146] In one embodiment, the second processing module is also used to track feature points based on feature data and a preset tracking algorithm to obtain parameter increments corresponding to the feature points to be detected, calculate feature point values ​​corresponding to the feature points to be detected based on the feature data and the parameter increments, and obtain the coordinates of the facial feature points corresponding to the image to be processed based on the feature point values.

[0147] For the specific definition of the facial feature point coordinate acquisition device, please refer to the definition of the facial feature point coordinate acquisition method above, which will not be repeated here. Each module in the above facial feature point coordinate acquisition device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store images to be processed, feature data, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for obtaining the coordinates of facial feature points is implemented.

[0149] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0151] Get the image to be processed;

[0152] Perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0153] Perform feature point mapping on 3D face parameters according to the average 3D face parameters to obtain 2D face parameters;

[0154] According to the 2D face parameters, calculate the face rotation angle corresponding to the image to be processed;

[0155] Determine a feature point detection method according to the face rotation angle, and determine the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship;

[0156] According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0157] The computer device for acquiring the coordinates of the facial feature points performs feature point detection on the image to be processed to obtain the 3D face average parameters and 3D face parameters, performs feature point mapping on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters, calculates the face rotation angle corresponding to the image to be processed according to the 2D face parameters, determines the feature point detection method according to the face rotation angle, determines the corresponding feature points to be detected according to the feature point detection method and the preset feature point detection method-the correspondence between the feature points to be detected, and obtains the facial feature point coordinates corresponding to the image to be processed according to the feature points to be detected. The whole process can estimate the head posture by analyzing the image to be processed, obtain the accurate face rotation angle, and then use the face rotation angle to determine the feature point detection method, so that the corresponding feature points to be detected can be obtained on the basis of determining the feature point detection method, and the facial feature point coordinates can be obtained according to the feature points to be detected, so as to achieve accurate facial feature point detection.

[0158] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0159] Feature point detection is performed on the processed image according to the trained face average model to obtain 3D face average parameters, and feature point detection is performed on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model.

[0160] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0161] According to the 3D face parameters, the 3D face average parameters are screened to obtain the target 3D face average parameters, and the target 3D face average parameters and the 3D face parameters represent the same facial features;

[0162] Calculate the motion vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, wherein the first expression is an equation describing the corresponding relationship between the 3D face parameters, the target 3D face average parameters, and the face rotation angle, and the second expression is an equation describing the corresponding relationship between the 3D face parameters, the motion vector, and the 2D face parameters;

[0163] According to the 3D face parameters, the movement vector, the first expression and the second expression, the 2D face parameters are obtained.

[0164] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0165] Get the current 2D feature parameters corresponding to the 2D face parameters;

[0166] Calculate the feature difference between the current 2D feature parameter and each corresponding feature point in the 2D face parameter, and obtain a third expression, where the third expression is an equation describing the corresponding relationship between the feature difference and the rotation angle change;

[0167] According to the feature difference and the third expression, the change in the rotation angle is calculated, and the rotation angle of the face at the previous moment corresponding to the image to be processed is obtained;

[0168] According to the rotation angle change and the face rotation angle at the previous moment, the face rotation angle corresponding to the image to be processed is obtained.

[0169] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0170] Determine head posture based on face rotation angle;

[0171] Determine the feature point detection method based on the head posture comparison preset posture-feature point detection method correspondence.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] Perform feature data query based on the feature points to be detected;

[0174] When the feature data corresponding to the feature point to be detected is queried, the feature point is tracked according to the feature data to obtain the coordinates of the facial feature point corresponding to the image to be processed;

[0175] When feature data corresponding to the feature points to be detected are not found, feature point detection is performed based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0176] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0177] Track the feature points according to the feature data and the preset tracking algorithm to obtain the parameter increment corresponding to the feature point to be detected;

[0178] Calculate the feature point value corresponding to the feature point to be detected according to the feature data and parameter increment;

[0179] According to the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0180] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0181] Get the image to be processed;

[0182] Perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters;

[0183] Perform feature point mapping on 3D face parameters according to the average 3D face parameters to obtain 2D face parameters;

[0184] According to the 2D face parameters, calculate the face rotation angle corresponding to the image to be processed;

[0185] Determine a feature point detection method according to the face rotation angle, and determine the corresponding feature point to be detected according to the feature point detection method and the preset feature point detection method-feature point to be detected correspondence relationship;

[0186] According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0187] The above-mentioned facial feature point coordinates are obtained from the storage medium, feature point detection is performed on the image to be processed to obtain the 3D face average parameters and 3D face parameters, feature point mapping is performed on the 3D face parameters according to the 3D face average parameters to obtain 2D face parameters, and the face rotation angle corresponding to the image to be processed is calculated according to the 2D face parameters, and the feature point detection method is determined according to the face rotation angle, and the corresponding feature points to be detected are determined according to the feature point detection method and the preset feature point detection method-the correspondence relationship between the feature points to be detected, and the facial feature point coordinates corresponding to the image to be processed are obtained according to the feature points to be detected. The whole process can estimate the head posture through the analysis of the image to be processed, and obtain the accurate face rotation angle, and then use the face rotation angle to determine the feature point detection method, so that the corresponding feature points to be detected can be obtained on the basis of determining the feature point detection method, and the facial feature point coordinates are obtained according to the feature points to be detected, so as to achieve accurate facial feature point detection.

[0188] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0189] Feature point detection is performed on the processed image according to the trained face average model to obtain 3D face average parameters, and feature point detection is performed on the processed image according to the trained face feature point detection model to obtain 3D face parameters. The trained face average model is obtained by training the initial active shape model.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] According to the 3D face parameters, the 3D face average parameters are screened to obtain the target 3D face average parameters, and the target 3D face average parameters and the 3D face parameters represent the same facial features;

[0192] Calculate the motion vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, wherein the first expression is an equation describing the corresponding relationship between the 3D face parameters, the target 3D face average parameters, and the face rotation angle, and the second expression is an equation describing the corresponding relationship between the 3D face parameters, the motion vector, and the 2D face parameters;

[0193] According to the 3D face parameters, the movement vector, the first expression and the second expression, the 2D face parameters are obtained.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] Get the current 2D feature parameters corresponding to the 2D face parameters;

[0196] Calculate the feature difference between the current 2D feature parameter and each corresponding feature point in the 2D face parameter, and obtain a third expression, where the third expression is an equation describing the corresponding relationship between the feature difference and the rotation angle change;

[0197] According to the feature difference and the third expression, the change in the rotation angle is calculated, and the rotation angle of the face at the previous moment corresponding to the image to be processed is obtained;

[0198] According to the rotation angle change and the face rotation angle at the previous moment, the face rotation angle corresponding to the image to be processed is obtained.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0200] Determine head posture based on face rotation angle;

[0201] Determine the feature point detection method based on the head posture comparison preset posture-feature point detection method correspondence.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0203] Perform feature data query based on the feature points to be detected;

[0204] When the feature data corresponding to the feature point to be detected is queried, the feature point is tracked according to the feature data to obtain the coordinates of the facial feature point corresponding to the image to be processed;

[0205] When feature data corresponding to the feature points to be detected are not found, feature point detection is performed based on the feature points to be detected to obtain the coordinates of the facial feature points corresponding to the image to be processed.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] Track the feature points according to the feature data and the preset tracking algorithm to obtain the parameter increment corresponding to the feature point to be detected;

[0208] Calculate the feature point value corresponding to the feature point to be detected according to the feature data and parameter increment;

[0209] According to the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

[0210] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0211] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0212] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining coordinates of facial feature points, characterized in that: The method comprises: Get the image to be processed; Performing feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters; According to the 3D face parameters, the 3D face average parameters are screened to obtain target 3D face average parameters, wherein the target 3D face average parameters and the 3D face parameters represent the same facial features; Calculating a motion vector between the 3D face parameter and the target 3D face average parameter, and obtaining a first expression and a second expression, wherein the first expression is an equation describing a corresponding relationship between the 3D face parameter, the target 3D face average parameter, and a face rotation angle, and the second expression is an equation describing a corresponding relationship between the 3D face parameter, the motion vector, and a 2D face parameter; Obtaining 2D face parameters according to the 3D face parameters, the movement vector, the first expression, and the second expression; Calculating a face rotation angle corresponding to the image to be processed according to the 2D face parameters; Determine a feature point detection method according to the face rotation angle, and determine corresponding feature points to be detected according to the feature point detection method and a preset feature point detection method-feature point to be detected correspondence relationship; According to the feature points to be detected, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

2. The method according to claim 1, characterized in that The performing feature point detection on the image to be processed to obtain the 3D face average parameter and the 3D face parameter comprises: Feature point detection is performed on the image to be processed according to the trained average face model to obtain 3D average face parameters, and feature point detection is performed on the image to be processed according to the trained feature point detection model to obtain 3D face parameters, and the trained average face model is obtained by training the initial active shape model.

3. The method according to claim 1, characterized in that The calculating, according to the 2D face parameters, a face rotation angle corresponding to the image to be processed comprises: Acquire current 2D feature parameters corresponding to the 2D face parameters, where the current 2D feature parameters refer to parameters obtained by detecting feature points of the image to be processed, and are used to characterize the two-dimensional position of the feature points on the image to be processed; Calculate the feature difference between the current 2D feature parameter and each corresponding feature point in the 2D face parameter, and obtain a third expression, where the third expression is an equation describing the corresponding relationship between the feature difference and the rotation angle change; Calculate the rotation angle change according to the feature difference and the third expression, and obtain the face rotation angle at the previous moment corresponding to the image to be processed; The face rotation angle corresponding to the image to be processed is obtained according to the rotation angle change and the face rotation angle at the previous moment.

4. The method according to claim 1, characterized in that: The feature point detection method determined according to the face rotation angle includes: Determining the head posture according to the rotation angle of the face; The feature point detection method is determined according to the head posture comparison with the preset posture-feature point detection method correspondence relationship.

5. The method according to claim 1, characterized in that The step of obtaining the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected comprises: Performing feature data query according to the feature points to be detected; When the feature data corresponding to the feature point to be detected is queried, the feature point is tracked according to the feature data to obtain the coordinates of the facial feature point corresponding to the image to be processed; When feature data corresponding to the feature points to be detected are not found, feature point detection is performed based on the feature points to be detected to obtain coordinates of facial feature points corresponding to the image to be processed.

6. The method according to claim 5, characterized in that Tracking feature points according to the feature data to obtain the coordinates of facial feature points corresponding to the image to be processed includes: Tracking feature points according to the feature data and a preset tracking algorithm to obtain parameter increments corresponding to the feature points to be detected; Calculating a feature point value corresponding to the feature point to be detected according to the feature data and the parameter increment; According to the feature point values, the coordinates of the facial feature points corresponding to the image to be processed are obtained.

7. A device for acquiring coordinates of facial feature points, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed; A detection module, used to perform feature point detection on the image to be processed to obtain 3D face average parameters and 3D face parameters; a mapping module, configured to screen the 3D face average parameters according to the 3D face parameters to obtain target 3D face average parameters, wherein the target 3D face average parameters and the 3D face parameters represent the same facial features; calculate a movement vector between the 3D face parameters and the target 3D face average parameters, and obtain a first expression and a second expression, wherein the first expression is an equation describing a corresponding relationship between the 3D face parameters, the target 3D face average parameters, and a face rotation angle, and the second expression is an equation describing a corresponding relationship between the 3D face parameters, the movement vector, and 2D face parameters; and obtain 2D face parameters according to the 3D face parameters, the movement vector, the first expression, and the second expression; A calculation module, used for calculating the face rotation angle corresponding to the image to be processed according to the 2D face parameters; A first processing module, configured to determine a feature point detection method according to the face rotation angle, and determine corresponding feature points to be detected according to the feature point detection method and a preset feature point detection method-feature point to be detected correspondence relationship; The second processing module is used to obtain the coordinates of the facial feature points corresponding to the image to be processed according to the feature points to be detected.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.