Training Method and Device for Facial Keypoint Detection Model

Abstract

An embodiment of the present application provides a training method and device for a face key point detection model. In this method, for each training sample in the training sample set of the face key point detection model, the face key point detection model is used to perform face key point localization on the training sample to obtain the predicted positions of at least one face key point in the training sample. For each face key point, the position error between the predicted position and the true position of the face key point is calculated, and the loss function value corresponding to the face key point is determined according to the position error; when the loss function value of the face key point detection model is lower than the set threshold, it is determined that the face key point detection model meets the training end condition. Thereby, the performance of the trained face key point detection model can be effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of the Internet. Specifically, the embodiments relate to a method and device for training a face key point detection model. Background Art

[0002] With the rapid development of artificial intelligence technology, neural networks and machine learning models have been widely applied in various fields, bringing convenience to people's lives and work. Exemplarily, applying a machine learning model can efficiently identify the user identity corresponding to face information, so as to complete operations such as payment or attendance check-in.

[0003] In addition, face key point detection is a key step in many face applications. As a basic module, it is widely used in face alignment, face recognition, three-dimensional face reconstruction, and expression analysis and other face attribute analyses.

[0004] In recent years, compared with traditional methods, the face key point detection method based on deep learning has greatly improved the detection effect. However, there are still some defects in the performance of the existing face key point detection models based on deep learning. For example, the performance in terms of the convergence speed and robustness of the model is still not ideal, etc. Summary of the Invention

[0005] In view of the above problems, the embodiments of the present application provide a method and device for training a face key point detection model, so as to at least solve the problem that the performance of the current face key point detection model is poor.

[0006] According to one aspect of the embodiments of the present application, a method for training a face key point detection model is provided, including: for each training sample in the training sample set of the face key point detection model, using the face key point detection model to perform face key point positioning on the training sample to obtain the predicted position of at least one face key point in the training sample, where the training sample is marked with the true position of the face key point; for each of the face key points, calculating the position error between the predicted position and the true position of the face key point, and determining the loss function value corresponding to the face key point according to the position error; calculating the loss function value of the face key point detection model according to the loss function values respectively corresponding to each face key point in the training sample set; when the loss function value of the face key point detection model is lower than a set threshold, determining that the face key point detection model meets the training end condition.

[0007] According to another aspect of the embodiments of the present application, there is also provided a training device for a face key point detection model, including: a model prediction unit configured to perform face key point localization on each training sample in the training sample set of the face key point detection model by using the face key point detection model to obtain the predicted positions of at least one face key point in the training sample, wherein the training sample is marked with the true positions of the face key points; a key point loss calculation unit configured to calculate the position error between the predicted position and the true position of each face key point and determine the loss function value corresponding to the face key point according to the position error; a model loss calculation unit configured to calculate the loss function value of the face key point detection model according to the loss function values respectively corresponding to each face key point in the training sample set; a model convergence determination unit configured to determine that the face key point detection model meets the training end condition when the loss function value of the face key point detection model is lower than a set threshold.

[0008] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: at least one processor; and a memory storing instructions, when the instructions are executed by the at least one processor, causing the at least one processor to execute the training method of the face key point detection model as described above.

[0009] According to another aspect of the embodiments of the present application, there is also provided a machine-readable storage medium storing executable instructions, the instructions when executed causing the machine to execute the training method of the face key point detection model as described above.

[0010] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects:

[0011] For each training sample in the training sample set of the face key point detection model, calculate the position error between the predicted position and the true position of each face key point in the training sample respectively, and determine the loss function value corresponding to the face key point according to the position error, and then determine whether the training end condition (or convergence condition) is met through the loss function values of each face key point in the training sample set. Thus, comprehensively considering the loss function values of each face key point in the training sample set to determine the loss of the model and using this to identify whether the model converges can ensure that the face key point detection model obtained after training has high performance. Description of the Drawings

[0012] By referring to the following drawings, a further understanding of the essence and advantages of the content of the embodiments of the present application can be achieved. In the drawings, similar components or features may have the same reference numerals. The drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the application, together with the following specific implementation manners, for explaining the embodiments of the present application, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0013] Figure 1 A flowchart showing an example of a method for training a face key point detection model according to an embodiment of the present application is shown;

[0014] Figure 2 A flowchart showing an example of calculating the loss function value of face key points according to an embodiment of the present application is shown;

[0015] Figure 3 A flowchart showing an example of determining the loss function value corresponding to face key points according to an embodiment of the present application is shown;

[0016] Figure 4 A flowchart showing an example of determining whether a face key point detection model meets the training end condition according to an embodiment of the present application is shown;

[0017] Figure 5 A flowchart showing an example of expanding a training sample set according to an embodiment of the present application is shown;

[0018] Figure 6 A structural block diagram showing an example of a training device for a face key point detection model according to an embodiment of the present application is shown; and

[0019] Figure 7 A hardware structure diagram showing an example of an electronic device for training a face key point detection model according to an embodiment of the present application is shown. Detailed Description of the Embodiments

[0020] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0021] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0022] It should also be understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0023] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrases "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context. Additionally, in the description of the specification and appended claims of this application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0024] Reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0025] Figure 1 A flowchart showing an example of a method for training a face key point detection model according to an embodiment of this application is shown.

[0026] As Figure 1 In the process 100 shown, in step 110, for each training sample in the training sample set of the face key point detection model, the face key point detection model is used to perform face key point localization on the training sample to obtain the predicted position of at least one face key point in the training sample. Here, the training sample is marked with the true position of the face key point. For example, the true position of each face key point can be manually marked on the face image in advance to generate the corresponding training sample.

[0027] In step 120, for each face key point, the position error between the predicted position and the true position of the face key point is calculated, and the loss function value corresponding to the face key point is determined according to the position error. Exemplarily, the position error can be substituted into the loss function pre-configured in the model to calculate the corresponding loss function value.

[0028] In step 130, according to the loss function values corresponding to each facial key point in the training sample set, calculate the loss function value of the facial key point detection model. Exemplarily, the loss function values of all facial key points under each training sample in the training sample set can be superimposed to obtain the loss function value of the facial key point detection model.

[0029] In step 140, when the loss function value of the facial key point detection model is lower than the set threshold, it is determined that the facial key point detection model meets the training end condition. Here, the obtained loss function value of the facial key point detection model can be compared with the set threshold, and when the loss function value of the facial key point detection model is less than the set threshold, it can be determined that the model has converged during training, and at this time, the training end condition is met. In addition, when the loss function value of the facial key point detection model is greater than or equal to the set threshold, the model needs to be continuously trained.

[0030] Figure 2 A flowchart showing an example of calculating the loss function value of facial key points according to an embodiment of the present application is shown.

[0031] As Figure 2 shown, in step 210, the position error is matched with each preset position error segment to determine the corresponding set loss function. Here, a preset number of preset position error segments and a preset number of set loss functions uniquely corresponding to the preset position error segments are configured in the facial key point detection model. <00>

[0032] Exemplarily, a training sample set with n training samples {S1, S2,..., Sn} can be prepared in advance, and these training samples (e.g., facial images) are used to perform training operations on the facial key point detection model. When the facial key point detection model uses sample S1 for training, the facial key point detection model can locate the facial key points (e.g., facial contour points) in training sample S1 to obtain the predicted positions of each facial key point in S1. Then, the predicted positions of each facial key point (E1, E2,..., En) in sample S1 are respectively compared with the corresponding true positions, so as to obtain the position errors (x1, x2,..., x n ) of each facial key point (E1, E2,..., En) in sample S1.

[0033] Here, a plurality of set loss functions uniquely corresponding to a plurality of preset position error segments are configured in the facial key point detection model. For example, the set loss function f1 for the position error segment [s1, s m , and the set loss function for the position error segment [s m , sn The set loss function f2 for the position error is segmented according to [s n , s p The set loss function f3, and so on. Furthermore, the position error of the obtained facial key points is matched with the corresponding segmented position error to determine the loss function applicable to the facial key points, and the corresponding loss function value is calculated. For example, when x1 is within the segment [s1, s m , the set loss function f1 is used to calculate the loss function value corresponding to the key point E1 of the sample S1.

[0034] In step 220, according to the position error and the determined set loss function, the loss function value corresponding to the facial key point is calculated. Exemplarily, the position error can be substituted into the appropriate loss function to calculate the corresponding loss function value.

[0035] In the embodiment of the present application, there are a preset number (for example, multiple) of set loss functions corresponding to preset position error segments in the facial key point detection model, and adaptive loss functions are respectively adopted for different position errors, which can accelerate the convergence speed of the model and ensure the robustness of the obtained facial key point detection model.

[0036] In some embodiments, the set loss functions corresponding to different position error segments in the facial key point detection model can also be configured or defined according to product or service requirements, so as to meet the requirements of various products and application scenarios.

[0037] In the current related technologies, the facial key point detection model generally adopts a single loss function and cannot be applied to all stages of model training, resulting in poor model performance. Exemplarily, in the early stage of model training, there is a problem of unstable training (L2 loss) caused by a large error (the difference between the predicted value and the true value). In the later stage of model training, there is a problem that the error is too small, resulting in the model network fluctuating near the stable value, and it is difficult for the model to converge to a higher accuracy (L1 loss), the interference of outlier samples and difficult samples (L2 loss), and the model network is mainly dominated by large errors and is not friendly to small errors, resulting in the problem that it cannot converge to the optimal value (smooth L1Loss).

[0038] In view of this, in some examples of the embodiments of the present application, among the preset number or multiple set loss functions configured in the face key point detection model, the modulus of the gradient of the set loss function regarding the position error in one or more preset position error segments corresponding to a relatively large value can be relatively large. Specifically, the set loss function (i.e., f(x)) is a functional formula related to the position error (i.e., x), and the modulus of the gradient can represent the amplitude of the change of the loss function value f(x) as x changes. Thus, when the position error is in a relatively large preset position error segment, the loss function can quickly reduce the error and enable the model to converge quickly.

[0039] In some examples of the embodiments of the present application, for a preset position error segment with a particularly small value, such as one or more consecutive preset position error segments corresponding to the smallest value (for example, the error segment including |x| = 0), the face key points may complete training in this error segment. At this time, the gradient of the set loss function corresponding to this error segment can be an inverse proportional function related to the position error x (i.e., k / x). Thus, when approaching convergence during the model iteration process, the smaller the error, the larger the gradient, so that when integrating the contributions of all key points, the gradient will mainly be dominated by small errors, which can enable the network to converge to a higher accuracy.

[0040] Furthermore, a bias amount for the position error is provided in the set loss function corresponding to one or more consecutive preset position error segments corresponding to the smallest value. In this way, it can prevent the loss function from being overly affected by small errors, avoid the model network from fluctuating near the stable value, and thus accelerate the model convergence.

[0041] It should be understood that the training quality of the face key point detection model largely depends on the design of the loss function. In the embodiments of the present application, multiple loss functions are configured for the face key point detection model and can be adaptively adjusted according to the size of the error.

[0042] It should be noted that in the face key point task, there are more frontal face images and fewer large-pose face images. In addition, when the pose is relatively large, the visible range of the face is reduced, and the difficulty of key point positioning will increase.

[0043] In view of this, Figure 3 FIG. shows a flowchart of an example for determining the loss function value corresponding to the face key points according to the embodiments of the present application.

[0044] As Figure 3 shown, in step 310, based on the predicted positions of each face key point of the training sample, the corresponding face pose angle is calculated. Here, the face pose angle can be calculated through various pose calculation methods, such as calculating according to the relative positions of each face key point.

[0045] In step 320, based on the position error and the calculated face pose angles, determine the loss function values corresponding to the face key points. Exemplarily, the loss functions (e.g., each set loss function) configured in the face key point detection model are functional expressions related to the face pose angles (e.g., the face turning angles in multiple dimensions). Therefore, after obtaining the predicted positions of at least one face key point in the training sample, the corresponding face pose angles can also be calculated based on the predicted positions of each face key point in the training sample. Furthermore, after selecting the corresponding set loss function, the face pose angles can be substituted to obtain the corresponding loss function values.

[0046] Thus, by considering the geometric information of the face pose angles in the loss penalty, the model training can pay more attention to the face samples with larger poses, thereby improving the training quality of the face key point detection model and enabling better detection of the face key points in the large pose face images.

[0047] In some examples of the embodiments of the present application, the expressions of multiple set loss functions AdaptiveLose(x) in the face key point detection model can be as shown in the following formula (1):

[0048]

[0049] where x represents the position error between the predicted position and the true position, θ represents the face pose angle, C is a constant (e.g., it can take 2.2), K is 3, and θ k represents the face pitch angle, roll angle or yaw angle.

[0050] In this way, when the error is small (x < 10), since the gradient of the ln(x) function is 1 / x, the smaller the error, the larger the gradient value. When integrating the contributions of all key points, the gradient will mainly be dominated by small errors, which can make the model network converge to a higher precision; and there is a bias in the loss function when x < 10. The bias amount ln1 in ln(1 + 0.5*|x|) can prevent the loss function from being overly affected by small errors. In addition, when the error is large (x > 10), since the gradient of |x| is ±1, the loss function can relatively quickly reduce the error and make the network converge to a stable value, and |x|, that is, the L1 loss function is more robust to outliers.

[0051] As described above, in the face key point task, there are many frontal face image samples and few large pose face image samples. In addition, when the pose is large, the visible range of the face shrinks, which increases the difficulty of key point localization. Therefore, in Equation (1), the pose angle θ is considered in the loss penalty to help solve the training quality problem. In this way, when the pose angle θ is large, (1 - cosθ) will also increase, that is, more penalties are imposed on the loss values of large pose samples. Thus, it can help to deal with the problem of data imbalance of large pose faces and enable the network to learn geometric information to more accurately locate the face key points in large pose face images.

[0052] Figure 4 FIG. shows a flowchart of an example for determining whether a face key point detection model meets the training end condition according to an embodiment of the present application. Here, the face key points may include overall face key points (which can be used to fit the overall structure of the face) and local face key points (for example, key points of facial features, contours, etc.). In addition, the face key point detection model is configured with a first loss weight for the overall face key points and a second loss weight for the local face key points.

[0053] As Figure 4 shown, in step 410, according to the loss function value and the first loss weight corresponding to the overall face key points in the training sample set, together with the loss function value and the second loss weight corresponding to the local face key points, calculate the loss function value of the face key point detection model.

[0054] Exemplarily, the loss function values of all the overall face key points in the training sample set can be summed up and calibrated using the first loss weight. In addition, the loss function values of all the local face key points in the training sample set can be summed up and calibrated using the second loss weight. Then, the two are aggregated to obtain the loss function value of the face key point detection model.

[0055] In step 420, determine whether the loss function value of the face key point detection model is less than a set threshold. Here, the set threshold can be defined according to the requirements of the face detection service.

[0056] When the judgment result in step 420 indicates that the loss function value of the face key point detection model is less than the set threshold, jump to step 431. When the judgment result in step 420 indicates that the loss function value of the face key point detection model is greater than or equal to the set threshold, jump to step 433.

[0057] In step 431, determine that the face key point detection model meets the training end condition. At this time, the model completes training and converges.

[0058] In step 433, continue to perform the training operation on the face key point detection model.

[0059] In the embodiment of the present application, the face key point detection model can detect the overall key points and local key points of the face, not only fitting the overall structure of the face key points, but also being able to precisely locate the key points of each local part of the face, having a wider application scenario. In addition, the center of gravity of the key points concerned by the face key point detection model can be adjusted by setting the first loss weight and the second loss weight. For example, when it is necessary to focus on the detection performance of the local key point features of the face, a relatively high second loss weight can be set.

[0060] Combined with the example in formula (1) above, the calculation method of the total loss function of the face key point detection model in the embodiment of the present application is as follows:

[0061]

[0062]

[0063]

[0064]

[0065]

[0066] loss = α·lose face +β·(lose leye +lose reye +lose mouth +lose profile ) Formula (7)

[0067] Where, pred i and gth i respectively represent the predicted value and the true value of the i-th key point. lose face , lose leye , lose leye , lose profile , lose mouth and loss respectively represent the loss (loss function) values of 68 key points of the face, 6 key points of the left eye, 6 key points of the right eye, 20 key points of the mouth, and 17 key points of the face contour. Here, α and β respectively represent the loss weights of the overall key points of the face and the local key points of the face. For example, α = 1 and β = 1.1. In this way, the face key point detection model can not only fit the overall structure of the face key points, but also further optimize the key point detection of the facial features and contour.

[0068] It should be noted that in the training sample set, there are generally more complete frontal face image samples, while the number of samples with occlusion, exaggerated expressions, or large poses (e.g., tilted faces) is relatively small. As a result, the obtained face key point detection model often has difficulty processing difficult samples with occlusion, exaggerated expressions, or large poses.

[0069] In view of this, the embodiments of the present application also propose to construct new difficult samples and expand them into the training sample set to balance the sample data in the training sample set and improve the performance of the face key point detection model, so that the face key point detection model can also have a better detection effect on face images with occlusion, exaggerated expressions, or large poses.

[0070] Figure 5 The flowchart shows an example of expanding the training sample set according to an embodiment of the present application.

[0071] As Figure 5 shown, in step 510, a rotation operation within a corresponding set angle range is performed on the training sample to generate a first new training sample corresponding to the training sample.

[0072] Exemplarily, by randomly rotating the training set within a set angle range (e.g., -25° to 25° in the plane direction). In this way, the diversity of sample poses in the training sample set can be increased.

[0073] In step 520, an occluder is superimposed at the predicted positions of one or more face key points in the training sample to generate a second new training sample.

[0074] Exemplarily, an occluder of random size can be added to the training sample at random positions on the facial features of the face (e.g., the positions of the facial features can be determined by key point localization) with a certain probability (e.g., 40%). In some examples, the occluder can be randomly selected from 1000 small objects crawled from the network. Thus, through random occlusion, the occluded samples in the training sample set are enriched, and the relative balance of the training sample set is achieved.

[0075] In step 530, when the predicted positions of multiple face key points in the training sample satisfy a preset position distribution condition, the training sample is copied to generate a corresponding third new training sample.

[0076] Here, the positions of the face key points that meet the preset position distribution condition can match the positions of the respective face key points distributed in a face image with an exaggerated expression. In this way, by appropriately copying such samples, there can be a relatively large number of training samples corresponding to exaggerated expressions in the training sample set.

[0077] It should be noted that steps 510 to 530 can be executed in whole or in part.

[0078] In step 540, one or more of the first new training sample, the second new training sample, and the third new training sample are augmented into the training sample set.

[0079] Through the embodiments of the present application, face samples with occlusion, exaggerated expressions, and large poses are supplemented in the training sample set, which can solve the problem of data imbalance and improve the performance of the face key point detection model.

[0080] Figure 6 The structural block diagram of an example of the training device of the face key point detection model according to the embodiments of the present application is shown.

[0081] As Figure 6 shown, the training device 600 of the face key point detection model includes a model prediction unit 610, a key point loss calculation unit 620, a model loss calculation unit 630, a model convergence determination unit 640, a training sample set supplement unit 650, and a face pose angle calculation unit 660.

[0082] The model prediction unit 610 is configured to perform face key point localization on each training sample in the training sample set of the face key point detection model by using the face key point detection model to obtain the predicted positions of at least one face key point in the training sample, wherein the training sample is marked with the true positions of the face key points. For more details about the model prediction unit 610, reference can be made to the description of step 110 in the above reference Figure 1 in.

[0083] The key point loss calculation unit 620 is configured to calculate the position error between the predicted position and the true position of each face key point, and determine the loss function value corresponding to the face key point according to the position error. For more details about the key point loss calculation unit 620, reference can be made to the description of step 120 in the above reference Figure 1 in.

[0084] In some examples of the embodiments of the present application, the face key point detection model is configured with a preset number of preset position error segments and a preset number of set loss functions respectively and uniquely corresponding to the preset position error segments. Correspondingly, the key point loss calculation unit 620 includes a loss function determination module (not shown) and a loss function value calculation module (not shown). The loss function determination module is configured to match the position error with each preset position error segment to determine the corresponding set loss function, and the loss function value calculation module is configured to calculate the loss function value corresponding to the face key point according to the position error and the determined set loss function.

[0085] The model loss calculation unit 630 is configured to calculate the loss function value of the face key point detection model according to the loss function values corresponding to each face key point in the training sample set. For more details about the model loss calculation unit 630, reference can be made to the description of step 130 in the above reference Figure 1 in

[0086] The model convergence determination unit 640 is configured to determine that the face key point detection model meets the training end condition when the loss function value of the face key point detection model is lower than a set threshold. For more details about the model convergence determination unit 640, reference can be made to the description of step 140 in the above reference Figure 1 in

[0087] In some embodiments, the face key points include overall face key points and local face key points, and the face key point detection model is configured with a first loss weight for the overall face key points and a second loss weight for the local face key points. At this time, the model loss calculation unit 630 is further configured to calculate the loss function value of the face key point detection model based on the loss function value and the first loss weight corresponding to the overall face key points in the training sample set, together with the loss function value and the second loss weight corresponding to the local face key points; and, when the loss function value of the face key point detection model is less than the set threshold, determine that the face key point detection model meets the training end condition.

[0088] The training sample set supplement unit 650 is configured to expand one or more of the first new training sample, the second new training sample, and the third new training sample to the training sample set; wherein, the first new training sample is generated by performing a rotation operation within a corresponding set angle range on the training sample to generate a first new training sample corresponding to the training sample; the second new training sample is generated by superimposing an occluder at the predicted positions of one or more face key points in the training sample to generate a second new training sample; the third new training sample is generated by copying the training sample when the predicted positions of multiple face key points in the training sample meet a preset position distribution condition. For more details about the training sample set supplement unit 650, reference can be made to the process description in the above reference Figure 5 in

[0089] The face pose angle calculation unit 660 is configured to calculate the corresponding face pose angle based on the predicted positions of each face key point of the training sample. Correspondingly, the key point loss calculation unit 620 is configured to determine the loss function value corresponding to the face key point according to the position error and the calculated face pose angle.

[0090] It should be noted that some units in the training device 600 of the face key point detection model described above are non-essential or optional in some application scenarios. Exemplarily, in some embodiments, the training sample set supplementing unit 650 and the face pose angle calculation unit 660 may not be retained.

[0091] In some embodiments, among the multiple set loss functions, the gradients of the set loss functions corresponding to the smallest one or consecutive multiple preset position error segments in the preset number of preset position error segments are inverse proportional functions related to the position error.

[0092] In some embodiments, a bias amount for the position error is provided in the set loss functions corresponding to the smallest one or consecutive multiple preset position error segments in the preset number of preset position error segments.

[0093] In some embodiments, the expression of the preset number of set loss functions AdaptiveLose(x) is:

[0094]

[0095] where x represents the position error between the predicted position and the true position, θ represents the face pose angle, C is a constant, K is 3, and θ k represents the face pitch angle, roll angle, or yaw angle.

[0096] As described above with reference to Figures 1 to 6 , embodiments of the training method and device for the face key point detection model according to the embodiments of the present application have been described. The details mentioned in the above description of the method embodiments also apply to the embodiments of the device of the present application. The above-mentioned training device for the face key point detection model can be implemented by hardware, or by software, or by a combination of hardware and software.

[0097] Figure 7 FIG. shows a hardware structure diagram of an example of an electronic device 700 for training a face key point detection model according to an embodiment of the present application. As Figure 7 shown, the electronic device 700 may include at least one processor 710, a memory (such as a non-volatile memory) 720, a memory 730, and a communication interface 740, and at least one processor 710, the memory 720, the memory 730, and the communication interface 740 are connected together via a bus 760. At least one processor 710 executes at least one computer-readable instruction stored or encoded in the memory (i.e., the above-mentioned elements implemented in software form).

[0098] In one embodiment, computer-executable instructions are stored in a memory, which, when executed, cause at least one processor 710 to: for each training sample in a training sample set of a facial key point detection model, use the facial key point detection model to locate the facial key points of the training sample to obtain a predicted position of at least one facial key point in the training sample, wherein the training sample is marked with the true position of the facial key point; for each facial key point, calculate the position error between the predicted position and the true position of the facial key point, and determine the loss function value corresponding to the facial key point based on the position error; calculate the loss function value of the facial key point detection model based on the loss function value corresponding to each facial key point in the training sample set; when the loss function value of the facial key point detection model is lower than a set threshold, determine that the facial key point detection model meets the training end condition.

[0099] It should be understood that the computer executable instructions stored in the memory 720, when executed, cause the at least one processor 710 to perform the above combined operations in various embodiments of the present application. Figures 1 - 5 Describes the various operations and functions.

[0100] In this specification, the electronic device 700 may include but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile electronic devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable electronic devices, consumer electronic devices, etc.

[0101] According to one embodiment, a program product such as a machine-readable medium is provided. The machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in software form), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of the present application. Figures 1 - 6 Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes for implementing the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can be enabled to read and execute the instructions stored in the readable storage medium.

[0102] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of the present invention.

[0103] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROM. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0104] Those skilled in the art should understand that various modifications and variations can be made to the above-described embodiments of the application without departing from the essence of the invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

[0105] It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structures described in the above embodiments can be physical structures or logical structures. That is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities separately, or some components in multiple independent devices may be jointly implemented.

[0106] In the above embodiments, the hardware units or modules can be implemented mechanically or electrically. For example, a hardware unit, module, or processor can include permanent dedicated circuits or logic (such as a dedicated processor, FPGA, or ASIC) to perform corresponding operations. The hardware unit or processor can also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which can be temporarily set by software to perform corresponding operations. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0107] The specific embodiments described above in conjunction with the accompanying drawings describe exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the protection scope of the claims. The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration", and does not mean "preferred" or "advantageous" compared to other embodiments. For the purpose of providing an understanding of the described technology, the specific embodiments include specific details. However, these technologies can be implemented without these specific details. In some instances, well-known structures and devices are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0108] The foregoing description of the subject matter of this application is provided to enable any person of ordinary skill in the art to make or use this application. Various modifications to this application will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of this application. Therefore, this application is not limited to the examples and designs described herein, but is consistent with the broadest scope that conforms to the principles and novel features of this application.

Claims

1. A training method for a facial key point detection model, comprising: For each training sample in a training sample set of a facial key point detection model, using the facial key point detection model to locate facial key points of the training sample to obtain a predicted position of at least one facial key point in the training sample, wherein the training sample is marked with the actual position of the facial key point; Calculating corresponding facial pose angles based on the predicted positions of the facial key points of the training sample; For each of the facial key points, the position error between the predicted position of the facial key point and the actual position is calculated, and the loss function value corresponding to the facial key point is determined based on the position error and the calculated facial posture angle, including: matching the position error with each preset position error segment to determine the corresponding set loss function; substituting the position error into the determined set loss function to calculate the loss function value corresponding to the facial key point; wherein the facial key point detection model is configured with a preset number of preset position error segments and a preset number of set loss functions that uniquely correspond to the preset position error segments; wherein the expression of multiple set loss functions AdaptiveLose(x) is shown below: Where x represents the position error between the predicted position and the true position, θ represents the face pose angle, C is a constant, K is 3, and θ k Indicates the pitch angle, roll angle or yaw angle of the face; the facial key points include the overall facial key points and the local facial key points, and the facial key point detection model is configured with a first loss weight for the overall facial key points and a second loss weight for the local facial key points; Calculating the loss function value of the facial key point detection model according to the loss function value corresponding to each facial key point in the training sample set, specifically comprising: calculating the loss function value of the facial key point detection model according to the loss function value and the first loss weight corresponding to the overall facial key point in the training sample set, together with the loss function value and the second loss weight corresponding to the local facial key point; When the loss function value of the facial key point detection model is lower than a set threshold, it is determined that the facial key point detection model meets the training end condition.

2. The method for training a facial key point detection model according to claim 1, further comprising: Adding one or more of a first new training sample, a second new training sample, and a third new training sample to the training sample set; The first new training sample is generated by performing a rotation operation corresponding to a set angle range on the training sample to generate a first new training sample corresponding to the training sample; The second new training sample is generated by superimposing an occluder at the predicted position of one or more facial key points in the training sample to generate the second new training sample; The third new training sample is generated in the following manner: when the predicted positions of the plurality of facial key points in the training sample meet a preset position distribution condition, the training sample is copied to generate a corresponding third new training sample.

3. A training device for a facial key point detection model, comprising: a model prediction unit configured to, for each training sample in a training sample set of a facial key point detection model, locate facial key points on the training sample using the facial key point detection model to obtain a predicted position of at least one facial key point in the training sample, wherein the training sample is marked with the actual position of the facial key point; a face pose angle calculation unit configured to calculate corresponding face pose angles based on the predicted positions of the facial key points of the training sample; A key point loss calculation unit is configured to calculate the position error between the predicted position and the actual position of each facial key point, and determine the loss function value corresponding to the facial key point based on the position error and the calculated facial posture angle, including: a loss function determination module, configured to match the position error with each preset position error segment to determine the corresponding set loss function; a loss function value calculation module, configured to substitute the position error into the determined set loss function to calculate the loss function value corresponding to the facial key point; wherein the facial key point detection model is configured with a preset number of preset position error segments and a preset number of set loss functions that uniquely correspond to the preset position error segments; wherein the expression of multiple set loss functions AdaptiveLose(x) is shown below: Where x represents the position error between the predicted position and the true position, θ represents the face pose angle, C is a constant, K is 3, and θ k Indicates the pitch angle, roll angle or yaw angle of the face; the facial key points include the overall facial key points and the local facial key points, and the facial key point detection model is configured with a first loss weight for the overall facial key points and a second loss weight for the local facial key points; a model loss calculation unit, configured to calculate the loss function value of the facial key point detection model according to the loss function value corresponding to each facial key point in the training sample set, specifically comprising: calculating the loss function value of the facial key point detection model according to the loss function value and the first loss weight corresponding to the overall facial key point in the training sample set, together with the loss function value and the second loss weight corresponding to the local facial key point; The model convergence determination unit is configured to determine that the face key point detection model meets the training end condition when the loss function value of the face key point detection model is lower than a set threshold.

4. An electronic device comprising: at least one processor; as well as A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to perform the method according to claim 1 or 2.

5. A machine-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the method according to claim 1 or 2.

Images