Face key point acquisition result anomaly detection method and system

Abstract

The embodiment of the invention discloses a face key point acquisition result anomaly detection method, system and device and a storage medium. According to the technical scheme provided by the embodiment of the invention, the method comprises the steps: obtaining a first face key point set collected in a current frame of face image and a second face key point set collected in a previous frame of face image; calculating a relative deviation value of each pre-divided face region based on the first face key point set and the second face key point set, wherein the relative deviation value represents a displacement condition of the face key point of the corresponding face region between the previous frame of face image and the current frame of face image; and calculating a key point anomaly index based on the relative deviation value, determining a comparison result of the key point anomaly index and a set index threshold, and determining a face key point anomaly detection result of the current frame of face image based on the comparison result. By adopting the above technical means, efficient and accurate anomaly detection can be carried out on the face key point acquisition result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method and system for detecting anomalies in the results of face key point collection. Background Art

[0002] Currently, in various face image processing scenarios, the application of face key point collection technology is often involved. Face key point collection is a task for locating specific feature points on a human face. For example, in live interactive scenarios such as beauty filters, makeup, and special effects, face key point collection is an essential and pre - requisite signal detection step. In common face key point collection schemes, the input is generally a three - channel RGB image, and the output is the row and column coordinate values of a specific number (such as 68, 96, 106, 240) of face key points in the original image. The face key point collection results can provide the positions of the user's face feature points in the picture to support subsequent processing. Most common face key point collection schemes are implemented based on deep learning models. Since the data distribution of the dataset used for model training largely determines the overall effect of the deep learning model, the deep learning model is prone to poor performance in some business scenarios.

[0003] In related technologies, in order to optimize the accuracy of face key point collection, it is usually necessary to manually screen abnormal face key point detection results to improve the face key point collection model based on the collected anomaly detection results, thereby improving the face key point collection effect. However, most of the related anomaly detection schemes for face key point collection results adopt the method of manual screening. The time and labor costs invested in manual screening are high, the anomaly detection method is relatively inefficient, and it is difficult to cover all scenarios with poor anomaly detection performance in real - time, resulting in poor improvement effects of the face key point collection model. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, device, and storage medium for detecting anomalies in the results of face key point collection, which can efficiently and accurately detect anomalies in the results of face key point collection, reduce the cost of anomaly detection of face key point collection results, and solve the technical problems of low efficiency and high cost investment in the anomaly detection process of face key point collection results.

[0005] In a first aspect, the embodiments of the present application provide a method for detecting anomalies in the results of face key point collection, including:

[0006] Obtain a first set of face key points collected in the current - frame face image and a second set of face key points collected in the previous - frame face image;

[0007] Calculate the relative offset values of each pre-divided face region based on the first set of face key points and the second set of face key points. The relative offset value characterizes the displacement of the face key points in the corresponding face region between the previous frame of face image and the current frame of face image;

[0008] Calculate the key point anomaly index based on the relative offset value, determine the comparison result between the key point anomaly index and the set index threshold, and determine the face key point anomaly detection result of the current frame of face image based on the comparison result.

[0009] In a second aspect, an embodiment of the present application provides an abnormal detection system for face key point acquisition results, including:

[0010] An acquisition module configured to acquire a first set of face key points collected in the current frame of face image and a second set of face key points collected in the previous frame of face image;

[0011] An offset calculation module configured to calculate the relative offset values of each pre-divided face region based on the first set of face key points and the second set of face key points. The relative offset value characterizes the displacement of the face key points in the corresponding face region between the previous frame of face image and the current frame of face image;

[0012] An abnormal detection module configured to calculate the key point anomaly index based on the relative offset value, determine the comparison result between the key point anomaly index and the set index threshold, and determine the face key point anomaly detection result of the current frame of face image based on the comparison result.

[0013] In a third aspect, an embodiment of the present application provides an abnormal detection device for face key point acquisition results, including:

[0014] A memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the abnormal detection method for face key point acquisition results as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are configured to execute the abnormal detection method for face key point acquisition results as described in the first aspect when executed by a computer processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which contains instructions that, when running on a computer or a processor, cause the computer or the processor to execute the abnormal detection method for the face key point acquisition result as described in the first aspect.

[0019] In the embodiment of the present application, a first set of face key points collected in the current frame face image and a second set of face key points collected in the previous frame face image are obtained; based on the first set of face key points and the second set of face key points, the relative offset values of each pre-divided face area are calculated, and the relative offset value characterizes the displacement of the face key points in the corresponding face area between the previous frame face image and the current frame face image; based on the relative offset values, a key point abnormality index is calculated, the comparison result between the key point abnormality index and the set index threshold is determined, and based on the comparison result, the abnormal detection result of the face key points in the current frame face image is determined. By adopting the above technical means, through calculating the relative offset values of the face images of two consecutive frames and performing abnormal detection on the face key point acquisition based on the relative offset values, the situation of high abnormal detection cost investment caused by the manual screening method is avoided, so as to realize efficient and accurate abnormal detection of the face key point acquisition result, improve the abnormal detection effect of the face key point acquisition result, further improve the improvement effect of the face key point acquisition model, and improve the accuracy and stability of the face key point acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flowchart of an abnormal detection method for a face key point acquisition result provided by an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of the division of face key point grouping in an embodiment of the present application;

[0022] Figure 3 is a flowchart of calculating the relative offset value in an embodiment of the present application;

[0023] Figure 4 is a flowchart of determining the abnormal detection result in an embodiment of the present application;

[0024] Figure 5 is a schematic structural diagram of an abnormal detection system for a face key point acquisition result provided by an embodiment of the present application;

[0025] Figure 6 is a schematic structural diagram of an abnormal detection device for a face key point acquisition result provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following provides a more detailed description of specific embodiments of this application with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. Additionally, it should be noted that for ease of description, only parts related to this application rather than all content are shown in the drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0027] The abnormal detection method for the face key point acquisition results provided by this application aims to calculate the relative offset value between two consecutive frames of face images and perform abnormal detection of face key point acquisition based on the relative offset value, avoiding the situation of high abnormal detection cost input caused by manual screening methods, so as to achieve efficient and accurate abnormal detection of the face key point acquisition results.

[0028] Face key point detection is a task for locating specific feature points on a human face. In live interaction scenarios such as beauty filters, makeup, and special effects, face key point detection is a necessary and pre - requisite signal detection step, which is used to provide the positions of the user's face feature points in the picture to support subsequent processing. In a common solution for face key point detection currently, its input is generally a three - channel RGB image, and the output is the row and column coordinate values of a specific number (such as 68, 96, 106, 240) of face key points in the original image.

[0029] Deep - learning - based face key point detection is a commonly used solution in the current industry. Since the distribution of data in the training dataset used for training largely determines the overall effect of the deep - learning model, the deep - learning model is prone to poor performance in some business scenarios. Generally, the common approach at this time is to collect data from similar scenarios and supplement it to the training set of the deep - learning model to improve the effect.

[0030] In related technologies, to optimize the accuracy of face key point acquisition, it is usually necessary to manually screen abnormal face key point detection results to improve the face key point acquisition model based on the collected abnormal detection results. The time and labor costs of manual screening are relatively high, and the abnormal detection method is relatively inefficient, making it difficult to cover scenarios where the model performs poorly in real - time and difficult to cover the diversity of users, etc.

[0031] Based on this, a method for detecting anomalies in facial key point collection results according to an embodiment of the present application is provided to solve the technical problems of inefficient and high cost of detecting anomalies in facial key point collection results. For the deep learning facial key point collection model, while it is actually running key point detection, it automatically detects scenes where the key point detection effect is poor. Thus, such scenes can be efficiently and massively collected, added to the model training data set, and the effect of the key point collection model can be continuously optimized.

[0032] Example:

[0033] Figure 1 A flowchart of a method for detecting anomalies in a face key point acquisition result provided in an embodiment of the present application is provided. The method for detecting anomalies in a face key point acquisition result provided in this embodiment can be performed by an anomaly detection device for detecting anomalies in a face key point acquisition result. The anomaly detection device for detecting anomalies in a face key point acquisition result can be implemented by software and / or hardware. The anomaly detection device for detecting anomalies in a face key point acquisition result can be composed of two or more physical entities, or can be composed of one physical entity. Generally speaking, the anomaly detection device for detecting anomalies in a face key point acquisition result can be a computing device such as a server host.

[0034] The following description is made by taking the abnormality detection device of the face key point collection result as an example to perform the abnormality detection method of the face key point collection result. Figure 1 , the abnormality detection method of the face key point collection result specifically includes:

[0035] S110, obtaining a first set of facial key points collected in a current frame of facial image and a second set of facial key points collected in a previous frame of facial image.

[0036] When performing anomaly detection on the results of facial key point collection, the present application automatically detects scenarios where the key point model performs poorly when the key point model is running, determines the corresponding anomaly detection results, and uses the anomaly detection results to specifically expand the data set for training the key point model, and uses the data set to continuously optimize the key point model effect.

[0037] When the key point model is running, the captured face images are continuously input into the key point model, and the face key points are identified based on the key point model, and the output is the row and column coordinate values ​​of a specific number (such as 68, 96, 106, 240) of face key points in the original face image, so as to obtain the face key point set corresponding to a face image. In the process of face key point anomaly detection, the face key point sets of the two frames of face images are obtained to determine whether the key point model is abnormal by comparing the two face key point sets.

[0038] Exemplarily, in application scenarios of key-point models such as video editing, short videos, and live streaming, the key-point model is usually first used to detect key points such as the face contour, eyes, nose, lips, and eyebrows to achieve effects such as beauty makeup and special effect stickers.

[0039] For example, in the application scenario of beauty makeup effects during live streaming, the key-point model is used to detect key points in areas such as eyes, lips, and eyebrows, and virtual eyebrow makeup, eyelash makeup, lip makeup, etc. are added to the user through beauty makeup algorithms. While the key-point model is running, anomaly detection of the key-point model is performed based on the present application. By detecting scenarios of abnormal key-point positions and reporting relevant anomaly detection result data, it is used to expand the training set subsequently to improve the key-point model.

[0040] Specifically, as Figure 2 shown, the set of face key points detected by the key-point model contains a total of 106 key-point coordinates. The set of face key points describes the specific positions of the face contour, eyebrows, eyes, nose, and mouth in the image. These key-point coordinates can be used as the input of beauty and beauty makeup algorithms, and beauty and beauty makeup effects are added to the user's face through relevant algorithms.

[0041] When performing anomaly detection of the key-point model in the present application, the set of face key points of consecutive frames is recorded for subsequent comparison of the set of face key points of the front and back two face images to determine whether the key-point positions given by the key-point model are abnormal. Among them, the set of face key points collected in the current frame of face image is defined as the first set of face key points, and the set of face key points collected in the previous frame of face image is defined as the second set of face key points.

[0042] Specifically, for the current frame of face image input corresponding to the key-point model, the first set of face key points is output where x is the image column coordinate, y is the image row coordinate, and n is an integer from 0 to 105; for the previous frame of face image input corresponding to the key-point model, the second set of face key points is output where x is the image column coordinate, y is the image row coordinate, and n is an integer from 0 to 105; the widths of the face frames of the current frame of face image and the previous frame of face image are defined as w, and the heights are defined as h.

[0043] S120. Calculate the relative offset values of each pre-divided face area based on the first set of face key points and the second set of face key points. The relative offset value represents the displacement situation of the face key points in the corresponding face area between the previous frame of face image and the current frame of face image.

[0044] By dividing a face image into several regions, these regions can be based on the natural distribution of facial features, and each region contains a set of key point groupings. Based on the first set of face key points and the second set of face key points determined above, the relative offset value of each face region is calculated to characterize the displacement of the face key points in the corresponding face region between the previous frame of the face image and the current frame of the face image. It can be understood that the relative offset value provides a quantitative description of the movement of the face region between consecutive frames. By comparing these offset values with a preset threshold or a statistically based criterion (such as standard deviation), abnormal key points or regions can be automatically detected, thereby accurately performing anomaly detection on the face key point acquisition results.

[0045] Optionally, the relative offset value can calculate the coordinate difference based on the face key point groupings of the corresponding face regions in the first set of face key points and the second set of face key points, and calculate the mean value based on the coordinate difference to represent the relative offset value. The relative offset value can also be the offset information of each key point calculated based on the face key point groupings of the corresponding face regions in the first set of face key points and the second set of face key points, and filter out the key point coordinates with excessive offset (such as exceeding the threshold set by the mean value of the offset information) based on the offset information as the relative offset value. The present application does not make a fixed limitation on the calculation method of the relative offset value and will not elaborate here.

[0046] Specifically, referring to Figure 3 , calculating the relative offset value of each pre-divided face region based on the first set of face key points and the second set of face key points includes:

[0047] S1201. Calculate the global offset value between the previous frame of the face image and the current frame of the face image based on all the key point coordinates in the first set of face key points and the second set of face key points;

[0048] S1202. Calculate the local offset value of the corresponding face region between the previous frame of the face image and the current frame of the face image based on the key point coordinates of the corresponding face region in the first set of face key points and the second set of face key points;

[0049] S1203. Calculate the relative offset value of the corresponding face region based on the global offset value and the local offset value.

[0050] Since the key points output by the key point model move smoothly and within a limited range following the user's face in a normal scenario, extreme abnormal scenarios such as severe occlusion and rapid movement will cause jumps in the key points. Based on this, the present application detects the jump amplitude of the key point positions in adjacent videos to describe the abnormal degree of the key point positions.

[0051] Such as Figure 2As shown, for the first set of facial key points l of the facial image of the current frame output by the key point model t , it is divided into several groups according to the facial area: left cheek {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}, chin {12, 13, 14, 15, 16, 17, 18, 19, 20}, right cheek {21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32}, left eye {52, 53, 57, 54, 55, 56, 72, 73, 74}, right eye {58, 59, 63, 60, 61, 62, 75, 76, 77}, nose {80, 82, 47, 48, 49, 50, 81, 83, 51, 43, 44, 45, 46}, mouth {84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95}, left eyebrow {33, 34, 64, 36, 37, 66, 67, 35, 65}, right eyebrow {38, 39, 68, 69, 41, 42, 71, 40, 70}.

[0052] Furthermore, based on all the key point coordinates in the first set of facial key points and the second set of facial key points, calculate the global offset value between the facial image of the previous frame and the facial image of the current frame, including:

[0053] Calculate the first global average coordinate based on all the key point coordinates of the first set of facial key points;

[0054] Calculate the second global average coordinate based on all the key point coordinates of the second set of facial key points;

[0055] Calculate the global offset value between the facial image of the previous frame and the facial image of the current frame according to the first global average coordinate and the second global average coordinate.

[0056] Among them, calculate the first global average coordinate for all the key point coordinates of the first set of facial key points Calculate the second global average coordinate for all the key point coordinates of the second set of facial key points Furthermore, calculate the global offset value between the two Among them, G is used to identify the global key points.

[0057] Optionally, it is also possible to calculate the displacement of each key point in the first set of facial key points and the second set of facial key points between the current frame and the previous frame. Average the displacements of all the key points to obtain the global offset value.

[0058] On the other hand, based on the key point coordinates of the corresponding facial areas in the first set of facial key points and the second set of facial key points, calculate the local offset value of the corresponding facial areas between the facial image of the previous frame and the facial image of the current frame, including:

[0059] Calculate the first local average coordinate based on the key point coordinates of the corresponding face area in the first face key point set;

[0060] Calculate the second local average coordinate based on the key point coordinates of the corresponding face area in the second face key point set;

[0061] Calculate the local offset value of the corresponding face area between the previous frame of face image and the current frame of face image according to the first local average coordinate and the second local average coordinate.

[0062] For a corresponding face area, calculate the average coordinate of the corresponding key point group of the current frame of face image Calculate the average coordinate of the corresponding key point group of the previous frame of face image Furthermore, calculate the local offset value between the two where N is the index number of the above key point group. The above calculation of the average coordinate value is mainly to exclude the anomalies of individual points, so that the significant anomalies of the whole area can be better reflected in the index.

[0063] Further, calculate the relative offset value of the corresponding face area based on the global offset value and the local offset value, including:

[0064] Determine the difference result between the global offset value and the local offset value of the corresponding face area, and use the difference result as the relative offset value of the corresponding face area.

[0065] Finally, by calculating the relative offset value between each key point group and the global:

[0066]

[0067] where The relative offset value describes the displacement of the key point group relative to the whole face between the current frame and the previous frame.

[0068] Through the above steps, the relative offset value of each pre-divided face area can be obtained, and these values can be used in subsequent anomaly detection tasks to more accurately identify the anomalies in the face key point acquisition results.

[0069] S130. Calculate the key point anomaly index based on the relative offset value, determine the comparison result between the key point anomaly index and the set index threshold, and determine the face key point anomaly detection result of the current frame of face image based on the comparison result.

[0070] After that, one or more metrics capable of quantifying the degree of abnormality of key points are selected. These metrics can be statistics based on the relative offset values of each face region, such as mean, standard deviation, maximum value, minimum value, etc., or more complex functions, such as weighted sum of relative offset values, historical change rate of offset, etc. According to the selected metrics, calculations are performed using the relative offset values. For example, if the standard deviation is selected as the abnormality metric, then the standard deviation of the relative offset values of each region needs to be calculated.

[0071] In practical applications, one or more thresholds need to be set for each abnormality metric based on experience or experimental data. These thresholds are used to distinguish normal and abnormal situations in key point detection. The calculated abnormality metrics are compared with the set thresholds. If an abnormality metric exceeds the threshold, it is considered that the face region or key point represented by this metric is abnormal. Based on the comparison results of the abnormality metrics for each key point or region, a comprehensive judgment is made on whether there are abnormalities in the face key points of the current frame of face image.

[0072] When an abnormality occurs, the current frame of face image can be re-annotated and used as training data to retrain the key point model to improve the key point model. By detecting in real time scenes where the key point model performs poorly during the operation of the key point model (for example, when using beauty makeup or some special effect stickers), and reporting them to the server to collect relevant scene data. The server regularly annotates the collected data manually or by machine according to the abnormality detection results, and supplements the data to the training dataset of the key point model to fine-tune and optimize the key point model. The optimized key model can obtain better key point detection results in relevant scenarios, which are reflected in relevant special effect effects and improve the user experience.

[0073] Optionally, calculating the key point abnormality metric based on the relative offset value includes:

[0074] Obtain the horizontal offset value and vertical offset value of the relative offset value, obtain the face width and face height in the current frame of face image, and input the horizontal offset value, vertical offset value, face width and face height into a set formula to calculate the key point abnormality metric.

[0075] Among them, calculating the key point abnormality metric is as follows:

[0076]

[0077] where w and h are the width and height of the face bounding box. The key point abnormality metric describes the jump of each key point group between frames relative to the overall face within the face bounding box. When this metric is large, it means that the jump of the key points in this group between frames is relatively significant relative to the face bounding box. When the metric value of a certain group is large enough to reach the corresponding metric threshold, it indicates that the detection abnormality is triggered.

[0078] Further, referring to Figure 4 , based on the comparison result, determine the face key point anomaly detection result of the current frame face image, including:

[0079] S1301. Determine the cumulative number of times the corresponding face area triggers detection anomalies based on the comparison result of the corresponding face area, determine the number of abnormal areas of the current frame face image based on the comparison results of each face area, determine the face orientation information of the current frame face image and the face position information of the corresponding face area;

[0080] S1302. Determine the anomaly level of the current frame face image according to the cumulative number of times, the number of abnormal areas, the face orientation information and the face position information, and use the anomaly level as the face key point anomaly detection result of the current frame face image.

[0081] When an anomaly is triggered, according to the face information attached to the key point model, such as face confidence, key point visibility, face orientation, etc., rules can be set to initially divide the anomaly cause and anomaly level. The rules are usually defined according to the optimization requirements. For example:

[0082] 1. If an anomaly is triggered multiple times within T frames, divide the anomaly level according to the number of trigger times;

[0083] 2. Divide the anomaly level according to the face position when the anomaly is triggered, whether the face is near the middle of the screen, at the edge of the screen, or half of the face is outside the screen;

[0084] 3. Combine the face orientation and the anomaly index value. When the face is basically facing the screen, a lower anomaly index can be assigned a higher anomaly level, and a larger angle face requires a higher anomaly index to be assigned a higher anomaly level;

[0085] 4. If multiple key points are grouped to trigger an anomaly, divide the anomaly level according to the number of groups of triggered anomalies.

[0086] According to actual requirements, the anomaly level of the current frame face image can be determined by combining information such as the cumulative number of times, the number of abnormal areas, the face orientation information and the face position information and the corresponding rules. Furthermore, according to different anomaly levels, corresponding anomaly handling can be adaptively performed. For example, when the anomaly level is low, select the current frame image for annotation and use it as model training data to improve the key point model. When the anomaly level is high, it is necessary to re-collect training data for model training. Different processing methods can be set for different anomaly detection results in this application, and there is no fixed limit here.

[0087] As described above, by obtaining the first set of facial key points collected in the current frame of facial image and the second set of facial key points collected in the previous frame of facial image; calculating the relative offset values of each pre-divided facial region based on the first set of facial key points and the second set of facial key points, where the relative offset value represents the displacement of the facial key points in the corresponding facial region between the previous frame of facial image and the current frame of facial image; calculating the key point anomaly index based on the relative offset value, determining the comparison result between the key point anomaly index and the set index threshold, and determining the facial key point anomaly detection result of the current frame of facial image based on the comparison result. By adopting the above technical means, through calculating the relative offset values of two consecutive frames of facial images and performing anomaly detection on the collection of facial key points based on the relative offset values, it is possible to avoid the situation of high cost investment in anomaly detection caused by manual screening methods, thereby realizing efficient and accurate anomaly detection of the collection results of facial key points, improving the anomaly detection effect of the collection results of facial key points, further improving the improvement effect of the facial key point collection model, and enhancing the accuracy and stability of facial key point collection.

[0088] Based on the above embodiments, Figure 5 This is a schematic structural diagram of an anomaly detection system for the collection results of facial key points provided by the present application. Refer to Figure 5 In this embodiment, the anomaly detection system for the collection results of facial key points specifically includes: an acquisition module 21, an offset calculation module 22, and an anomaly detection module 23.

[0089] Among them, the acquisition module 21 is configured to obtain the first set of facial key points collected in the current frame of facial image and the second set of facial key points collected in the previous frame of facial image;

[0090] The offset calculation module 22 is configured to calculate the relative offset values of each pre-divided facial region based on the first set of facial key points and the second set of facial key points, where the relative offset value represents the displacement of the facial key points in the corresponding facial region between the previous frame of facial image and the current frame of facial image;

[0091] The anomaly detection module 23 is configured to calculate the key point anomaly index based on the relative offset value, determine the comparison result between the key point anomaly index and the set index threshold, and determine the facial key point anomaly detection result of the current frame of facial image based on the comparison result.

[0092] Specifically, calculating the relative offset values of each pre-divided facial region based on the first set of facial key points and the second set of facial key points includes:

[0093] Calculating the global offset value between the previous frame of facial image and the current frame of facial image based on all the key point coordinates in the first set of facial key points and the second set of facial key points;

[0094] Calculate the local offset value of the corresponding face region between the previous frame of face image and the current frame of face image based on the key point coordinates of the corresponding face region in the first face key point set and the second face key point set;

[0095] Calculate the relative offset value of the corresponding face region based on the global offset value and the local offset value.

[0096] Among them, calculate the global offset value between the previous frame of face image and the current frame of face image based on all the key point coordinates in the first face key point set and the second face key point set, including:

[0097] Calculate the first global average coordinate based on all the key point coordinates of the first face key point set;

[0098] Calculate the second global average coordinate based on all the key point coordinates of the second face key point set;

[0099] Calculate the global offset value between the previous frame of face image and the current frame of face image according to the first global average coordinate and the second global average coordinate.

[0100] Calculate the local offset value of the corresponding face region between the previous frame of face image and the current frame of face image based on the key point coordinates of the corresponding face region in the first face key point set and the second face key point set, including:

[0101] Calculate the first local average coordinate based on the key point coordinates of the corresponding face region in the first face key point set;

[0102] Calculate the second local average coordinate based on the key point coordinates of the corresponding face region in the second face key point set;

[0103] Calculate the local offset value of the corresponding face region between the previous frame of face image and the current frame of face image according to the first local average coordinate and the second local average coordinate.

[0104] Calculate the relative offset value of the corresponding face region based on the global offset value and the local offset value, including:

[0105] Determine the difference result between the global offset value and the local offset value of the corresponding face region, and use the difference result as the relative offset value of the corresponding face region.

[0106] Specifically, calculate the key point anomaly index based on the relative offset value, including:

[0107] Obtain the horizontal offset value and the vertical offset value of the relative offset value, obtain the face width and face height in the current frame of face image, and input the horizontal offset value, the vertical offset value, the face width and the face height into the set formula to calculate the key point anomaly index.

[0108] Specifically, determining the face key-point anomaly detection result of the current frame face image based on the comparison result includes:

[0109] Determining the cumulative number of times that the corresponding face region triggers detection anomalies based on the comparison result of the corresponding face region, determining the number of anomaly regions of the current frame face image based on the comparison results of each face region, determining the face orientation information of the current frame face image, and the face position information of the corresponding face region;

[0110] Determining the anomaly level of the current frame face image according to the cumulative number of times, the number of anomaly regions, the face orientation information, and the face position information, and using the anomaly level as the face key-point anomaly detection result of the current frame face image.

[0111] As described above, by obtaining the first set of face key points collected in the current frame face image and the second set of face key points collected in the previous frame face image; calculating the relative offset value of each pre-divided face region based on the first set of face key points and the second set of face key points, where the relative offset value represents the displacement of the face key points in the corresponding face region between the previous frame face image and the current frame face image; calculating the key-point anomaly index based on the relative offset value, determining the comparison result between the key-point anomaly index and the set index threshold, and determining the face key-point anomaly detection result of the current frame face image based on the comparison result. By adopting the above technical means, by calculating the relative offset value of two consecutive frame face images and performing anomaly detection on the face key-point acquisition based on the relative offset value, it avoids the situation of high anomaly detection cost input caused by the manual screening method, thereby realizing efficient and accurate anomaly detection of the face key-point acquisition result, improving the anomaly detection effect of the face key-point acquisition result, further improving the improvement effect of the face key-point acquisition model, and improving the accuracy and stability of the face key-point acquisition.

[0112] The anomaly detection system for the face key-point acquisition result provided by the embodiment of the present application can be configured to execute the anomaly detection method for the face key-point acquisition result provided by the above embodiment, and has the corresponding functions and beneficial effects.

[0113] On the basis of the above actual example, the embodiment of the present application further provides an anomaly detection device for the face key-point acquisition result, referring to Figure 6, the abnormal detection device for the face key point acquisition result includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the abnormal detection method for the face key point acquisition result described in any embodiment of the present application (for example, the acquisition module, offset calculation module, and abnormal detection module in the abnormal detection system for the face key point acquisition result). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, to implement the above-mentioned abnormal detection method for the face key point acquisition result. The input device can be configured to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device may include a display device such as a display screen. The above-provided abnormal detection device for the face key point acquisition result can be configured to execute the abnormal detection method for the face key point acquisition result provided in the above embodiment, and has corresponding functions and beneficial effects.

[0114] Based on the above embodiment, an embodiment of the present application further provides a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are configured to execute an abnormal detection method for the face key point acquisition result when executed by a computer processor. The storage medium can be any various types of memory devices or storage devices. Of course, for the non-volatile computer-readable storage medium provided in the embodiment of the present application, the computer-executable instructions are not limited to the above-mentioned abnormal detection method for the face key point acquisition result, and can also execute related operations in the abnormal detection method for the face key point acquisition result provided in any embodiment of the present application.

[0115] Based on the above embodiment, an embodiment of the present application further provides a computer program product. Essentially, or the part that makes a contribution to the prior art, or all or part of the technical solution of the present application can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions for causing a computer device, a mobile terminal, or a processor therein to execute all or part of the steps of the abnormal detection method for the face key point acquisition result described in each embodiment of the present application.

Claims

1. A method for detecting anomalies in facial key point collection results, characterized in that: include: Obtaining a first set of facial key points collected in a current frame of facial image and a second set of facial key points collected in a previous frame of facial image; Calculating relative offset values ​​of each pre-divided face region based on the first face key point set and the second face key point set, wherein the relative offset values ​​represent displacement of face key points of the corresponding face region between a previous face image frame and a current face image frame; A key point anomaly index is calculated based on the relative offset value, a comparison result between the key point anomaly index and a set index threshold is determined, and a face key point anomaly detection result of the current frame face image is determined based on the comparison result.

2. The method for detecting anomalies of facial key point collection results according to claim 1, characterized in that: The calculating the relative offset value of each pre-divided face area based on the first face key point set and the second face key point set includes: Calculating a global offset value between a previous face image and a current face image based on the coordinates of all key points in the first face key point set and the second face key point set; Calculating a local offset value of a corresponding face area between a previous face image frame and a current face image frame based on the coordinates of the key points of the corresponding face area in the first face key point set and the second face key point set; A relative offset value of a corresponding face area is calculated based on the global offset value and the local offset value.

3. The method for detecting anomalies of facial key point collection results according to claim 2, characterized in that: The calculating, based on the coordinates of all key points in the first face key point set and the second face key point set, a global offset value between a previous face image and a current face image, comprises: Calculate a first global average coordinate based on the coordinates of all key points in the first facial key point set; Calculate a second global average coordinate based on the coordinates of all key points in the second facial key point set; A global offset value between a previous face image frame and a current face image frame is calculated according to the first global average coordinate and the second global average coordinate.

4. The method for detecting anomalies of facial key point collection results according to claim 2, characterized in that: The calculating, based on the key point coordinates of the corresponding face area in the first face key point set and the second face key point set, a local offset value of the corresponding face area between the previous face image and the current face image, comprises: Calculating first local average coordinates based on the coordinates of key points corresponding to the face area in the first face key point set; Calculating a second local average coordinate based on the coordinates of the key points corresponding to the face area in the second face key point set; A local offset value of a corresponding face region between a previous face image frame and a current face image frame is calculated according to the first local average coordinates and the second local average coordinates.

5. The method for detecting anomalies of facial key point collection results according to claim 2, characterized in that: The calculating the relative offset value of the corresponding face area based on the global offset value and the local offset value includes: A difference result between the global offset value and the local offset value corresponding to the face area is determined, and the difference result is used as a relative offset value corresponding to the face area.

6. The method for detecting anomalies in facial key point collection results according to any one of claims 1 to 5, characterized in that: The calculating of the key point abnormality index based on the relative offset value includes: Obtain the horizontal coordinate offset value and the vertical coordinate offset value of the relative offset value, obtain the face width and the face height in the current frame face image, and input the horizontal coordinate offset value, the vertical coordinate offset value, the face width and the face height into a set formula to calculate the key point abnormality index.

7. The method for detecting anomalies in facial key point collection results according to any one of claims 1 to 5, characterized in that: The determining of the facial key point abnormality detection result of the current frame facial image based on the comparison result includes: Determine the cumulative number of times the corresponding face area triggers detection abnormality based on the comparison result of the corresponding face area, determine the number of abnormal areas of the current frame face image based on the comparison results of each face area, determine the face orientation information of the current frame face image and the face position information of the corresponding face area; The abnormality level of the current frame face image is determined according to the accumulated number of times, the number of abnormal areas, the face orientation information and the face position information, and the abnormality level is used as the face key point abnormality detection result of the current frame face image.

8. A system for detecting anomalies in facial key point collection results, characterized in that: include: An acquisition module configured to acquire a first set of facial key points collected in a current frame of facial image and a second set of facial key points collected in a previous frame of facial image; an offset calculation module, configured to calculate a relative offset value of each pre-divided face area based on the first face key point set and the second face key point set, wherein the relative offset value represents a displacement of the face key points of the corresponding face area between a previous face image frame and a current face image frame; The anomaly detection module is configured to calculate a key point anomaly index based on the relative offset value, determine a comparison result between the key point anomaly index and a set index threshold, and determine a face key point anomaly detection result of the current frame face image based on the comparison result.

9. A device for detecting abnormalities in facial key point collection results, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting anomalies in facial key point collection results as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by a computer processor, are configured to execute the method for detecting anomalies in facial key point acquisition results as described in any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the method for detecting anomalies of facial key point collection results as described in any one of claims 1 to 7.

Images