Blink detection method and device, computer equipment and storage medium

Abstract

The invention relates to a blink detection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring an initial blink detection result of a face image sequence and a target intersection point corresponding to a face image, wherein the target intersection point is used for representing the change degree of a target face key point in a blink process; determining an average distance corresponding to each face image according to the target intersection point and each target eye key point in the face key point set; performing distance dispersion degree calculation on the average distance corresponding to each face image to obtain a distance dispersion degree corresponding to the face image sequence; and obtaining a blink detection result of the face image sequence based on the initial blink detection result and the distance dispersion degree. By adopting the method, resources can be saved, and the blink detection robustness can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to a blink detection method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] With the development of Internet technologies, identity verification technologies have emerged, including combinations of passwords, fingerprint recognition, face recognition, iris recognition, etc. In the use of face recognition, during verification, interactions with the verification object, such as prompting the verification object to blink, open the mouth, nod, etc., are usually used to detect whether the verification object is a real person rather than a static picture or a 3D model, thereby effectively improving the security level of identity verification. Currently, when performing blink detection, malicious objects can disguise blinks through abnormal means, resulting in security risks. To avoid security risks, usually more neural network models are stacked for security verification. For example, blink detection is performed after occlusion detection passes through the model. However, stacking more neural network models for security verification requires consuming additional computing resources and time resources. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a blink detection method, apparatus, computer device, computer-readable storage medium, and computer program product that can save resources and improve the robustness of blink detection.

[0004] In a first aspect, the present application provides a blink detection method, including:

[0005] Obtaining an initial blink detection result of a sequence of face images;

[0006] Obtaining a target intersection point corresponding to a face image in the sequence of face images, where the target intersection point is used to characterize the degree of change of target face key points during a blink;

[0007] Determining an average distance corresponding to each face image according to the target intersection point and each target eye key point in the set of face key points;

[0008] Calculating the degree of distance dispersion of the average distance corresponding to each face image to obtain the degree of distance dispersion corresponding to the sequence of face images;

[0009] Obtaining a blink detection result of the sequence of face images based on the initial blink detection result and the degree of distance dispersion.

[0010] In a second aspect, the present application further provides a blink detection apparatus, including:

[0011] A blink detection module, configured to obtain an initial blink detection result of a sequence of face images;

[0012] An intersection acquisition module, configured to acquire a target intersection corresponding to a face image in a face image sequence, where the target intersection is used to characterize the change degree of target face key points during a blinking process;

[0013] A distance calculation module, configured to determine an average distance corresponding to each face image according to the target intersection and each target eye key point in a face key point set;

[0014] A dispersion degree calculation module, configured to calculate a distance dispersion degree of an average distance corresponding to each face image to obtain a distance dispersion degree corresponding to the face image sequence;

[0015] A blinking result obtaining module, configured to obtain a blinking detection result of the face image sequence based on an initial blinking detection result and the distance dispersion degree.

[0016] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] Obtain an initial blinking detection result of a face image sequence;

[0018] Obtain a target intersection corresponding to a face image in a face image sequence, where the target intersection is used to characterize the change degree of target face key points during a blinking process;

[0019] Determine an average distance corresponding to each face image according to the target intersection and each target eye key point in a face key point set;

[0020] Calculate a distance dispersion degree of an average distance corresponding to each face image to obtain a distance dispersion degree corresponding to the face image sequence;

[0021] Obtain a blinking detection result of the face image sequence based on the initial blinking detection result and the distance dispersion degree.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0023] Obtain an initial blinking detection result of a face image sequence;

[0024] Obtain a target intersection corresponding to a face image in a face image sequence, where the target intersection is used to characterize the change degree of target face key points during a blinking process;

[0025] Determine an average distance corresponding to each face image according to the target intersection and each target eye key point in a face key point set;

[0026] Calculate the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the face image sequence;

[0027] Obtain the blink detection result of the face image sequence based on the initial blink detection result and the distance dispersion degree.

[0028] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0029] Obtain the initial blink detection result of the face image sequence;

[0030] Obtain the target intersection points corresponding to the face images in the face image sequence, where the target intersection points are used to characterize the change degree of the target face key points during the blink process;

[0031] Determine the average distance corresponding to each face image according to the target intersection points and each target eye key point in the face key point set;

[0032] Calculate the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the face image sequence;

[0033] Obtain the blink detection result of the face image sequence based on the initial blink detection result and the distance dispersion degree.

[0034] The above blink detection method, device, computer device, storage medium and computer program product obtain the initial blink detection result of the face image sequence; obtain the target intersection points corresponding to the face images in the face image sequence, where the target intersection points are used to characterize the change degree of the target face key points during the blink process; determine the average distance corresponding to each face image according to the target intersection points and each target eye key point in the face key point set, calculate the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the face image sequence; obtain the blink detection result of the face image sequence based on the initial blink detection result and the distance dispersion degree. That is, by obtaining the initial blink detection result and verifying the blink detection result by calculating the distance dispersion degree, it is not necessary to stack a neural network model that consumes a large amount of resources for verification, thereby reducing the consumption of computing resources and time resources. And the final blink detection result is determined by combining the initial blink detection result and the distance dispersion degree calculated using the target intersection points. The target intersection points are used to characterize the change degree of the target face key points during the blink process, and then the blink detection result is determined by the dispersion degree of the average distance between the target intersection points and the target eye key points during the blink process, which can effectively determine the change of the eye key points during the blink process, and thus can effectively improve the robustness of the blink detection.<== Description of the Drawings

[0035] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0036] Figure 1 It is an application environment diagram of the blink detection method in an embodiment;

[0037] Figure 2 It is a schematic flowchart of the blink detection method in an embodiment;

[0038] Figure 3 It is a schematic flowchart of obtaining the initial successful blink detection result in an embodiment;

[0039] Figure 4 It is a schematic diagram of the eye key points in the left eye in a specific embodiment;

[0040] Figure 5 It is a schematic diagram of the eye key points in the right eye in a specific embodiment;

[0041] Figure 6 It is a schematic flowchart of obtaining the difference in eye region changes in an embodiment;

[0042] Figure 7 It is a schematic flowchart of obtaining the target intersection point in an embodiment;

[0043] Figure 8 It is a schematic flowchart of blink detection in a specific embodiment;

[0044] Figure 9 It is a schematic diagram of the face key points used during blink detection in a specific embodiment;

[0045] Figure 10 It is a functional block diagram of blink detection in a specific embodiment;

[0046] Figure 11 It is a structural block diagram of the blink detection device in an embodiment;

[0047] Figure 12 It is an internal structure diagram of a computer device in an embodiment;

[0048] Figure 13 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The blink detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 obtains the initial blink detection result of the facial image sequence; the terminal 102 obtains the target intersection point corresponding to the facial image in the facial image sequence, and the target intersection point is used to characterize the degree of change of the target facial key point during the blink process; the terminal 102 determines the average distance corresponding to each facial image based on the target intersection point and each target eye key point in the facial key point set; the terminal 102 calculates the distance dispersion degree of the average distance corresponding to each facial image to obtain the distance dispersion degree corresponding to the facial image sequence; the terminal 102 obtains the blink detection result of the facial image sequence based on the initial blink detection result and the distance dispersion degree. The terminal 102 can send the successful blink detection result corresponding to the facial image sequence to the server 104, and the server 104 can save the successful blink detection result corresponding to the facial image sequence to the data storage system. Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0051] Currently, when performing blink detection, blink action detection is usually performed directly. However, direct blink action detection can be easily deceived by malicious users through abnormal means, such as tentatively shaking chopsticks, pens, or other long objects around the eyes, or quickly blocking and moving objects such as notebooks, causing some key points of the human eye to jitter or "float". This unstable jittering state has a certain probability of causing the key points in the human eye area to change to meet the logic of the blink action, thereby passing the blink detection. To address this problem, currently, more models are usually superimposed for security verification, such as connecting an occlusion detection model in series to detect whether the eyes are blocked. However, the additional computing overhead and latency brought by connecting occlusion detection models in series make it difficult to deploy on mobile devices.

[0052] The blink detection provided by this application, after obtaining the initial blink detection result of the face image sequence, obtains the target intersection point, then calculates the distance dispersion degree corresponding to the face image sequence according to the target intersection point and each target eye key point in the face key point set, and finally obtains the blink detection result of the face image sequence according to the initial blink detection result and the distance dispersion degree. That is, the blink detection result of the face image sequence is obtained through different blink judgment logics, without the need to superimpose a neural network model that consumes a large amount of resources for verification, thereby reducing the consumption of computing resources and time resources and enabling convenient deployment on mobile devices.

[0053] In an exemplary embodiment, as Figure 2 shown, a blink detection method is provided. Taking the example that this method is applied to Figure 1 the terminal therein for illustration, it can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 202 to 210. Among them:

[0054] S202, obtain the initial blink detection result of the face image sequence.

[0055] Among them, the face image sequence refers to a sequence of continuous face images. The face images in the face image sequence are consecutive image frames, and the face image sequence may include at least four face images. The initial blink detection result refers to the blink detection result that has not been verified, including the blink detection success result and the blink detection failure result. The initial blink detection result is a detection result indicating whether there is a blink in the face image sequence. It can be understood that in a face image sequence, there may be one or more blink processes. In this embodiment, as long as one blink process can be detected in the face image sequence, the initial blink detection result is the initial blink detection success result.

[0056] Specifically, the terminal can obtain the initial blink detection result of the face image sequence from the memory, or obtain the face image sequence sent by the server. The terminal can also obtain the initial blink detection result of the face image sequence uploaded by the user.

[0057] Optionally, the server or the terminal can perform face key point detection on the face images in the face image sequence to obtain a face key point set. It can be understood that since the face actions or expressions in the face image sequence may change, the positions of the same face key point in the face key point set may be different in different face images. Obtain the eye key point set in the face key point set, and perform blink detection based on the changes of the eye key points in the eye key point set in the face image sequence to obtain the initial blink detection result of the face image sequence.

[0058] In one embodiment, the terminal can acquire a sequence of face images, perform face key point detection on each face image in the sequence of face images to obtain a set of face key points corresponding to each face image. Then, an eye key point set corresponding to each face image is determined from the set of face key points, and eye blink detection is performed based on the eye key points in the eye key point set corresponding to each face image to obtain an initial blink detection result corresponding to the sequence of face images.

[0059] Among them, face key points refer to the coordinate points at key positions in the face region. The set of face key points refers to the set of all face key points corresponding to a face image. Each face image is detected to obtain corresponding face key points. The eye key point set refers to the set of key points in the eye region.

[0060] Specifically, the terminal can acquire a sequence of face images in real time. The terminal can also obtain a sequence of face images from the memory. The terminal can also obtain a sequence of face images sent by the server. The terminal can also acquire a face video and obtain a sequence of face images from the face video. Then, face key point detection is performed on each face image in the sequence of face images using a face key point detection algorithm to obtain a set of face key points corresponding to each face image. Among them, the face key point detection algorithm is an algorithm for detecting the coordinate points at key positions in the face region. For example, it can be the Mediapipe (a multi-task model running on a terminal device) algorithm. By using the Mediapipe algorithm to detect the key points of a face image, 468 face key points corresponding to the face image are obtained. Each key point is represented in the form of (x, y, z), where x and y are coordinate values based on the pixel coordinate system of the image and are normalized by the width and height of the input image, and z represents the offset of the key point from the face centroid. The terminal can also use a face key point detection algorithm annotated with other numbers of key points. For example, a detection algorithm with 68 key points annotated on the face, a detection algorithm with 106 or 186 key points annotated on the face, and so on.

[0061] The terminal filters the eye key points corresponding to the face image from the set of face key points according to the annotation index of the eye key points to obtain an eye key point set. This eye key point set can be a left-eye key point set, a right-eye key point set, or a set of left-eye and right-eye key points. Traverse the set of face key points corresponding to each face image to obtain an eye key point set corresponding to each face image. Then, eye blink detection is performed using the eye key points in the eye key point set. Among them, the change trend of the eye key points corresponding to the face image in the sequence of face images can be calculated. For example, the change in the eye aspect ratio or the change in the area of the eye region, etc. Then, an initial blink detection result corresponding to the sequence of face images is determined according to the change trend of the eye key points.

[0062] In one embodiment, the terminal inputs a sequence of face images into a trained blink detection neural network model to obtain an initial blink detection result. The blink detection neural network model is obtained by training a neural network using historical videos and corresponding training labels. The training labels include blink labels and non-blink labels. The neural network can be a convolutional neural network, a recurrent neural network, a temporal neural network, etc.

[0063] S204. Obtain a target intersection point corresponding to the face image in the sequence of face images. The target intersection point is used to characterize the degree of change of the target face key points during the blink process.

[0064] Among them, each face image during the blink process has a corresponding target intersection point. The target intersection point corresponding to the face image is used to characterize the degree of change of the target face key points during the blink process. The degree of change of the target face key points refers to the change in the distance between the target face key points during the blink process. For example, it can be the change in the distance between two target face key points in a target face key point pair, or the change in the average distance between the target face key points in each target face key point pair. The target face key points can be key points in the area of the face other than the eye area. The target face key points refer to the face key points with a degree of change less than a threshold during the blink process, that is, the change in the distance between the target face key points during the blink process is less than the threshold. The face key points refer to the key points in the set of face key points, and the set of face key points can be obtained by performing key point extraction on the face image.

[0065] Specifically, the terminal can directly obtain the target intersection points corresponding to the face images in the face image sequence. The target intersection points can be set manually according to historical experience. For example, the target intersection points can be determined based on the historical target intersection point positions during historical blinking. The terminal can also use the target face key points to determine the target intersection points according to the degree of change of the target face key points during the blinking process. Among them, the target face key points can be the face key points determined according to human experience, such as the face key points of the nose tip, the face key points of the corners of the mouth, etc. Then, the target intersection points can be determined using the target face key points determined according to human experience. The target face key points can also be the face key points selected from the set of face key points corresponding to the face image with a degree of change less than a threshold value. The threshold value can be set in advance and is determined according to human experience. Then, the target intersection points can be determined using the target face key point pairs. The degree of change refers to the change in the distance between the face key points in the target face key point pairs in the face image during the blinking process. The face key points in the target face key point pairs can be determined and indexed in advance. The annotation indexes of the target face key points corresponding to different face images can be the same. During the blinking process, the degree of change of the target face key points in the target face key point pairs is less than the threshold value. The degree of change of the target face key points can be characterized by the change in the distance between the target face key points, that is, the change in the distance between the target face key points in the target face key point pairs during the blinking process is less than the threshold value.

[0066] S206. Determine the average distance corresponding to each face image according to the target intersection points and each target eye key point in the set of face key points.

[0067] Among them, the target eye key points are the eye key points that are preset to calculate the pixel distance with the target intersection points. The target eye key points can be the key points of the eye corners or the key points of the eye boundaries. The average distance is used to characterize the stability of the position of the target face key points during the blinking process. The average distance can reflect the state of the deviation of the target face key points from the true position and is used to determine whether there is occlusion in the eye area. When the degree of change of the average distance corresponding to the face image during the blinking process is smaller, it indicates that the deviation of the target face key points from the true position is smaller, that is, the possibility of occlusion in the eye area is smaller.

[0068] Specifically, the terminal calculates the average distance corresponding to each face image according to the target intersection points of each face image and each target eye key point in the set of face key points of each face image, that is, calculates the pixel distance between the target intersection points and each target eye key point, obtains the distance between each target eye key point and the target intersection points, and then calculates the average distance corresponding to the face image according to the sum of all distances and the number of target eye key points.

[0069] S208. Calculate the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the face image sequence.

[0070] Among them, the distance dispersion degree is used to characterize the degree to which the target face key points deviate from the true position during the blinking process. This distance dispersion degree can be variance.

[0071] Specifically, the terminal calculates the change degree of the average distance corresponding to all face images. The distance dispersion degree can be calculated using the average distance corresponding to each face image, that is, calculate the variance of all average distances, and use this variance as the distance dispersion degree corresponding to the face image sequence.

[0072] S210. Obtain the blinking detection result corresponding to the face image sequence based on the initial blinking detection result and the distance dispersion degree.

[0073] Among them, the blinking detection result refers to the blinking detection result that has passed the stability verification based on the initial blinking detection result being the initial blinking detection success result for the face image sequence, including the blinking detection success result and the blinking detection failure result. The blinking detection result is the final detection result obtained by characterizing the blinking stability in the face image sequence during blinking. For example, if the stability detection index meets the preset blinking success condition, it means that the blinking is stable and the blinking detection result is the blinking detection success result; if the stability detection index does not meet the preset blinking success condition, it means that the blinking is unstable and the blinking detection result is the blinking detection failure result. Among them, the stability detection index includes the distance dispersion degree and the distance uniformity of the face image sequence.

[0074] Specifically, the terminal determines whether the initial blinking detection result and the distance dispersion degree meet the preset blinking success condition. When the initial blinking detection result and the distance dispersion degree meet the preset blinking success condition, the blinking detection success result corresponding to the face image sequence is obtained. When the initial blinking detection result and the distance dispersion degree do not meet the preset blinking success condition, the blinking detection failure result corresponding to the face image sequence is obtained. The preset blinking success condition refers to the conditions for successful blinking detection set in advance, which can include the conditions corresponding to the initial blinking detection result and the conditions corresponding to the distance dispersion degree. For example, it can be the initial blinking detection success result and the distance dispersion degree is less than the threshold.

[0075] The above-mentioned blink detection method includes: obtaining the initial blink detection result of a sequence of face images; obtaining the target intersection points corresponding to the face images in the sequence of face images, where the target intersection points are used to characterize the degree of change of target face key points during the blink process; determining the average distance corresponding to each face image according to the target intersection points and each target eye key point in the face key point set, calculating the degree of distance dispersion of the average distance corresponding to each face image to obtain the degree of distance dispersion corresponding to the sequence of face images; and obtaining the blink detection result of the sequence of face images based on the initial blink detection result and the degree of distance dispersion. That is, by obtaining the initial blink detection result and verifying the blink detection result by calculating the degree of distance dispersion, it is not necessary to superimpose a neural network model that consumes a large amount of resources for verification, thereby reducing the consumption of computing resources and time resources. And by combining the initial blink detection result and the degree of distance dispersion calculated using the target intersection points to determine the final blink detection result, where the target intersection points are used to characterize the degree of change of target face key points during the blink process, and then determining the blink detection result by the degree of dispersion of the average distance between the target intersection points and the target eye key points during the blink process, it can effectively determine the change of the eye key points during the blink process, and thus can effectively improve the robustness of blink detection.

[0076] In one embodiment, as Figure 3 shown, determining the set of eye key points corresponding to the face image from the face key point set, and performing eye blink detection based on the eye key points in the set of eye key points to obtain the initial blink detection result of the sequence of face images, including:

[0077] S302, determining the set of eye key points corresponding to the face image from the face key point set according to the correspondence between the eye key point identifier and the eye key point.

[0078] Among them, the eye key point identifier is used to uniquely identify the eye key point, which can be the name, index, identification symbol, etc. of the eye key point, and is preset. Different eye key points correspond to different eye key point identifiers.

[0079] Specifically, the terminal filters out the eye key points corresponding to each eye key point identifier from the face key points in the face key point set according to the correspondence between the eye key point identifier and the eye key point, so as to obtain the set of eye key points corresponding to the face image, and traverses the set of face key points corresponding to each face image to obtain the set of eye key points corresponding to each face image.

[0080] S304, performing eye high value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain the eye high value corresponding to the face image.

[0081] S306. Calculate the eye width value corresponding to the face image based on the eye key points in the eye key point set corresponding to the face image.

[0082] Among them, the eye height value is the pixel distance value used to represent the height of the eye region. The eye width value is the pixel distance value used to represent the width of the eye region.

[0083] Specifically, the terminal obtains the eye key point pair for calculating the height of the eye region from the eye key point set corresponding to the face image. This eye key point pair is obtained based on the face key points corresponding to the upper and lower boundaries of the eye region. Then calculate the average of the distances between the eye key points in all eye key point pairs to obtain the eye height value. At the same time, the terminal can obtain the eye key point pair for calculating the width of the eye region from the eye key point set corresponding to the face image. This eye key point pair is obtained based on the eye key points corresponding to the left eye corner region and the right eye corner region. For example, it can include one eye key point in the left eye corner region and one eye key point in the right eye corner region. Then calculate the average of the distances between the eye key points in all eye key point pairs to obtain the eye width value.

[0084] S308. Calculate the ratio of the eye height value to the eye width value to obtain the eye region ratio corresponding to the face image.

[0085] S310. Traverse and calculate the eye region ratio of each face image in the face image sequence, and calculate the change of the eye region ratio of each face image to obtain the blink change information corresponding to the face image sequence. Determine the initial blink detection result corresponding to the face image sequence according to the blink change information.

[0086] Among them, the blink change information is used to represent the change trend of the eye region corresponding to the face image.

[0087] Specifically, the terminal calculates the ratio of the eye height value to the corresponding eye width value, where the eye height value and the corresponding eye width value are of the same eye. For example, it can be the ratio of the right eye height value to the right eye width value, or the ratio of the left eye height value to the left eye width value. Then the terminal calculates the eye area ratio corresponding to each face image, and then determines the blink change information corresponding to the face image sequence according to the change of the eye area ratio of each face image. Then when the blink change information meets the preset blink condition, the initial blink detection success result corresponding to the face image sequence is obtained. When the blink change information does not meet the preset blink condition, the initial blink detection failure result corresponding to the face image sequence is obtained. The preset blink condition refers to the preset change trend condition of the eyes during blinking. For example, blinking is a process in which the eye area ratio changes from large to small and then to large. When the change of the eye area ratio of the face image during blinking conforms to the process of changing from large to small and then to large, the initial blink detection success result is obtained; otherwise, the initial blink detection failure result is obtained.

[0088] In the above embodiment, by calculating the eye width value and the eye height value, then calculating the ratio of the eye height value to the corresponding eye width value, determining the blink change information corresponding to the face image sequence according to the ratio, and finally obtaining the initial blink detection result according to the blink change information, the accuracy of blink detection is improved, and a model is not required for blink detection, which can reduce the calculation amount and save computing resources.

[0089] In one embodiment, S304, that is, calculating the eye height value based on the eye key points in the eye key point set corresponding to the face image to obtain the eye height value corresponding to the face image, includes the steps of:

[0090] Obtain the upper and lower eye key point pairs corresponding to the face image from the eye key point set corresponding to the face image, and calculate the pixel distance corresponding to the upper and lower eye key point pairs; perform average distance calculation based on the pixel distance of the upper and lower eye key point pairs and the number of the upper and lower eye key point pairs to obtain the eye height value corresponding to the face image.

[0091] Among them, the upper and lower eye key point pairs refer to two corresponding eye key points on the upper and lower boundaries of the eyes.

[0092] Specifically, the terminal can find the upper and lower eye key point pairs from the eye key point set corresponding to the face image according to the indexes of the upper and lower eye key points set in advance, and the upper and lower eye key point pairs can include at least two. Then use the Euclidean distance algorithm to calculate the distance between the upper eye key point and the lower eye key point in the upper and lower eye key point pairs to obtain the pixel distance corresponding to the upper and lower eye key point pairs. Among them, the Euclidean distance algorithm can be calculated using the following formula (1).

[0093] Dist = sqrt((x1 – x2) 2 + (y1 – y2) 2 ) Formula (1)

[0094] Among them, (x1, y1) are the position coordinates of the eye key points. (x2, y2) are the position coordinates of another eye key point. Dist is the pixel distance. sqrt is the square root operator.

[0095] Then the terminal calculates the sum of the pixel distances corresponding to all the upper and lower eye key point pairs in the face image, and then calculates the ratio of this sum to the number of upper and lower eye key point pairs, obtains the upper and lower average distance corresponding to the eye area in the face image, and then takes this upper and lower average distance as the eye height value, so as to obtain the eye height value corresponding to the face image.

[0096] In one embodiment, the terminal can calculate the eye height value corresponding to the right eye and / or the left eye, obtain the upper and lower eye key point pairs corresponding to the right eye and / or the left eye, and calculate the pixel distances corresponding to the upper and lower eye key point pairs. Finally, based on the pixel distances corresponding to the upper and lower eye key point pairs and the number of upper and lower eye key point pairs, the average distance is calculated to obtain the eye height value corresponding to the right eye and / or the left eye. This eye height value is the smallest and close to 0 when the eyes are closed, and the largest when the eyes are open.

[0097] In a specific embodiment, as Figure 4 shown, it is a schematic diagram of the eye key points in the left eye detected by the Mediapipe algorithm. Among them, the digital indexes of 5 upper and lower eye key point pairs are obtained, including S1=(161, 163), S2=(160, 144), S3=(159, 145), S4=(158, 153), S5=(157, 154). Among them, the upper and lower eye key point pair S1 includes the 161st face key point and the 163rd face key point, and the same applies to other upper and lower eye key point pairs. Then, according to the digital indexes of the upper and lower eye key point pairs, the pixel coordinates of the eye key points in the upper and lower eye key point pairs are determined, and the Euclidean distances between the two eye key points in each upper and lower eye key point pair are calculated using the pixel coordinates, obtaining the pixel distances corresponding to 5 upper and lower eye key point pairs, including L S1 , L S2 , L S3 , L S4 and L S1 . Then the terminal calculates the ratio of the sum of the pixel distances to the total number of upper and lower eye key point pairs, obtaining the eye height value H=(L S1 +L S2 +L S3 +L S4 +L S5 ) / 5. Among them, L S1Characterizes the distance between the eye key point 161 and the eye key point 163 in the upper and lower eye key point pair S1 = (161, 163), L S2 The distance between the eye key point 160 and the mouth key point 144 in the upper and lower eye key point pair S2 = (160, 144) is represented, and the distances between other upper and lower eye key points are deduced in the same way.

[0098] In the above embodiment, upper and lower eye key point pairs are obtained and the pixel distances corresponding to the upper and lower eye key point pairs are calculated. Finally, an average distance is calculated based on the pixel distances corresponding to the upper and lower eye key point pairs and the number of upper and lower eye key point pairs to obtain the eye height value corresponding to the facial image. The average distance is used as the eye height value. The eye height value of the facial image is represented by the average of the pixel distances of multiple upper and lower eye key point pairs. This can avoid errors in the eye height value caused by the randomness of the pixel distances of a particular upper and lower eye key point pair, thereby improving the accuracy of the obtained eye height value.

[0099] In one embodiment, S306, i.e., calculating the eye width value based on the eye key points in the eye key point set corresponding to the face image to obtain the eye width value corresponding to the face image, includes the steps of:

[0100] The left and right eye key point pairs corresponding to the face image are obtained from the eye key point set corresponding to the face image, and the pixel distance corresponding to the left and right eye key point pairs is calculated; the average distance is calculated based on the pixel distance corresponding to the left and right eye key point pairs and the number of left and right eye key point pairs to obtain the eye width value corresponding to the face image.

[0101] The left and right eye key point pairs are obtained based on the eye key points in the left eye corner area and the corresponding eye key points in the right eye corner area.

[0102] Specifically, the terminal can obtain the left and right eye key point pairs from the eye key point set corresponding to the face image according to the pre-set index of the left and right eye key points, and the left and right eye key point pairs include at least two. Then, the Euclidean distance algorithm is used to calculate the distance between the left eye key point and the right eye key point in the left and right eye key point pairs to obtain the pixel distance corresponding to the left and right eye key point pairs. Then, the terminal calculates the sum of the pixel distances corresponding to all left and right eye key point pairs in the face image, and then calculates the ratio of the sum to the number of left and right eye key point pairs to obtain the left and right average distance corresponding to the eye area in the face image, and then uses the left and right average distance as the eye width value to obtain the eye width value corresponding to the face image. The eye width value does not change with blinking.

[0103] In one embodiment, the terminal may calculate the eye width value corresponding to the right eye and / or the left eye, obtain the left and right eye key point pairs corresponding to the right eye and / or the left eye, and calculate the pixel distance corresponding to the left and right eye key point pairs. Finally, based on the pixel distance corresponding to the left and right eye key point pairs and the number of the left and right eye key point pairs, an average distance calculation is performed to obtain the eye width value corresponding to the right eye and / or the left eye.

[0104] In a specific embodiment, as Figure 5 shown, it is a schematic diagram of the eye key points in the right eye detected by the Mediapipe algorithm. Among them, the digital indices of 3 left and right eye key point pairs are obtained, including K6=(362,263), K7=(463,359), K8=(464,446). Among them, the left and right eye key point pair K6 includes the 362nd face key point and the 263rd face key point, and the same applies to other left and right eye key point pairs. Then, according to the digital indices of the left and right eye key point pairs, the pixel coordinates of the eye key points in the left and right eye key point pairs are determined, and the Euclidean distance between the two eye key points in each left and right eye key point pair is calculated using the pixel coordinates to obtain the pixel distances corresponding to the 3 left and right eye key point pairs, including L K6 , L K7 and L K8 . Then, the ratio of the sum of the terminal pixel distances to the total number of the left and right eye key point pairs is obtained to get the eye width value W=(L K6 +L K7 +L K8 ) / 3. It is easy to understand that in this example, the mouth height value is obtained from 3 left and right eye key point pairs. In an actual application scenario, the number of the left and right mouth key point pairs can be set customarily as needed. For example, the number of the left and right eye key point pairs is 5, and the number of the left and right eye key points is 4. Among them, the more the number of the left and right eye mouth key points, the more accurate the corresponding eye height value obtained.

[0105] In the above embodiment, by obtaining the left and right eye key point pairs and calculating the pixel distances corresponding to the left and right eye key point pairs. Finally, based on the pixel distances corresponding to the left and right eye key point pairs and the number of the left and right eye key point pairs, an average distance calculation is performed to obtain the eye width value corresponding to the face image, and the average distance is used as the eye width value, which improves the accuracy of the obtained eye width value.

[0106] In one embodiment, the blink change information includes the change trend of the eye region and the change difference of the eye region;

[0107] As Figure 6 shown, S310, that is, calculating the change of the eye region ratio of each face image to obtain the blink change information corresponding to the face image sequence, and determining the initial blink detection result corresponding to the face image sequence according to the blink change information, including the steps:

[0108] S602 , calculating a ratio change trend based on the eye region ratio of each face image to obtain an eye region change trend corresponding to the face image sequence.

[0109] S604: Determine a maximum eye area ratio and a minimum eye area ratio from the eye area ratios of each face image, calculate the difference between the maximum eye area ratio and the minimum eye area ratio, and obtain the eye area change difference corresponding to the face image sequence.

[0110] S606 , determining an initial blink detection result corresponding to the facial image sequence based on the eye region change trend and the eye region change difference.

[0111] The eye area change trend refers to the change in the eye area ratio, such as first decreasing and then increasing, or first increasing and then decreasing, or always decreasing, or always increasing, or remaining unchanged, etc. The eye area change difference refers to the maximum difference in the eye area ratio.

[0112] Specifically, the terminal determines the eye area change trend corresponding to the facial image sequence based on the size of the eye area ratio corresponding to each facial image. It then calculates the difference between the maximum eye area ratio and the minimum eye area ratio to obtain the eye area change difference. Finally, the terminal determines whether the eye area change trend conforms to a preset blink change trend and whether the eye area change difference exceeds a preset difference threshold. When the eye area change trend conforms to the preset blink change trend and the eye area change difference exceeds the preset difference threshold, an initial blink detection success result is obtained for the facial image sequence. When the eye area change trend does not conform to the preset blink change trend or the eye area change difference does not exceed the preset difference threshold, an initial blink detection failure result is obtained for the facial image sequence. The preset blink change trend refers to a preset change trend that indicates a successful initial blink detection result. The preset blink change trend may be one in which the eye area ratio first decreases and then increases. The preset difference threshold refers to a preset maximum difference threshold for the eye area ratios, which can be set as needed, for example, 0.045.

[0113] In one embodiment, the initial blink detection result corresponding to the sequence of face images can also be obtained by calculating the area change trend of the eye region. Specifically, the center point of the eye pupil and the key points on the edge of the eye region are obtained, and then two adjacent key points and the center point are combined to form a triangle, obtaining a triangular region. The sum of the areas of all triangular regions is calculated to obtain the area of the eye region corresponding to the face image. By traversing each face image, the area of the eye region corresponding to each face image is obtained, and then the eye area change trend and the eye area change difference are determined based on the size of the eye region area. The eye area change difference refers to the difference between the maximum eye region area and the minimum eye region area. The initial blink detection result corresponding to the sequence of face images is determined based on the eye area change trend and the eye area change difference.

[0114] In the above embodiment, the eye region change trend and the eye region change difference are calculated based on the size of the ratio of the eye region corresponding to each face image. Finally, the initial blink detection result corresponding to the sequence of face images is determined based on the eye region change trend and the eye region change difference, improving the accuracy of obtaining the initial blink detection result.

[0115] In one embodiment, the eye key points include the left eye key points and / or the right eye key points. That is, the terminal can perform blink detection on the right eye, or on the left eye, or on both the left and right eyes simultaneously. That is, the left eye key point set and / or the right eye key point set can be used for eye blink detection to obtain the initial blink detection result. Then, the average distance between the target intersection point and the target eye key points in the left eye key point set and / or the right eye key point set is calculated to determine the distance dispersion degree. Finally, the blink detection result of the left eye and / or the right eye is determined based on the initial blink detection result and the distance dispersion degree, that is, blink detection can be performed on any of the left and right eyes, improving the flexibility of blink detection.

[0116] In one embodiment, the target intersection point is calculated based on the target face key point pair. The face key points in the target face key point pair are the face key points in the set of face key points with a change degree less than the threshold during the blink process. The set of face key points is obtained by extracting key points from the face image.

[0117] In this embodiment, facial key point detection can be performed on each frame of facial image in the facial image sequence to obtain a set of facial key points of the facial image. Facial key point pairs with a change degree less than a threshold during the blinking process are determined from the set of facial key points as target facial key point pairs, and target intersection points are calculated based on the target facial key point pairs. Among them, the change degree is used to characterize the change situation of the target distance in each frame of facial image during the blinking process, and the target distance refers to the distance between the target facial key points in the target facial key point pair of the facial image. The target facial key points can be pre-determined and annotated with indexes, and the annotation indexes of the target facial key points of different facial images can be the same. The threshold is pre-set and can be determined according to human experience. The target facial key point pair can be two. During the blinking process, the change degree of the target facial key points in the target facial key point pair is less than the threshold, and the change degree of the target facial key points can be characterized by the change situation of the distance between the target facial key points, that is, the change situation of the distance between the target facial key points in the target facial key point pair during the blinking process is less than the threshold. Optionally, the server can determine the target facial key point pair of each frame of facial image from the set of facial key points according to the annotation index of the target facial key points. The server can also screen out the facial key points with a change degree less than the threshold during the blinking process from the set of facial key points as the target facial key point pair. Intersection point calculation is performed based on the straight line where the target facial key point pair of each frame of facial image is located to obtain the target intersection point of each frame of facial image. In one example, the server can determine the annotation index of the target facial key points according to the facial key points with a change degree less than the threshold in the historical facial image sequence. For example, the annotation indexes of the facial key points that are easy to locate can be screened from the facial key points of the historical facial image, such as the focus, intersection point, or corner point in the facial image, which actually correspond to the corners of the mouth, eyes, or nose, etc., and then the indexes of the facial key points with a change degree less than the threshold are screened from the facial key points that are easy to locate to obtain the annotation index of the target facial key points.

[0118] Optionally, any two key points can be selected from the set of facial key points in the facial image including the historical facial image sequence of the blinking process as the initial facial key points, and the distance between the two facial key points in the initial facial key point pair of each frame of facial image is calculated to obtain the change of the distance of the same initial facial key point pair in the historical facial image sequence. The initial facial key point pair with the distance change less than the threshold is used as the target facial key point pair, so as to determine the target facial key point pair of the facial image in the facial image sequence according to the annotation index of the target facial key point pair. That is to say, during the blinking process, the change of the distance between the facial key points in the target facial key point pair is less than the threshold. For example, during the blinking process, the difference between the maximum distance and the minimum distance between the facial key points in the target facial key point pair is less than the first threshold, or the variance of the distance between the facial key points in the target facial key point pair during the blinking process is less than the second threshold. In one embodiment, as Figure 7 shown, S204, that is, obtaining the target intersection corresponding to the facial image in the facial image sequence, includes:

[0119] S702, obtaining the quantization distance corresponding to the facial image in the facial image sequence. The quantization distance is obtained by obtaining two current facial key point pairs from the set of facial key points of the facial image, calculating the current distance corresponding to each current facial key point pair, and performing an average calculation based on the current distance. The current distance is the distance between the facial key points in the current facial key point pair;

[0120] S704, calculating the degree of dispersion based on the quantization distance of the facial images in the facial image sequence to obtain the quantization distance dispersion degree corresponding to the facial image sequence; when the quantization distance dispersion degree meets the target dispersion degree condition, the two current facial key point pairs that meet the target dispersion degree condition are used as the two target facial key point pairs.

[0121] Among them, the current facial key point pair refers to the facial key point pair for determining whether it is the target facial key point pair currently. The facial key points in the current facial key point pair can be arbitrarily selected from the facial key point set. The facial key points in the current facial key point pair can also be selected according to those from the focus of the facial image, and the focus can be key points at facial positions such as the corners of the mouth, the corners of the eyes, the tip of the nose, etc. The facial key points in the current facial key point pair can also be selected from the facial key points that are easily locatable in the facial key point set. The quantization distance dispersion is used to characterize the change in the quantization distance of the facial image during the blinking process and can be the variance of the quantization distance. When the quantization distance dispersion is smaller, the corresponding facial key point pair has a higher possibility of being the target facial key point pair; on the contrary, when the quantization distance dispersion is larger, the corresponding facial key point pair has a lower possibility of being the target facial key point pair. The target dispersion condition refers to the quantization distance dispersion condition for the current facial key point pair to be the target facial key point pair that is preset. The target dispersion condition can be that the quantization distance dispersion is less than the preset threshold, or that the quantization distance dispersion corresponding to the current facial key point pair is the smallest among the quantization distance dispersions corresponding to all facial key point pairs.

[0122] Specifically, the terminal can obtain two current face key point pairs from the face key point set of the face image, and then calculate the Euclidean distance between the two face key points in each current face key point pair. The Euclidean distance between the two face key points in each current face key point pair can be calculated using the pixel coordinates of the face key points, so as to obtain the current distance corresponding to each current face key point pair. Then calculate the sum of all the current distances of the current face key point pairs, and perform an average calculation on the sum of the current distances and the number of current distances to obtain the quantization distance of the face image. Among them, the number of current distances is the number of current face key point pairs, and each current face key point pair corresponds to a current distance. Then traverse and calculate the quantization distances of these two current face key point pairs in each face image during the blink process. The two current face key point pairs can be determined from each face image during the blink process according to the indexes of the face key points in the two current face key point pairs. At this time, the terminal uses the quantization distances of each face image in the face image sequence to calculate the degree of dispersion, and obtains the degree of dispersion of the quantization distance during the blink process, that is, the degree of dispersion of the quantization distance corresponding to the face image sequence. For example, the variance between the quantization distances of all face images during the blink process can be calculated to obtain the degree of dispersion of the quantization distance during the blink process. Then determine whether the degree of dispersion of the quantization distance during the blink process meets the target dispersion condition. When the target dispersion condition is not met, for example, the degree of dispersion of the quantization distance calculated by these two current face key point pairs is not less than the pre-set threshold or the degree of dispersion of the quantization distance calculated by these two current face key point pairs is not the smallest among the degrees of dispersion of the quantization distances calculated by all face key point pairs, it means that these two current face key point pairs are not the target face key pairs. When the degree of dispersion of the quantization distance calculated by the selected two current face key point pairs meets the target dispersion condition, the two current face key point pairs that meet the target dispersion condition are used as two target face key point pairs.

[0123] In a specific embodiment, the indexes for obtaining two current face key point pairs from the face key points detected by the Mediapipe algorithm in the face image include (4, 2), (64, 294), that is, the target face key point pair of the 4th face key point and the 2nd face key point and the target face key point pair of the 64th face key point and the 294th face key point. Then use formula (2) shown below to calculate the quantization distance.

[0124] Base = (R1 + R2) / 2 Formula (2)

[0125] Among them, Base refers to the quantization distance. R1 refers to the distance between the 4th facial key point and the 2nd facial key point. R2 refers to the distance between the 64th facial key point and the 294th facial key point. Then, calculate the quantization distance of all facial images during the blinking process, that is, use the indices (4, 2), (64, 294) of the facial key point pairs to obtain the pixel coordinates of the facial key points with the same indices in all facial images, then calculate the distance between the facial key points in the facial key point pairs according to the pixel coordinates, and then perform an average calculation to obtain the quantization distance of all facial images.

[0126] S706. Perform a straight-line calculation based on the two target facial key point pairs to obtain a straight line for each target facial key point pair, and perform an intersection calculation based on the straight line of each target facial key point pair to obtain the target intersection corresponding to the facial image in the facial image sequence.

[0127] Among them, the straight line is used to represent the straight line formed by two facial key points in the target facial key point pair.

[0128] Specifically, the terminal uses the two facial key points in each target facial key point pair of the facial image to calculate the straight-line slope and the straight-line intercept, determines the straight line corresponding to each facial key point pair according to the straight-line slope and the straight-line intercept, so as to obtain two straight lines corresponding to the facial image. Then use the method of calculating the intersection of two straight lines to calculate the intersection corresponding to the two straight lines, so as to obtain the target intersection of the facial image. In one embodiment, the terminal can also use the vector cross product method to calculate the target intersection. Then calculate the target intersection corresponding to each facial image according to the two target facial key pairs of each facial image during the blinking process.

[0129] In the above embodiment, by calculating the dispersion degree of the quantization distance of two facial key point pairs during the blinking process, two target facial key point pairs are screened to improve the accuracy of the obtained two target facial key point pairs, and then the two target facial key point pairs are used to calculate the target intersection corresponding to the facial image, so as to improve the accuracy of the obtained target intersection.

[0130] In one embodiment, S204. Obtaining the target intersection corresponding to the facial image in the facial image sequence includes the steps of:

[0131] Obtain each candidate intersection corresponding to the facial image in the facial image sequence. The candidate intersection is calculated according to two candidate facial key point pairs, and the facial key points in each candidate facial key point pair are determined from the facial key point set according to the positions of the facial key points;

[0132] Obtain the candidate distance dispersion corresponding to each candidate intersection point. The candidate distance dispersion is calculated based on the candidate quantization distances corresponding to the face images in the face image sequence. The candidate quantization distance is obtained by calculating the candidate distances corresponding to each pair of candidate face key points and then performing an average calculation based on the candidate distances. The candidate distance is the distance between the face key points in the pair of candidate face key points.

[0133] Determine the minimum candidate distance dispersion from the candidate distance dispersions corresponding to each candidate intersection point, and use the candidate intersection point corresponding to the minimum candidate distance dispersion as the target intersection point corresponding to the face image in the face image sequence.

[0134] Among them, the candidate intersection point is the intersection point to be screened. The candidate intersection point is calculated based on two pairs of candidate face key points. The face key points in the pair of candidate face key points can be key points that are easy to locate. For example, the face key point at the corner of the mouth is used as the face key point in the pair of candidate face key points. The face key points in the pair of candidate face key points can also be determined according to the index of the pre-set face key points.

[0135] Specifically, the terminal obtains two pairs of candidate face key points, then calculates the corresponding two straight lines based on the two pairs of candidate face key points, and then calculates the intersection point of the two straight lines to obtain the candidate intersection point. The terminal can determine the two pairs of candidate face key points corresponding to each candidate intersection point to be calculated from the set of face key points of the face image, so as to calculate each candidate intersection point corresponding to the face image. Then the terminal can directly obtain the candidate distance dispersion corresponding to each candidate intersection point. The terminal can also calculate the distance between the face key points in the two pairs of candidate face key points corresponding to the face image, and then perform an average calculation to obtain the candidate quantization distance corresponding to the face image. Then calculate the candidate quantization distances obtained when calculating the two pairs of candidate face key points with the same face key point index for all face images during the blinking process, and then calculate the dispersion of all candidate quantization distances during the blinking process, that is, calculate the variance of all candidate quantization distances during the blinking process. Finally, use the candidate intersection point corresponding to the minimum candidate distance dispersion as the target intersection point corresponding to the face image in the face image sequence.

[0136] In the above embodiment, by directly obtaining each candidate intersection point and then determining the target intersection point from each candidate intersection point according to the candidate distance dispersion of the candidate intersection point, the efficiency of obtaining the target intersection point can be improved.

[0137] In one embodiment, S206, that is, determining the average distance corresponding to each face image according to the target intersection point and each target eye key point in the set of face key points, includes the steps:

[0138] Calculate the point distance between the target intersection point and each target eye key point; based on the point distance and the number of target eye key points in the face key point set, perform an average distance calculation to obtain the average distance corresponding to the face image.

[0139] Specifically, the terminal calculates the pixel distance between the target intersection point and each target eye key point in the eye key point set respectively, to obtain the point distance between the target intersection point and the target eye key point, then calculates the ratio of the sum of all point distances to the number of target eye key points, to obtain the average distance between the target intersection point and the target eye key point, and then uses this average distance to characterize the blink detection stability corresponding to the face image.

[0140] In a specific embodiment, determine the key point pairs for which the point distance is to be calculated according to the indexes of the preset target eye key points, including B1=(226, R), B2=(130, R), B3=(33, R), B4=(133, R), B5=(243, R), B6=(244, R), where R refers to the target intersection point, and the key point pair B1 includes the 226th face key point and the target intersection point, and the same applies to other key point pairs. Then use the following formula (3) to calculate the average distance.

[0141] M = (L B1 +L B2 +L B3 +L B4 +L B5 +L B6 ) / 6 Formula (3).

[0142] Wherein, M refers to the blink detection stability, that is, the average distance. L B1 refers to the Euclidean distance of the key point pair B1. L B2 refers to the Euclidean distance of the key point pair B2. L B3 refers to the Euclidean distance of the key point pair B3. L B4 refers to the Euclidean distance of the key point pair B4. L B5 refers to the Euclidean distance of the key point pair B5. L B6 refers to the Euclidean distance of the key point pair B6. That is, by calculating the point distance between the target intersection point and all target eye key points, and then performing an average calculation, the average distance corresponding to the face image is obtained, and the stability during the blink process is characterized by the average distance, so as to improve the accuracy of blink detection.

[0143] In an embodiment, the face images in the face image sequence are continuous. S208, that is, calculate the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the face image sequence, including the steps:

[0144] Obtain the quantization distance corresponding to the face image in the face image sequence, calculate the ratio of the average distance corresponding to each face image to the corresponding quantization distance to obtain the distance quantization value corresponding to each face image, and calculate the degree of dispersion of the quantization values corresponding to each face image to obtain the degree of dispersion of the quantization values corresponding to the face image sequence; obtain the ratio of the eye regions corresponding to the face images in the face image sequence, calculate the difference between adjacent eye region ratios in the ratio of the eye regions corresponding to each face image, and calculate the degree of dispersion of the differences between adjacent eye region ratios to obtain the degree of dispersion of the region ratios corresponding to the face image sequence; use the degree of dispersion of the quantization values and the degree of dispersion of the region ratios as the degree of dispersion of the distances corresponding to the face image sequence.

[0145] Specifically, the terminal can obtain the quantization distance corresponding to each face image, and this quantization distance is the average of the sum of the distances corresponding to the target face key points. Then, parallelly calculate the ratio of the average distance of the face image to the quantization distance to obtain the distance quantization value corresponding to each face image, and then calculate the variance of all the distance quantization values to obtain the degree of dispersion of the quantization values corresponding to the face image sequence. That is, use the quantization distance to perform normalization calculation on the average distance and then perform variance calculation, which further improves the accuracy of the degree of dispersion of the quantization values. Then the terminal obtains the ratio of the eye regions corresponding to the face images in the face image sequence, then calculates the difference between adjacent eye region ratios in the ratio of the eye regions of the face image, and calculates the degree of dispersion of all the differences, that is, the variance, to obtain the degree of dispersion of the region ratios corresponding to the face image sequence. The terminal uses the degree of dispersion of the quantization values and the degree of dispersion of the region ratios as the degree of dispersion of the distances corresponding to the face image sequence, and then judges that when the degree of dispersion of the quantization values is less than the preset stability threshold and the degree of dispersion of the region ratios is less than the preset stability threshold, the result of passing the blink stability detection corresponding to the face image sequence is obtained. When the degree of dispersion of the quantization values is not less than the preset stability threshold or the degree of dispersion of the region ratios is not less than the preset stability threshold, the result of failing the blink stability detection corresponding to the face image sequence is obtained. The preset stability threshold refers to the preset threshold of the degree of dispersion, and can take the value of 0.01. The preset stability threshold corresponding to the degree of dispersion of the quantization values can be the same as the preset stability threshold corresponding to the degree of dispersion of the region ratios, or can be different, and can be set according to requirements.

[0146] In a specific embodiment, the distance quantization value can be calculated using the formula Q = M / base. By traversing each face image, the distance quantization value corresponding to each face image is obtained, that is, a list of distance quantization values Lq is obtained, Lq = [Q1, Q2, …, Qn], where n is the number of face images, a positive integer, Q1 refers to the distance quantization value corresponding to the first frame of the face image sequence, Q2 refers to the distance quantization value corresponding to the first frame of the face image sequence, and Qn refers to the distance quantization value corresponding to the first frame of the face image sequence. Then, it is determined whether the variance of Lq is close to 0 or less than the threshold Tq. At the same time, the variance of the difference in the ratio of adjacent eye regions is calculated, and then it is determined whether the variance is close to 0 or less than the threshold Tr. The normal blinking process is a roughly uniform process without sudden changes. Therefore, for the ratio of eye regions that satisfies the blinking logic, the change between adjacent values is not too large, that is, the variance of the difference between adjacent ratios of eye regions is close to 0 or less than the threshold Tr. At this time, the blinking stability detection result can be determined according to the judgment result.

[0147] In the above embodiment, the discrete degree of the quantization value corresponding to the face image training and the discrete degree of the region ratio are determined by calculating the distance quantization value and the difference in the ratio of adjacent eye regions. Finally, the discrete degree of the quantization value and the discrete degree of the region ratio are used as the distance discrete degree corresponding to the face image sequence, that is, the discrete degree of the quantization value and the discrete degree of the region ratio are used to reflect the stability during the blinking process, further improving the accuracy of blinking detection.

[0148] In a specific embodiment, as Figure 8 shown, a flowchart of blinking detection is provided. Specifically: The terminal obtains a video or an image frame sequence to be subjected to blinking detection. Then, face key point detection is performed through the Mediapipe algorithm to obtain the position coordinates of 478 face key points corresponding to each face image. Then, the normalized Base is calculated. Among them, the Base value is first calculated, that is, the quantization distance corresponding to the face image. Then, the target face key point pair is obtained. The target face key point pair can be the face key points in the mouth region or the face key points in the nose tip region, which can be set according to requirements. At this time, the target face key point is the face key point in the nose tip region. As Figure 9 shown, it is a schematic diagram of face key points used for blinking detection. The indexes (4, 2) and (64, 294) of two target face key point pairs are obtained. These two target face key point pairs are determined by comparing the variances of the Base values of multiple different pairs of two face key points during the blinking process and selecting the pair of two face key points with the smallest variance. According to the target face key point pair indexes 4 and 2, the position coordinates of face key points 4 and 2 are found from the key point position coordinates. The distance between face key points 4 and 2 is calculated according to the position coordinates, that is, the length of R1, denoted as LR1 , then according to the target face key points, the position coordinates of face key points 64 and 294 are found from the key point position coordinates for indices 64 and 294. Calculate the distance between face key points 64 and 294 according to the position coordinates, that is, the length of R2, denoted as L R2 . At this time, first calculate the quantization distance Base = (L R1 + L R2 ) / 2. Then calculate the position coordinates of the target intersection point R of R1 and R2.

[0149] The terminal calculates the stable quantization index. Obtain the combination of the target eye key points of the left eye and the target intersection point, including B1 = (226, R), B2 = (130, R), B3 = (33, R), B4 = (133, R), B5 = (243, R), B6 = (244, R). Then calculate the distance between each target eye key point and the target intersection point according to the position coordinates, and get L B1 to L B6 . Then calculate the average distance M L = (L B1 + L B2 + L B3 + L B4 + L B5 + L B6 ) / 6, where L B1 represents the distance between the eye key point 226 and the target intersection point R in the target eye key point pair B1 = (226, R) of the left eye, L B2 represents the distance between the eye key point 130 and the target intersection point R in the target eye key point pair B2 = (130, R) of the left eye, and so on for the distances between other target eye key points of the left eye and the target intersection point. Finally, take this average distance as the stable quantization index of the left eye. Then the terminal can calculate the stability quantization index M R of the right eye in the same way.

[0150] Then the terminal calculates the H / W ratio of the eye region, that is, the height-width ratio. The terminal obtains the pre-set eye key point pairs, including S1 = (161, 163), S2 = (160, 144), S3 = (159, 145), S4 = (158, 153), S5 = (157, 154), S6 = (226, 244), S7 = (130, 243), S8 = (33, 133). Then calculate the distances between the eye key points according to the position coordinates, and get L S1 to L S8 , and then calculate the average distance height H L and the average distance width W L of the left eye. H L = (LS1 +L S2 +L S3 +L S4 +L S5 ) / 5 and W L = (L S6 +L S7 +L S8 ) / 3. Finally, calculate the aspect ratio Ratio of the left eye based on HL and WL L , and the terminal can calculate the aspect ratio Ratio of the right eye in the same way at the same time R .

[0151] Traverse and calculate each face image to obtain a list M of the left-eye stability quantization indicators corresponding to each face image list-l and a list Ratio of the aspect ratios of the left eye list-l . Where M list-l = L(M L1 , M L2, …, M Ln ), Ratio list-l = L(Ratio L1 , Ratio L2 ,…, Ratio Ln ). Among them, M L1 represents the first average distance of the first frame of the face image sequence, M L2 represents the first average distance of the second frame of the face image sequence, M Ln represents the first average distance of the nth frame of the face image sequence, Ratio L1 represents the aspect ratio of the left eye of the first frame of the face image sequence, Ratio L2 represents the aspect ratio of the left eye of the second frame of the face image sequence, Ratio Ln represents the aspect ratio of the left eye of the nth frame of the face image sequence. Then the terminal makes a preliminary blink judgment, that is, according to Ratio list-l and the predefined number of blink judgment image frames m, obtain the last m values of the aspect ratio list Ratio list-l , and judge whether the change trend of these m values satisfies the trend of first decreasing and then increasing, and whether the difference between the maximum value and the minimum value of these m values is greater than the threshold, and the threshold value is 0.045. If both are satisfied, it is judged as a successful result of the initial blink detection. If one of the conditions is not satisfied, it is judged as a failed result of the initial blink detection. The terminal can make a judgment on the right eye in the same way at the same time to obtain the initial blink detection result of the right eye

[0152] If it is judged as a successful result of the initial blink detection, at this time, according to M list-lThe list M of the normalized left-eye stability quantization metrics calculated with the Base value list-l-1 , then calculate the variance of the last m normalized left-eye stability quantization metrics, and determine whether it is less than the threshold of 0.01. Also, based on the last m normalized left-eye stability quantization metrics, determine whether the variance of the difference between every two adjacent values is less than the threshold of 0.01. Thus, the result of the blink stability judgment is obtained. The terminal can judge the right eye in the same way simultaneously to obtain the blink stability judgment result of the right eye. At this time, if the blink stability judgment passes, a successful blink detection result is obtained, and other subsequent processes are performed, such as face recognition, etc. Otherwise, it is prompted that the blink is not successful or continue to wait for a blink.

[0153] In a specific embodiment, as Figure 10 shown, a functional block diagram of blink detection is provided. Specifically: The terminal obtains the input RGB face image, performs key point detection on the face image to obtain the position coordinates of the face key points in the face image, and then performs blink judgment. When performing blink judgment, first calculate the normalized base, then calculate the stability quantization metrics, and then calculate the aspect ratio of the eye region height to width. At this time, perform normalized calculation on the stability quantization metrics, and use the aspect ratio of the eye region height to width for preliminary blink judgment. And after the blink judgment is successful, continue to perform blink stability judgment. And after the blink stability judgment is completed, perform subsequent processing, such as prompting that the blink is not successful or continue to wait for a blink when the blink detection fails, or prompting that the blink is successful and performing subsequent processing when the blink detection is successful. By normalizing the Base and the stability quantization metrics, the blink stability judgment is performed on this basis, thereby effectively improving the robustness of the blink judgment. At the same time, no additional occlusion detection model is introduced, and the actual overhead of the algorithm is small, and it can be conveniently deployed on the terminal, especially the mobile terminal. That is, without increasing too much additional computing power, the blink detection accuracy and effect are improved, and the efficiency of blink detection is increased.

[0154] In a specific embodiment, the blink detection method is applied to the payment process of a smart phone. Specifically, when a user makes a smart payment through the smart phone, the user can first determine whether it is the user himself / herself making the payment through blink detection. At this time, the smart phone's camera captures a sequence of face images of the user during the blink process, and then an initial blink detection result of the sequence of face images is obtained. At this time, the target intersection points corresponding to the face images in the sequence of face images are calculated, and based on the target intersection points and each target eye key point in the face key point set, the average distance corresponding to each face image is determined. Finally, the distance dispersion degree corresponding to the sequence of face images is calculated by calculating the distance dispersion degree of the average distance corresponding to each face image. The distance dispersion degree includes a quantization value dispersion degree and a region ratio dispersion degree. Then, when it is determined that the initial blink detection result is an initial blink detection success result and both the quantization value dispersion degree and the region ratio dispersion degree are less than a pre-set stability threshold, a blink detection success result of the sequence of face images is obtained. At this time, the smart phone determines that it is the user himself / herself making the smart payment and then executes the payment process.

[0155] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0156] Based on the same inventive concept, the embodiments of the present application also provide a blink detection device for implementing the above-mentioned blink detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the blink detection device provided below can refer to the limitations on the blink detection method in the above text and will not be repeated here.

[0157] In an exemplary embodiment, as Figure 11 shown, a blink detection device 1100 is provided, including: a blink detection module 1102, an intersection point acquisition module 1104, a distance calculation module 1106, a dispersion degree calculation module 1108, and a blink result obtaining module 1110, where:

[0158] A blink detection module 1102, configured to obtain an initial blink detection result of a sequence of face images;

[0159] An intersection point acquisition module 1104, configured to obtain a target intersection point corresponding to a face image in a sequence of face images, where the target intersection point is used to characterize the change degree of target face key points during a blink;

[0160] A distance calculation module 1106, configured to determine an average distance corresponding to each face image according to the target intersection point and each target eye key point in a set of face key points;

[0161] A dispersion degree calculation module 1108, configured to calculate a distance dispersion degree corresponding to a sequence of face images by calculating a distance dispersion degree of the average distance corresponding to each face image;

[0162] A blink result obtaining module 1110, configured to obtain a blink detection result of a sequence of face images based on the initial blink detection result and the distance dispersion degree.

[0163] In one embodiment, the blink detection module 1102 is further configured to obtain a sequence of face images, perform face key point detection on the face images in the sequence of face images to obtain a set of face key points of the face images; determine a set of eye key points corresponding to a face image from the set of face key points, and perform eye blink detection based on the eye key points in the set of eye key points to obtain an initial blink detection result of the sequence of face images.

[0164] In one embodiment, the blink detection module 1102 is further configured to determine a set of eye key points corresponding to a face image from the set of face key points according to the correspondence between eye key point identifiers and eye key points; perform an eye height value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain an eye height value corresponding to the face image, perform an eye width value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain an eye width value corresponding to the face image; calculate a ratio of the eye height value to the eye width value to obtain an eye area ratio corresponding to the face image; traverse and calculate the eye area ratios of each face image in the sequence of face images, and calculate the change of the eye area ratio of each face image to obtain blink change information corresponding to the sequence of face images, and determine the initial blink detection result corresponding to the sequence of face images according to the blink change information.

[0165] In one embodiment, the blink detection module 1102 is further configured to obtain an upper and lower eye key point pair corresponding to a face image from the set of eye key points corresponding to the face image, and calculate a pixel distance between the upper eye key point and the lower eye key point in the upper and lower eye key point pair; perform an average distance calculation based on the pixel distance of the upper and lower eye key point pair and the number of the upper and lower eye key point pairs to obtain an eye height value corresponding to the face image.

[0166] In one embodiment, the blink detection module 1102 is further configured to obtain a pair of left and right eye key points corresponding to the face image from the set of eye key points corresponding to the face image, and calculate the pixel distance between the left eye key point and the right eye key point in the pair of left and right eye key points; perform an average distance calculation based on the pixel distance of the pair of left and right eye key points and the number of pairs of left and right eye key points to obtain the eye width value corresponding to the face image.

[0167] In one embodiment, the blink change information includes the change trend of the eye region and the change difference of the eye region;

[0168] The blink detection module 1102 is further configured to calculate the change trend of the eye region corresponding to the face image sequence based on the eye region ratio of each face image; determine the maximum eye region ratio and the minimum eye region ratio from the eye region ratios of each face image, calculate the difference between the maximum eye region ratio and the minimum eye region ratio to obtain the change difference of the eye region corresponding to the face image sequence; determine the initial blink detection result corresponding to the face image sequence based on the change trend of the eye region and the change difference of the eye region.

[0169] In one embodiment, the intersection point obtaining module 1104 is further configured to obtain the quantization distance corresponding to the face image in the face image sequence. The quantization distance is obtained by obtaining two pairs of current face key points from the set of face key points of the face image, calculating the current distance corresponding to each pair of current face key points, and performing an average calculation based on the current distance. The current distance is the distance between the face key points in the pair of current face key points. Perform a dispersion calculation based on the quantization distance of the face images in the face image sequence to obtain the quantization distance dispersion corresponding to the face image sequence; when the quantization distance dispersion meets the target dispersion condition, use the two pairs of current face key points that meet the target dispersion condition as two pairs of target face key points, perform a straight line calculation according to the two pairs of target face key points to obtain the straight line of each pair of target face key points, and perform an intersection point calculation based on the straight line of each pair of target face key points to obtain the target intersection point corresponding to the face image in the face image sequence.

[0170] In one embodiment, the intersection point acquisition module 1104 is further configured to acquire each candidate intersection point corresponding to the face image in the face image sequence, where the candidate intersection point is calculated according to two candidate face key point pairs, and the face key points in each candidate face key point pair are determined from the set of face key points according to the positions of the face key points; acquire the candidate distance dispersion degree corresponding to each candidate intersection point, where the candidate distance dispersion degree is calculated based on the candidate quantization distances corresponding to the face images in the face image sequence, and the candidate quantization distance is calculated by calculating the candidate distance corresponding to each candidate face key point pair and performing an average calculation based on the candidate distances, and the candidate distance is the distance between the face key points in the candidate face key point pair; determine the minimum candidate distance dispersion degree from the candidate distance dispersion degrees corresponding to each candidate intersection point, and use the candidate intersection point corresponding to the minimum candidate distance dispersion degree as the target intersection point corresponding to the face image in the face image sequence.

[0171] In one embodiment, the distance calculation module 1106 is further configured to calculate the point distance between the target intersection point and each target eye key point; perform an average distance calculation based on the point distance and the number of target eye key points in the set of face key points to obtain the average distance corresponding to each face image.

[0172] In one embodiment, the dispersion degree calculation module 1108 is further configured to acquire the quantization distance corresponding to the face image in the face image sequence, calculate the ratio of the average distance corresponding to each face image to the corresponding quantization distance to obtain the distance quantization value corresponding to each face image, and perform a quantization value dispersion degree calculation on the distance quantization value corresponding to each face image to obtain the quantization value dispersion degree corresponding to the face image sequence; acquire the eye region ratio corresponding to the face image in the face image sequence, calculate the difference between adjacent eye region ratios in the eye region ratio corresponding to each face image, and perform a dispersion degree calculation on the difference between adjacent eye region ratios to obtain the region ratio dispersion degree corresponding to the face image sequence; use the quantization value dispersion degree and the region ratio dispersion degree as the distance dispersion degree corresponding to the face image sequence.

[0173] Each module in the above blink detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form so that the processor can call and execute the operations corresponding to each of the above modules.

[0174] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 12As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store face images, face key point indexes, eye key point indexes, blink detection result data, and so on. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a blink detection method.

[0175] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 13 shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a blink detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0176] Those skilled in the art can understand that Figure 12 or Figure 13The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0177] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0179] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0181] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0182] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0183] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A blink detection method, characterized in that, The method includes: Obtaining an initial blink detection result of a sequence of face images; Obtaining a target intersection point corresponding to a face image in the sequence of face images, where the target intersection point is used to characterize the change degree of target face key points during a blink; Determining an average distance corresponding to each face image according to the target intersection point and each target eye key point in a set of face key points; Calculating the distance dispersion degree of the average distance corresponding to each face image to obtain the distance dispersion degree corresponding to the sequence of face images; Obtaining a blink detection result of the sequence of face images based on the initial blink detection result and the distance dispersion degree.

2. The method according to claim 1, wherein The obtaining of the initial blink detection result of the sequence of face images includes: Obtaining a sequence of face images, performing face key point detection on the face images in the sequence of face images to obtain a set of face key points of the face images; Determining a set of eye key points corresponding to the face images from the set of face key points, and performing eye blink detection based on the eye key points in the set of eye key points to obtain the initial blink detection result of the sequence of face images.

3. The method according to claim 2, wherein The determining of the set of eye key points corresponding to the face images from the set of face key points, and performing eye blink detection based on the eye key points in the set of eye key points to obtain the initial blink detection result of the sequence of face images includes: Determining the set of eye key points corresponding to the face images from the set of face key points according to the correspondence between eye key point identifiers and eye key points; Performing eye high value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain the eye high value corresponding to the face image; Performing eye width value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain the eye width value corresponding to the face image; Calculating the ratio of the eye high value to the eye width value to obtain the eye region ratio corresponding to the face image; Traversing and calculating the eye region ratios of each face image in the sequence of face images, calculating the change of the eye region ratio of each face image to obtain the blink change information corresponding to the sequence of face images, and determining the initial blink detection result corresponding to the sequence of face images according to the blink change information.

4. The method according to claim 3, characterized in that, The performing of eye high value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain the eye high value corresponding to the face image includes: Obtaining the pair of upper and lower eye key points corresponding to the face image from the set of eye key points corresponding to the face image, and calculating the pixel distance between the upper eye key point and the lower eye key point in the pair of upper and lower eye key points; Performing average distance calculation based on the pixel distance of the pair of upper and lower eye key points and the number of the pair of upper and lower eye key points to obtain the eye high value corresponding to the face image.

5. The method according to claim 3, characterized in that, The performing of eye width value calculation based on the eye key points in the set of eye key points corresponding to the face image to obtain the eye width value corresponding to the face image includes: Obtain the left and right eye key point pairs corresponding to the face image from the set of eye key points corresponding to the face image, and calculate the pixel distance between the left eye key point and the right eye key point in the left and right eye key point pairs; Based on the pixel distance of the left and right eye key point pairs and the number of the left and right eye key point pairs, perform an average distance calculation to obtain the eye width value corresponding to the face image.

6. The method according to claim 3, characterized in that, The blink change information includes the change trend and change difference of the eye region; Calculating the change of the eye region ratio of each face image to obtain the blink change information corresponding to the face image sequence, and determining the initial blink detection result corresponding to the face image sequence according to the blink change information, including: Based on the eye region ratio of each face image, perform a ratio change trend calculation to obtain the change trend of the eye region corresponding to the face image sequence; Determine the maximum eye region ratio and the minimum eye region ratio from the eye region ratios of each face image, and calculate the difference between the maximum eye region ratio and the minimum eye region ratio to obtain the change difference of the eye region corresponding to the face image sequence; Based on the change trend of the eye region and the change difference of the eye region, determine the initial blink detection result corresponding to the face image sequence.

7. The method according to claim 1, characterized in that, The target intersection point is calculated according to the target face key point pair. The face key points in the target face key point pair are the face key points in the set of face key points with a change degree less than the threshold during the blink process. The set of face key points is obtained by extracting key points from the face image.

8. The method according to claim 7, wherein The obtaining of the target intersection point corresponding to the face image in the face image sequence includes: Obtain the quantization distance corresponding to the face image in the face image sequence. The quantization distance is obtained by extracting two current face key point pairs from the set of face key points of the face image, calculating the current distance corresponding to each current face key point pair, and performing an average calculation based on the current distance. The current distance is the distance between the face key points in the current face key point pair; Based on the quantization distance of the face images in the face image sequence, perform a dispersion calculation to obtain the quantization distance dispersion corresponding to the face image sequence; When the quantization distance dispersion meets the target dispersion condition, use the two current face key point pairs that meet the target dispersion condition as two target face key point pairs; According to the two target face key point pairs, perform a straight line calculation to obtain the straight line of each target face key point pair, and based on the straight lines of each target face key point pair, perform an intersection point calculation to obtain the target intersection point corresponding to the face image in the face image sequence.

9. The method according to claim 1, characterized in that, The obtaining of the target intersection point corresponding to the face image in the face image sequence includes: Obtain each candidate intersection point corresponding to the face image in the face image sequence. The candidate intersection point is calculated according to two candidate face key point pairs. The face key points in each candidate face key point pair are determined from the set of face key points according to the positions of the face key points; Obtain the candidate distance dispersion corresponding to each candidate intersection point, where the candidate distance dispersion is calculated based on the candidate quantization distances corresponding to the face images in the face image sequence. The candidate quantization distance is obtained by calculating the candidate distances corresponding to each pair of candidate face key points and performing an average calculation based on the candidate distances. The candidate distance is the distance between the face key points in the pair of candidate face key points. Determine the minimum candidate distance dispersion from the candidate distance dispersions corresponding to each candidate intersection point, and use the candidate intersection point corresponding to the minimum candidate distance dispersion as the target intersection point corresponding to the face image in the face image sequence.

10. The method according to claim 1, wherein The determining the average distance corresponding to each face image according to the target intersection point and each target eye key point in the face key point set includes: Calculate the point distance between the target intersection point and each target eye key point. Perform an average distance calculation based on the point distance and the number of target eye key points in the face key point set to obtain the average distance corresponding to each face image.

11. The method according to claim 1, wherein The face images in the face image sequence are consecutive. The calculating the distance dispersion corresponding to the face image sequence by performing a distance dispersion calculation on the average distance corresponding to each face image includes: Obtain the quantization distance corresponding to the face image in the face image sequence, calculate the ratio of the average distance corresponding to each face image to the corresponding quantization distance to obtain the distance quantization value corresponding to each face image, and perform a quantization value dispersion calculation on the distance quantization value corresponding to each face image to obtain the quantization value dispersion corresponding to the face image sequence. Obtain the eye region ratio corresponding to the face image in the face image sequence, calculate the difference between adjacent eye region ratios in the eye region ratios corresponding to each face image, and perform a dispersion calculation on the difference between the adjacent eye region ratios to obtain the region ratio dispersion corresponding to the face image sequence. Use the quantization value dispersion and the region ratio dispersion as the distance dispersion corresponding to the face image sequence.

12. A blink detection device, characterized in that, The device includes: A blink detection module for obtaining an initial blink detection result of the face image sequence. An intersection point obtaining module for obtaining the target intersection point corresponding to the face image in the face image sequence, where the target intersection point is used to characterize the change degree of the target face key points during the blink process. A distance calculation module for determining the average distance corresponding to each face image according to the target intersection point and each target eye key point in the face key point set. A dispersion calculation module for performing a distance dispersion calculation on the average distance corresponding to each face image to obtain the distance dispersion corresponding to the face image sequence. A blink result obtaining module for obtaining the blink detection result of the face image sequence based on the initial blink detection result and the distance dispersion.

13. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images