A method, device, electronic device and storage medium for face detection and tracking

Through the tracking method based on five key points of the face, the L-K optical flow algorithm and key point threshold judgment are used to solve the problems of face frames and loss and jitter in the existing technology, and a more stable face tracking effect is achieved.

CN113221841BActive Publication Date: 2024-11-08UNISOUND SHANGHAI INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110616942.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-02
Publication Date
2024-11-08
Estimated Expiration
2041-06-02

AI Technical Summary

Technical Problem

The existing face detection system can easily lead to face frame loss and jitter during the tracking process, especially when tracking a few points fails, resulting in inaccurate tracking results.

Method used

Using the tracking method based on five key points of the face, the key point position is tracked through the L-K optical flow algorithm, and good points and bad points are judged based on the key point threshold, and the offset is calculated to update the bad point coordinates and face frame coordinates to ensure tracking stability.

Benefits of technology

It improves the stability of face tracking, reduces the jitter and loss of face frames, and makes the tracking results more accurate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113221841B_ABST
    Figure CN113221841B_ABST
Patent Text Reader

Abstract

The present invention relates to a method, device, electronic device and storage medium for face detection and tracking, the method comprising: obtaining a video frame; judging whether the current video frame meets the conditions of the first frame of the video or the continuous tracking of four frames; if the conditions of the first frame of the video or the continuous tracking of four frames are met, detecting the first face frame coordinates and the position coordinates of the five key points of the first face; if the conditions of the first frame of the video or the continuous tracking of four frames are not met, tracking the position coordinates of the five key points of the second face according to the position coordinates of the five key points of the first face, and updating the second face frame coordinates. In the embodiment of the present application, the five key points of the face are used as the tracking target, so it is not easy to lose the tracking. The position coordinates of the five key points of the first face are obtained every five detections and then tracked. The position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face, and the second face frame coordinates are updated, so that the tracked second face frame coordinates are more stable and will not shake.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of face detection and tracking, and in particular to a method, device, electronic equipment and storage medium for face detection and tracking. Background Art

[0002] At present, the existing face detection system consists of two main modules: face detection and face tracking. First, the face is detected by the face detection model, and the face detection frame coordinates (x, y, w, h) are initialized. The tracking target target = (x, y, w, h) is initialized according to the detected face frame coordinates. The tracking module is called in subsequent frames to update the detection frame coordinates target = (x', y', w', h').

[0003] Existing detection-based face tracking solutions usually use two points on the diagonal of the face frame as the target of face tracking. Since these two points on the diagonal of the face frame are not obvious relative to the surrounding features, the face tracking results are often inaccurate and easy to lose. In the solution that uses five key points of the face to track, the tracking results of the five points are inconsistent, which often causes the face frame after tracking to shake. In particular, when the tracking of a few points fails, it is easy to cause the tracked face frame to deform. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for face detection and tracking, which can solve the above-mentioned technical problems of easy loss of tracking and deformation of the tracked face frame.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for face detection and tracking, comprising:

[0007] Get video frames;

[0008] Determine whether the current video frame meets the conditions of the first video frame or four consecutive tracking frames;

[0009] If the conditions of the first frame of the video or four consecutive tracking frames are met, the coordinates of the first face frame and the position coordinates of the five key points of the first face are detected;

[0010] If the conditions of the first frame of the video or four consecutive frames are not met, the position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face to update the coordinates of the second face frame.

[0011] In some embodiments, tracking the position coordinates of the five key points of the second face according to the position coordinates of the five key points of the first face and updating the coordinates of the second face frame includes:

[0012] Track the position coordinates of the five key points of the second face according to the LK optical flow algorithm to obtain the position coordinates of the five key points of the second face after tracking;

[0013] Determine good points and bad points in the coordinates of five key points of the tracked second face according to the key point threshold;

[0014] Determine the average value of the difference between the coordinates corresponding to the good point and the position coordinates of the five key points of the second face; the average value is the offset before and after tracking;

[0015] Update the coordinates of the bad point and the second face frame coordinates according to the offset.

[0016] In some embodiments, the above method further comprises:

[0017] If the position coordinates of the five key points of the tracked face are all bad points, re-detect the key points of the face.

[0018] In some embodiments, the method of determining good points and bad points in the coordinates of five key points of the tracked second face according to the key point threshold includes:

[0019] The coordinates of the five key points of the second face that are greater than the first key point threshold and less than the second key point threshold are good points;

[0020] The coordinates of the five key points of the second face that are smaller than the first key point threshold or larger than the second key point threshold are bad points;

[0021] Among them, the first key point threshold is smaller than the second key point threshold.

[0022] In some embodiments, in the above method, updating the coordinates of the bad point and the coordinates of the second face frame according to the offset includes:

[0023] The coordinates before bad point tracking are added with the offset to obtain the updated bad point coordinates and the second face frame coordinates.

[0024] In some embodiments, the position coordinates of the five key points in the above method include: the position coordinates of the left eye, the position coordinates of the right eye, the position coordinates of the nose, the position coordinates of the left corner of the mouth, and the position coordinates of the right corner of the mouth.

[0025] In a second aspect, an embodiment of the present invention provides a device for face detection and tracking, including:

[0026] Acquisition module: used to acquire video frames;

[0027] Judgment module: used to judge whether the current video frame meets the conditions of the first video frame or the continuous tracking of four frames;

[0028] The first detection module is used to detect the coordinates of the first face frame and the position coordinates of the five key points of the first face if the conditions of the first frame of the video or the continuous tracking of four frames are met;

[0029] The second detection module and tracking module are used to detect the position coordinates of the five key points of the second face and track the position coordinates of the five key points of the second face if the conditions of the first frame of the video or the continuous tracking of four frames are not met.

[0030] In some embodiments, the tracking module in the above device is further used to:

[0031] Track the position coordinates of the five key points of the second face according to the LK optical flow algorithm to obtain the position coordinates of the five key points of the second face after tracking;

[0032] Determine good points and bad points in the coordinates of five key points of the tracked second face according to the key point threshold;

[0033] Determine the average value of the difference between the coordinates corresponding to the good point and the position coordinates of the five key points of the second face; the average value is the offset before and after tracking;

[0034] Update the coordinates of the bad point and the second face frame coordinates according to the offset.

[0035] In a third aspect, an embodiment of the present invention further provides an electronic device, including: a processor and a memory;

[0036] The processor is used to execute any one of the above-mentioned methods for face detection and tracking by calling the program or instruction stored in the memory.

[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute a face detection and tracking method as described in any one of the above items.

[0038] The beneficial effects of the present invention are: obtaining a video frame; determining whether the current video frame meets the conditions of the first frame of the video or continuous tracking of four frames; if the conditions of the first frame of the video or continuous tracking of four frames are met, detecting the coordinates of the first face frame and the position coordinates of the five key points of the first face; if the conditions of the first frame of the video or continuous tracking of four frames are not met, tracking the position coordinates of the five key points of the second face according to the position coordinates of the five key points of the first face, and updating the coordinates of the second face frame. In the embodiment of the present application, the five key points of the face are used as the tracking target, which is not easy to lose, and the tracking is not always done. If the tracking is done, the position of the key points will deviate after a certain period of time, so the position coordinates of the five key points of the first face are detected every five times, and then the position coordinates of the five key points of the second face are tracked, and the position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face, and the coordinates of the second face frame are updated, so that the tracked face frame is more stable and will not shake. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A method diagram for face detection and tracking provided by an embodiment of the present invention;

[0040] Figure 2 Another method diagram for face detection and tracking provided by an embodiment of the present invention;

[0041] Figure 3 A diagram of a face detection and tracking device provided by an embodiment of the present invention;

[0042] Figure 4 A schematic block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0044] In order to more clearly understand the above-mentioned purpose, features and advantages of the present application, the present disclosure is further described in detail below in conjunction with the accompanying drawings and examples. It is to be understood that the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The specific embodiments described herein are merely used to explain the present disclosure, rather than to limit the present application. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0045] It should be noted that, in this document, relational terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0046] It should be noted that in the various embodiments of the present application, it involves obtaining relevant data such as images or videos of the object. When the various embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0047] Figure 1 A diagram of a method for face detection and tracking provided by an embodiment of the present invention.

[0048] First, combining Figure 1 The embodiment of the present invention provides a method for face detection and tracking, including three steps: S101, S102 and S103:

[0049] S101: Acquire video frames.

[0050] Specifically, the video frames acquired in the embodiment of the present application are multiple continuous pictures, and each picture in the video is a frame.

[0051] S102: Determine whether the current video frame meets the conditions of being the first video frame or tracking four frames continuously.

[0052] Specifically, in the embodiment of the present application, continuously tracking four frames means tracking four frames after the first frame, that is, judging whether the current video frame meets the conditions of the first frame, the sixth frame, the eleventh frame, etc. of the video. It should be understood that one frame, two frames, and three frames are counted by a counter.

[0053] S103: If the condition of the first frame of the video or four consecutive tracking frames is met, the coordinates of the first face frame and the position coordinates of the five key points of the first face are detected.

[0054] Specifically, if the judgment result satisfies the conditions of the first frame of the video or four consecutive tracking frames, that is, the current frame is the first frame, the sixth frame, the eleventh frame, etc. of the video, the face frame coordinates and the position coordinates of the five key points of the face in the first frame or the sixth frame, the eleventh frame, etc. are detected; the position coordinates of the five key points include: the position coordinates of the left eye, the position coordinates of the right eye, the position coordinates of the nose, the position coordinates of the left corner of the mouth, and the position coordinates of the right corner of the mouth.

[0055] If the conditions of the first frame of the video or four consecutive frames are not met, the position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face to update the coordinates of the second face frame.

[0056] Specifically, if the judgment result is that the condition of the first frame of the video or the continuous tracking of four frames is not met, that is, the current frame is the second frame, the third frame, the fourth frame, the fifth frame, etc., the position coordinates of the five key points of the face of the second frame or the third frame, the fourth frame, and the fifth frame are tracked according to the position coordinates of the five key points of the face of the first frame detected, and the face frame coordinates of the second frame or the third frame, the fourth frame, and the fifth frame are updated.

[0057] It should be understood that in the embodiment of the present application, the five key points of the face are used as the tracking target, it is not easy to lose the face, and the face frame coordinates are updated by tracking the position coordinates of the five key points of the second face according to the position coordinates of the five key points of the first face, so that the tracked face frame is more stable and will not shake.

[0058] In the embodiment of the present application, the face frame coordinates and the position coordinates of the five key points of the face are detected every four frames, and the face frame and the position of the key points are corrected. Practice shows that the face frame coordinates and the position coordinates of the five key points of the face are detected once every five frames and every ten frames. It is found that the time consumed for detection every five frames and every ten frames is not much different, but detection every five frames makes tracking more accurate. Detection every ten frames may result in loss of the seventh or eighth frame. Therefore, the judgment condition in the embodiment of the present application is to determine whether the current video frame meets the conditions of the first frame of the video or continuous tracking of four frames.

[0059] Figure 2 A diagram of another method for face detection and tracking provided by an embodiment of the present invention.

[0060] In some embodiments, in combination Figure 2 , tracking the position coordinates of the five key points of the second face according to the position coordinates of the five key points of the first face, updating the coordinates of the second face frame, including four steps S201, S202, S203 and S204:

[0061] S201: Tracking the position coordinates of the five key points of the second face according to the LK optical flow algorithm to obtain the tracked position coordinates of the five key points of the second face.

[0062] Specifically, in the embodiment of the present application, the LK optical flow algorithm can utilize a small number of key points for tracking, with low computational complexity and fast convergence. For objects moving slowly, the tracking effect is good. The LK optical flow algorithm tracking called by the present application will return an errorcode value. The errorcode value is used to measure the difference between the original image fragments and the tracking results. It is assumed that the errorcode values ​​of the five key points of the face after tracking are (0.6, 0.9, 0.2, 0.9, 0.3) respectively.

[0063] S202: Determine good points and bad points in the coordinates of five key points of the tracked second face according to the key point threshold.

[0064] Specifically, in an embodiment of the present application, good points and bad points in the coordinates of the five key points of the second face after tracking are judged according to the key point threshold, including: the coordinates of the five key points of the second face are greater than the first key point threshold and less than the second key point threshold are good points; the coordinates of the five key points of the second face are less than the first key point threshold or greater than the second key point threshold are bad points; wherein the first key point threshold is less than the second key point threshold.

[0065] Specifically, in the embodiment of the present application, two key point thresholds are set, track_quality_threshold=0.8 and min_track_dist_threshold=0.0001, and the good and bad points among the five points are judged according to the two key point thresholds. Specifically, in the embodiment of the present application, taking the errorcode values ​​of the five key points of the face tracked as (0.6, 0.9, 0.2, 0.9, 0.3) as an example, according to the conditions errorcode<0.8 and errorcode>0.0001, 0.0001 is the first key point threshold, 0.8 is the second key point threshold, and the three points corresponding to 0.6, 0.2, and 0.3 are judged to be good points, and the remaining two points corresponding to 0.9 are bad points, and they are lost.

[0066] The specific first key point threshold and the second key point threshold can be set flexibly and are not limited in the embodiments of the present application.

[0067] S203: Determine the average value of the difference between the coordinates corresponding to the good point and the position coordinates of the five key points of the second face; the average value is the key point offset before and after tracking.

[0068] Specifically, in the embodiment of the present application, the horizontal and vertical coordinates of the three good points are subtracted from the corresponding position coordinates before tracking, and then divided by three to obtain the offset of the key points after the entire tracking.

[0069] S204: Update the coordinates of the bad point and the coordinates of the second face frame according to the key point offset.

[0070] Specifically, in the embodiment of the present application, the coordinates of the bad pixel before tracking are added with the offset to obtain the updated bad pixel coordinates and the second face frame coordinates.

[0071] The key point offset is added to the coordinates of the two bad points before tracking to update the tracking coordinates of the two bad points, and the coordinates of the second face frame are updated together with the good point coordinates and the updated bad point coordinates. The five key points of the face are used as the tracking target, which is not easy to lose; the face frame coordinates are updated by the average of the offsets of the points with good tracking results and the coordinates before tracking, so that the tracked face frame is more stable and will not shake.

[0072] In some embodiments, the above method further comprises:

[0073] If the position coordinates of the five key points of the tracked face are all bad points, re-detect the key points of the face.

[0074] Assume that the errorcode values ​​of the five key points of the face after tracking are (0.9, 0.9, 0.9, 0.9, 0.9) respectively, that is, all are bad points, and detect the coordinates of the face frame and the position coordinates of the five key points of the face.

[0075] Specifically, in the embodiment of the present application, taking the errorcode values ​​of the five key points of the tracked face as (0.6, 0.9, 0.2, 0.9, 0.3) as an example, according to the conditions errorcode<0.8 and errorcode>0.0001, the three points corresponding to 0.6, 0.2, and 0.3 are determined to be good points, and the remaining two points corresponding to 0.9 are bad points and are lost.

[0076] Figure 3 A diagram of a face detection and tracking device provided in an embodiment of the present invention.

[0077] Second, combining Figure 3 , an embodiment of the present invention provides a device for face detection and tracking, comprising:

[0078] Acquisition module 301: used to acquire video frames.

[0079] Specifically, in the embodiment of the present application, the video frames acquired by the acquisition module 301 are multiple continuous pictures, and each picture in the video is a frame.

[0080] The judgment module 302 is used to judge whether the current video frame meets the conditions of being the first frame of the video or tracking four frames continuously.

[0081] Specifically, in the embodiment of the present application, continuously tracking four frames means tracking four frames after the first frame, that is, the judgment module 302 judges whether the current video frame meets the conditions of the first frame, the sixth frame, the eleventh frame, etc. of the video. It should be understood that one frame, two frames, and three frames are counted by a counter.

[0082] It should be understood that in the embodiment of the present application, based on the five key points of the face as the tracking target, it is not easy to lose the face, and the face is not tracked all the time. If it is tracked all the time, the position of the key points will deviate after a certain period of time, so it cannot be tracked all the time. In the embodiment of the present application, the face frame coordinates and the position coordinates of the five key points of the face are detected every four frames, and the face frame and the key point positions are corrected. Practice shows that the face frame coordinates and the position coordinates of the five key points of the face are detected once every five frames and every ten frames. It is found that the time spent on detecting once every five frames and once every ten frames is not much different, but detecting once every five frames makes the tracking more accurate. Detecting once every ten frames may cause the face to be lost in the seventh or eighth frame. Therefore, the judgment condition in the embodiment of the present application is to determine whether the current video frame meets the conditions of the first frame of the video or continuous tracking of four frames.

[0083] Detection module 303: if the conditions of the first frame of the video or four consecutive tracking frames are met, the detection module 303 detects the coordinates of the first face frame and the position coordinates of the five key points of the first face.

[0084] Specifically, if the current frame is determined to be the first frame, the sixth frame, the eleventh frame, etc. of the video, the detection module 303 detects the face frame coordinates and the position coordinates of the five key points of the face in the first frame, the sixth frame, the eleventh frame, etc.

[0085] Tracking module 304: if the condition of the first frame of the video or the continuous tracking of four frames is not met, the position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face to update the coordinates of the second face frame.

[0086] Specifically, if the condition of being the first frame of the video or tracking four frames continuously is not met, that is, the judgment result is that the current frame is the second frame, the third frame, the fourth frame, the fifth frame, etc., the tracking module 304 tracks the position coordinates of the five key points of the face of the second frame or the third frame, the fourth frame, and the fifth frame according to the position coordinates of the five key points of the face of the first frame detected, and updates the face frame coordinates of the second frame or the third frame, the fourth frame, and the fifth frame.

[0087] In some embodiments, the tracking module 304 in the above device is further used to: track the position coordinates of the five key points of the second face according to the LK optical flow algorithm to obtain the position coordinates of the five key points of the second face after tracking.

[0088] Determine the good and bad points in the coordinates of the five key points of the second face after tracking based on the key point threshold. Determine the average value of the difference between the coordinates corresponding to the good points and the position coordinates of the five key points of the second face; the average value is the offset before and after tracking. Update the coordinates of the bad points and the coordinates of the second face frame based on the offset.

[0089] In a third aspect, an embodiment of the present invention further provides an electronic device, including: a processor and a memory;

[0090] The processor is used to execute any one of the above-mentioned methods for face detection and tracking by calling the program or instruction stored in the memory.

[0091] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute a face detection and tracking method as described in any one of the above items.

[0092] Figure 4 It is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure.

[0093] like Figure 4 As shown, the electronic device includes: at least one processor 401, at least one memory 402 and at least one communication interface 403. The various components in the electronic device are coupled together through a bus system 404. The communication interface 403 is used to transmit information between external devices. It can be understood that the bus system 404 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus system 404 .

[0094] It can be understood that the memory 402 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0095] In some implementations, the memory 402 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system and application programs.

[0096] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. A program that implements any of the methods of face detection and tracking provided in an embodiment of the present application can be included in the application.

[0097] In an embodiment of the present application, the processor 401 calls a program or instruction stored in the memory 402, specifically, a program or instruction stored in an application, and the processor 401 is used to execute the steps of each embodiment of a method for face detection and tracking provided in an embodiment of the present application.

[0098] Get video frames;

[0099] Determine whether the current video frame meets the conditions of the first video frame or four consecutive tracking frames;

[0100] If the conditions of the first frame of the video or four consecutive tracking frames are met, the coordinates of the first face frame and the position coordinates of the five key points of the first face are detected;

[0101] If the conditions of the first frame of the video or four consecutive frames are not met, the position coordinates of the five key points of the second face are tracked according to the position coordinates of the five key points of the first face to update the coordinates of the second face frame.

[0102] Any of the methods for face detection and tracking provided in the embodiments of the present application can be applied to the processor 401, or implemented by the processor 401. The processor 401 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 401 or instructions in the form of software. The above processor 401 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0103] The steps of any method in the method for face detection and tracking provided in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software units in the decoding processor. The software unit can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402, and completes the steps of a method for face detection and tracking in combination with its hardware.

[0104] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0105] Those skilled in the art will appreciate that the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0106] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations all fall within the scope defined by the appended claims. The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

[0107] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for face detection and tracking, characterized in that: include: Get video frames; Determine whether the current video frame meets the conditions of the first video frame or four consecutive tracking frames; If the current video frame meets the conditions of the first frame of the video or four consecutively tracked frames, detecting the coordinates of the first face frame and the position coordinates of five key points of the first face from the current video frame; If the current video frame does not meet the conditions of the first frame of the video or the continuous tracking of four frames, tracking the position coordinates of the five key points of the second face in the current video frame according to the position coordinates of the five key points of the first face, so as to update the coordinates of the second face frame; The step of tracking the position coordinates of the five key points of the second face in the current video frame according to the position coordinates of the five key points of the first face to update the coordinates of the second face frame includes: According to the LK optical flow algorithm and the position coordinates of the five key points of the first face, the position coordinates and error values ​​corresponding to the five key points of the second face after tracking are obtained; Determining good points and bad points among the five key points of the tracked second face according to the key point threshold and the error values ​​respectively corresponding to the five key points; For each of the good points, determine a difference between a coordinate position of the good point in the second face and a coordinate position of the good point in the first face; Based on the difference values ​​corresponding to each of the good points, an average value is obtained; wherein the average value is the offset before and after tracking; Update the coordinate position of the bad point according to the offset; The second face frame coordinates are updated based on the coordinate positions corresponding to the good points and the updated coordinate positions of the bad points.

2. The method for face detection and tracking according to claim 1, characterized in that: The method further comprises: If all five key points of the tracked face are bad points, re-detect the key points of the face.

3. The method for face detection and tracking according to claim 1, characterized in that: The determining, according to the key point threshold and the error values ​​respectively corresponding to the five key points, good points and bad points among the five key points of the tracked second face comprises: For the five key points, if the error value corresponding to the key point is greater than the first key point threshold and less than the second key point threshold, the key point is determined to be a good point; if the error value corresponding to the key point is less than the first key point threshold or greater than the second key point threshold, the key point is determined to be a bad point; wherein the first key point threshold is less than the second key point threshold.

4. The method for face detection and tracking according to claim 1, characterized in that: The updating of the coordinate position of the bad point according to the offset includes: The position coordinates of the bad point in the second face are added to the offset to obtain an updated coordinate position of the bad point.

5. The method for face detection and tracking according to any one of claims 1 to 4, characterized in that: The position coordinates of the five key points include: the position coordinates of the left eye, the position coordinates of the right eye, the position coordinates of the nose, the position coordinates of the left corner of the mouth, and the position coordinates of the right corner of the mouth.

6. A device for face detection and tracking, characterized in that: include: Acquisition module: used to acquire video frames; Judgment module: used to judge whether the current video frame meets the conditions of the first video frame or the continuous tracking of four frames; A first detection module: used for detecting the coordinates of the first face frame and the position coordinates of five key points of the first face from the current video frame if the current video frame meets the conditions of the first frame of the video or the continuous tracking of four frames; The second detection module and tracking module are used to track the position coordinates of the five key points of the second face in the current video frame according to the position coordinates of the five key points of the first face if the current video frame does not meet the conditions of the first frame of the video or the continuous tracking of four frames, so as to update the coordinates of the second face frame; Among them, the tracking module is specifically used for: According to the LK optical flow algorithm and the position coordinates of the five key points of the first face, the coordinate positions and error values ​​corresponding to the five key points of the second face after tracking are obtained; Determining good points and bad points among the five key points of the tracked second face according to the key point threshold and the error values ​​respectively corresponding to the five key points; For each of the good points, determine a difference between a coordinate position of the good point in the second face and a coordinate position of the good point in the first face; Based on the difference values ​​corresponding to each of the good points, an average value is obtained; wherein the average value is the offset before and after tracking; Update the coordinate position of the bad point according to the offset; The second face frame coordinates are updated based on the coordinate positions corresponding to the good points and the updated coordinate positions of the bad points.

7. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute the face detection and tracking method as described in any one of claims 1 to 5 by calling the program or instruction stored in the memory.

8. A computer-readable storage medium, characterized in that: A non-transitory computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute a face detection and tracking method as claimed in any one of claims 1 to 5.