Face tracking method, device, computer equipment and storage medium

Abstract

The present invention discloses a face tracking method, comprising the following steps: S1. obtaining an original image of a current frame to be tested that contains a face; S2. detecting and intercepting the face area in the original image to obtain an initial face image; S3. inputting the initial face image into an ultra-lightweight face key point model to obtain an aligned face image; S4. inputting the aligned face image into a face judgment model to determine whether the aligned face image contains a face; S5. inputting the aligned face image into a face key point contour model to obtain the position coordinates of a high-definition face detection frame; S6. obtaining an original image of the next adjacent frame, and intercepting the initial face image of the next adjacent frame image based on the position coordinates of the high-definition face detection frame of the current frame image. The face tracking method of the present application only needs to perform face detection on one frame of image, thereby improving the speed of the face tracking algorithm and reducing the delay of the key point effect.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a face tracking method, device, computer equipment and storage medium. Background Art

[0002] With the continuous improvement and development of computer vision technology, face tracking, as a key technology in video surveillance, human-computer interaction, automatic recognition, and autonomous driving, has developed rapidly. Currently, because the detected face is in motion in real-time scenarios, the face's position constantly changes, often resulting in different face positions detected by the detector in each frame, making it difficult to obtain an accurate and stable face position. In the past, improving face detection accuracy could not avoid errors between previous and subsequent frames, and weighted smoothing of the face positions between previous and subsequent frames would lead to tracking delays, resulting in unsatisfactory results.

[0003] Most existing face tracking algorithms perform frame-by-frame detection on real-time camera data. Face detection is performed on each image. The position of the face in the image is obtained through face detection. The face image is cropped from the original image and then fed into the face contour key point model to obtain the key points of the face, such as Figure 1 As shown. Obtaining facial key points using this method requires repeated fine-tuning of the face detection and facial contour key point models, resulting in poor real-time performance and results. Furthermore, the accuracy of facial key points is not only related to the model's capabilities, but also to the position of the face in the input image. If the face's position in the image input to the key point model is not fixed, it will lead to deviations in the key point model input results. Real-time continuous detection will amplify the deviations, resulting in severe jitter. Face tracking relies heavily on the capabilities of face detection and facial key point models. There must be a trade-off between accuracy and speed. Models with high computational overhead have high accuracy but slow inference speed, while models with low computational overhead are fast but have poor accuracy. This creates an algorithmic dilemma.

[0004] Therefore, finding a method that can stably track human faces while improving face tracking accuracy is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a high-precision, low-latency face tracking algorithm, device, computer equipment and storage medium, which can ensure the stability of face tracking and greatly reduce the jitter of face tracking when computing power is limited and high precision requirements are required for face tracking.

[0006] To achieve the above object, the present invention provides a face tracking method, comprising the following steps:

[0007] S1. Obtain the original image of the current frame to be tested containing the face;

[0008] S2 detects and captures the face area in the original image to obtain an initial face image;

[0009] S3 inputs the initial face image into the ultra-lightweight face key point model, aligns the initial face image to obtain an aligned face image;

[0010] S4. Sending the aligned face image to the face recognition model to determine whether the aligned face image contains a face, if so, proceed to the next step, if not, return to step S2;

[0011] S5. Input the aligned face image into the face key point contour model, perform high-definition processing on the aligned face image to obtain the position coordinates of the high-definition face detection frame;

[0012] S6 obtains the original image of the next adjacent frame, and captures the initial face image of the next adjacent frame according to the position coordinates of the high-definition face detection frame of the current frame image;

[0013] Repeat steps S3-S6 to output high-definition face detection frames of subsequent consecutive frame images in sequence.

[0014] In one embodiment of the present invention, step S2 specifically includes the following contents:

[0015] S21. Detect the original image and obtain the position coordinates [l, t, r, b] of a first face detection frame, where (l, t) are the horizontal and vertical coordinates of the upper left corner of the first face detection frame on the original image, and (r, b) are the horizontal and vertical coordinates of the lower right corner of the first face detection frame on the original image;

[0016] S22. Proportionally enlarge the original image to obtain an enlarged image, detect the face region in the enlarged image, and obtain the position coordinates of a second face detection frame [l', t', r', b'], where (l', t') are the horizontal and vertical coordinates of the upper left corner of the second face detection frame on the enlarged image, and (r', b') are the horizontal and vertical coordinates of the lower right corner of the second face detection frame on the enlarged image;

[0017] S23. Use the second face detection frame to capture the face portion in the enlarged image to obtain an initial face image, where the height of the initial face image is b'-t' and the width is r'-l'.

[0018] In one embodiment of the present invention, step S3 specifically includes the following contents:

[0019] S31. Input the initial face image into the ultra-lightweight face key point model, and output the first low-precision face key point coordinate set of the face.

[0020] [x′0, y′0, x′1, y′1,…x′ n-1 , y′ n-1 ]

[0021] , where n represents the total number of facial key points, (x′0, y′0) represents the horizontal and vertical coordinates of the first facial key point, (x′ n-1 , y′ n-1 ) represents the horizontal and vertical coordinates of the nth facial key point;

[0022] S32. Obtain a second low-precision facial key point coordinate set of the face on the original image by adding the horizontal coordinate and vertical coordinate of each key point in the first low-precision facial key point coordinate set to the horizontal coordinate and vertical coordinate of the upper left corner position of the second face detection frame on the original image.

[0023] [x″0, y″0, x″1″, y″1″,…x″ n-1 , y″ n-1 ]

[0024] S33. By operating the second low-precision facial key point coordinate set with the preset standard facial key point coordinate set, a similarity transformation matrix M1 is obtained.

[0025] where R 00 =R 11 , R 01 =-R 10 , R 00 、R 01 、R 10 、R 11 , represents the rotation and scaling coefficient of the similarity transformation matrix, T x 、T y Represents the offset coefficient of the similarity transformation matrix;

[0026] S34. Based on the similarity transformation matrix M1 and the original image, align the initial face image to obtain an aligned face image.

[0027] In one embodiment of the present invention, step S5 specifically includes the following contents:

[0028] S51. Inputting the aligned face image into the face key point contour model, outputting a first high-precision face key point coordinate set of the face;

[0029] S52. Based on the inverse matrix of the similarity transformation matrix M1, perform an inverse coordinate transformation on the coordinates of each key point in the first high-precision facial key point coordinate set to obtain a second high-precision facial key point coordinate set of the face on the original image;

[0030] S53. Determine the position coordinates of the third face detection frame on the original image by calculating the maximum and minimum values ​​of the horizontal and vertical coordinates in the second high-precision facial key point coordinate set;

[0031] S54. Enlarge the original image in proportion, and calculate the position coordinates of the high-definition face detection frame using the position coordinates of the third face detection frame.

[0032] In one embodiment of the present invention, step S6 specifically includes the following contents:

[0033] Obtain the original image of the adjacent next frame, use the position coordinates of the high-definition face detection frame in the current frame image as the position coordinates of the second detection frame of the adjacent next frame image, and intercept the initial face image of the adjacent next frame image according to the position coordinates of the second detection frame of the adjacent next frame image.

[0034] The present application also provides a face tracking device, comprising:

[0035] An acquisition module is used to obtain the original image of the current frame to be tested containing the face;

[0036] A detection module, configured to detect a face region in the original image;

[0037] An alignment module, configured to align the face region to obtain an aligned face image;

[0038] A judgment module, configured to judge whether the aligned face image contains a face;

[0039] A determination module, configured to determine a high-definition face position in the original image;

[0040] The tracking module is used to obtain the next frame image of the current frame and obtain the initial face image in the next frame image according to the high-definition face position in the original image of the current frame.

[0041] In one embodiment of the present invention, the face tracking device further includes:

[0042] an enlargement module, configured to enlarge the original image by a predetermined multiple to obtain a face region on the enlarged image;

[0043] The interception module is used to intercept the face area from the enlarged image to obtain an initial face image.

[0044] In one embodiment of the present invention, the alignment module includes:

[0045] An extraction submodule, configured to extract facial key point information of the facial area;

[0046] A calculation submodule, configured to calculate a similarity transformation matrix for achieving face alignment using the facial key point information;

[0047] The alignment submodule is used to align the face region using a similarity transformation matrix.

[0048] The present invention also provides a computer device, comprising a memory and a processor; the memory stores a computer program that can be run by the processor; when the processor runs the computer program, the steps of the face tracking method are executed.

[0049] The present invention also provides a computer-readable storage medium, which includes program code. When the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the face tracking method.

[0050] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0051] 1. This application uses an ultra-lightweight facial key point model to extract low-precision facial key points in the original image, and uses the similarity transformation matrix calculated by the extracted low-precision facial key point coordinates and the standard facial key point coordinates to obtain the aligned facial image in the original image, and uses the face judgment model to determine whether the original image contains a face and whether face detection needs to be re-enabled, and then uses the facial key point contour model to perform high-definition processing on the aligned facial image, thereby improving the accuracy of face tracking and reducing time consumption.

[0052] 2. This application uses the face position obtained by face detection, ultra-lightweight face key point detection, similarity transformation, and high-precision face key point detection of the current frame image as the initial face position of the adjacent next frame image after amplification, and then uses ultra-lightweight face key point detection, similarity transformation, and high-precision face key point detection to form a complete model for face tracking. The face key points of the next frame image of this application are obtained through calculation. It is only necessary to detect the face position in one frame image, and there is no need to perform face detection on each frame image. Subsequent face tracking no longer relies on face detection, preventing the face from moving quickly and causing the restart of face detection or further processing of images that do not contain faces, thereby causing a delay in the face key point effect and affecting the processing speed of the face tracking algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a face tracking algorithm in the prior art.

[0054] Figure 2 is a flowchart of a face tracking algorithm according to an embodiment of the present invention;

[0055] Figure 3 yes Figure 2 Specific flow chart of step S2;

[0056] Figure 4 yes Figure 2 Specific flow chart of step S3;

[0057] Figure 5 yes Figure 2 Specific flow chart of step S5;

[0058] Figure 6 FIG. 1 is a schematic diagram of a face tracking device according to an embodiment of the present invention.

[0059] Reference numerals:

[0060] 100, acquisition module; 200, detection module; 300, alignment module; 400, judgment module; 500, determination module; 600, tracking module. DETAILED DESCRIPTION

[0061] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0062] Unless expressly stated otherwise, throughout the specification and claims, the term "comprise" or variations such as "include" or "comprising", etc., will be understood to include the stated elements or components but not to exclude other elements or other components.

[0063] like Figure 2-6 As shown, a preferred embodiment of the present invention provides a face tracking algorithm, comprising the following steps:

[0064] S1. Obtain the original image of the current frame to be tested containing the face;

[0065] S2. Detect and capture the face area in the original image to obtain an initial face image;

[0066] S3. Input the initial face image into the ultra-lightweight face key point model, align the initial face image, and obtain an aligned face image;

[0067] S4. The aligned face image is fed into the face recognition model to determine whether the aligned face image contains a face. If so, proceed to the next step; if not, return to step S2;

[0068] S5. Input the aligned face image into the face key point contour model, perform high-definition processing on the aligned face image, and obtain the position coordinates of the high-definition face detection frame;

[0069] S6 obtains the original image of the next adjacent frame, and captures the initial face image of the next adjacent frame according to the position coordinates of the high-definition face detection frame of the current frame image;

[0070] Repeat steps S3-S6 to output high-definition face detection frames of subsequent consecutive frame images in sequence.

[0071] In an embodiment of the present application, the face tracking algorithm detects the original image of any frame in the real-time camera data. The original image is an image of the facial key points to be determined. These images can be, for example, images to be used to add facial special effects or to perform facial AR operations. In addition, the image to be detected can be a real-time image taken by a terminal device, or a video frame image extracted from an existing video. The present disclosure does not impose any special restrictions on the resolution, image background, facial posture, number of faces, image color temperature, etc. of the image to be detected.

[0072] Here, facial landmarks refer to information that reflects facial features, such as eyes, eyebrows, nose, mouth, and facial contours. Facial landmark coordinates refer to the coordinates of these facial landmarks. The coordinates of each facial landmark can be represented by a two-dimensional array, for example, (x1, y1) or (x2, y2).

[0073] like Figure 3 As shown, in a further embodiment of the present application, step S2 specifically includes the following contents:

[0074] S21. Detect the original image and obtain the position coordinates of the first face detection frame [l, t, r, b], where (l, t) are the horizontal and vertical coordinates of the upper left corner of the first face detection frame on the original image, and (r, b) are the horizontal and vertical coordinates of the lower right corner of the first face detection frame on the original image.

[0075] S22. Scale up the original image to obtain an enlarged image, detect the face area in the enlarged image, and obtain the position coordinates of the second face detection frame [l', t', r', b'], where (l', t') are the horizontal and vertical coordinates of the upper left corner of the second face detection frame on the enlarged image, and (r', b') are the horizontal and vertical coordinates of the lower right corner of the second face detection frame on the enlarged image.

[0076] In this embodiment, the magnification ratio can be interpolated on the original image according to the magnification factor selected by the user (for example, the optional magnification factors include 1x, 2x, 4x, etc.), thereby magnifying the original image to the corresponding size. Optionally, the embodiment of the present application uses bilinear interpolation. While the original image is magnified into the first image, the facial key point information in the first image is also magnified to the corresponding size, thereby obtaining the facial key point information in the first image.

[0077] S23. Use the second face detection frame to capture the face portion in the enlarged image to obtain an initial face image. The height of the initial face image is b'-t', and the width is r'-1'.

[0078] S3. Input the initial face image into the ultra-lightweight face key point model, align the initial face image, and obtain an aligned face image.

[0079] like Figure 4 As shown, in a further embodiment of the present application, step S3 specifically includes the following contents:

[0080] S31. Input the initial face image into the ultra-lightweight face key point model and output the first low-precision face key coordinate set [x′0, y′0, x′1, y′1, …x′ n-1 , y′ n-1 ], where n represents the total number of facial key points, (x′0, y′0) represents the horizontal and vertical coordinates of the first facial key point, (x′ n-1 , y′ n-1 ) represents the horizontal and vertical coordinates of the nth facial key point.

[0081] By inputting the initial facial image into the ultra-lightweight facial landmark model, the facial landmark information in the initial facial image can be obtained. Using the ultra-lightweight facial landmark model to locate the face position greatly improves the accuracy of face tracking, and in actual use, the time consumption is low and the results are very stable.

[0082] S32. Obtain a second low-precision facial key point coordinate set [x″0, y″0, x″1, y″1, …x″] of the face on the original image by adding the horizontal coordinate and vertical coordinate of each key point in the first low-precision facial key point coordinate set to the horizontal coordinate and vertical coordinate of the upper left corner of the second face detection frame on the original image. n-1 , y″ n-1 ].

[0083] S33. By operating the second low-precision facial key point coordinate set with the preset standard facial key point coordinate set, a similarity transformation matrix M1 is obtained.

[0084] where R 00 =R 11 , R 01 =-R 10 , R 00 、R 01 、R 10 、R 11 Represents the rotation and scaling coefficients of the similarity transformation matrix, T x 、T y Represents the offset coefficient of the similarity transformation matrix; wherein the standard facial key point coordinate set is a preset facial key point coordinate set used as a reference base value.

[0085] S34. Based on the similarity transformation matrix M1 and the original image, the initial face image is aligned to obtain an aligned face image.

[0086] S4. The aligned face image is fed into the face recognition model to determine whether it contains a face. If so, the process proceeds to the next step; if not, the process returns to step S2. In this step, the face recognition model determines whether the aligned face image contains a face, and then determines whether to track the face. If the aligned face image does contain a face, tracking is maintained. If not, tracking is terminated and face detection is re-enabled on the original image.

[0087] S5. Input the aligned face image into the face key point contour model, perform high-definition processing on the aligned face image, and obtain the position coordinates of the high-definition face detection frame.

[0088] like Figure 5 As shown, in a further embodiment of the present application, step S5 specifically includes the following steps:

[0089] S51. Input the aligned face image into the face key point contour model and output the first high-precision face key point coordinate set of the face

[0090] [x″′0, y″′0, x″′1, y″′1,…x″′ n-1 , y″′ n-1 ].

[0091] S52. Based on the inverse matrix of the similarity transformation matrix M1, the coordinates of each key point in the first high-precision facial key point coordinate set are inversely transformed to obtain the second high-precision facial key point coordinate set of the face on the original image.

[0092] [x0, y0, x1, y1, ... x n-1 ,y n-1 ].

[0093] S53. Determine the position coordinates of the third face detection frame on the original image by calculating the maximum and minimum values ​​of the horizontal and vertical coordinates in the second high-precision facial key point coordinate set.

[0094] S54. Enlarge the original image proportionally, and calculate the position coordinates of the high-definition face detection frame using the position coordinates of the third face detection frame.

[0095] S6 obtains the original image of the next adjacent frame, and captures the initial face image of the next adjacent frame according to the position coordinates of the high-definition face detection frame of the current frame image;

[0096] Repeat steps S3-S6 to output high-definition face detection frames for subsequent frames. Using the coordinates of the high-definition face detection frame in the previous frame as the coordinates of the second detection frame in the next frame, and capturing the initial face image of the next frame, eliminates the need to perform face detection on every frame, improving face tracking efficiency.

[0097] This application conducts a face tracking comparison test by using different algorithms on the same imaging device. The experimental results are shown in Table 1.

[0098]

[0099] Table 1 is a comparison table of the face tracking algorithm of the embodiment of the present application and the original face tracking algorithm

[0100] Among them, the existing algorithm is to crop the face image from the original image and then feed it into the face contour key point model to obtain the key points of the face, such as Figure 1 shown.

[0101] The comparison algorithm performs face detection on the original image to obtain a face image, and sends the face image to the face judgment model to determine whether the face image contains a face. If it contains a face, the tracking state is maintained. If not, the tracking is exited and the original image is re-detected for face; the face image containing a face is sent to the face contour model. The specific processing steps of the face detection, face judgment model and face contour model in the comparison algorithm are the same as those of the face tracking algorithm in the preferred embodiment of the present application, and will not be repeated here.

[0102] From the comparison data, it can be seen that compared with the existing algorithms and the comparison algorithms, the applicant's face tracking algorithm has a small average total time consumption and high stability, and has a better effect of reducing face tracking jitter and improving face tracking stability.

[0103] The training data in the embodiments of the present application are all private data, which are obtained through semi-automatic labeling, manual detection and manual labeling.

[0104] This application uses the face position obtained by face detection, ultra-lightweight face key point detection, similarity transformation, and high-precision face key point detection in the current frame image, and uses it as the initial face position of the adjacent next frame image after amplification, and then uses ultra-lightweight face key point detection, similarity transformation, and high-precision face key point detection to form a complete model for face tracking. The face key points of the next frame image of this application are obtained through calculation. It is only necessary to detect the face position in one frame image, and there is no need to perform face detection on each frame image. Subsequent face tracking no longer relies on face detection, preventing the face from moving quickly and causing the restart of face detection or further processing of images that do not contain faces, thereby causing a delay in the face key point effect and affecting the processing speed of the face tracking algorithm.

[0105] like Figure 6 As shown, a preferred embodiment of the present application further provides a face tracking device, comprising:

[0106] The acquisition module 100 is used to acquire the original image of the current frame to be tested containing the face;

[0107] Detection module 200, used to detect the face area in the original image;

[0108] An alignment module 300 is used to align the face region to obtain an aligned face image;

[0109] A determination module 400 is used to determine whether the aligned face image contains a face;

[0110] A determination module 500 is used to determine the position of a high-definition face in the original image;

[0111] The tracking module 600 is used to obtain the next frame image of the current frame, and obtain the initial face image in the next frame image according to the high-definition face position in the original image of the current frame.

[0112] In a further embodiment of the present application, the face tracking device further includes:

[0113] An enlargement module, used to enlarge the original image by a predetermined multiple to obtain a face area on the enlarged image;

[0114] The cropping module is used to crop the face area from the enlarged image to obtain an initial face image.

[0115] In a further embodiment of the present application, the alignment module includes:

[0116] Extraction submodule, used to extract facial key point information of the face area;

[0117] A calculation submodule is used to calculate a similarity transformation matrix for achieving face alignment using facial key point information;

[0118] The alignment submodule is used to align the face regions using the similarity transformation matrix.

[0119] This application obtains the original image of the current frame to be tested containing a face through an acquisition module, and the magnification module magnifies the original image in proportion. The detection module is used to detect the face area in the original image and cut out to obtain an initial face image. The initial face image is input into the ultra-lightweight face key point model in the alignment module, the face key point information of the face area is extracted, the similarity transformation matrix is ​​calculated, and the face area is aligned using the similarity transformation matrix to obtain an aligned face image, and it is determined whether the aligned face image contains a face. If it does not contain a face, the face tracking is exited and the original image is re-detected; if it contains a face, the high-definition face position in the original image is determined, and the high-definition face position of the current frame is used as the initial face image of the next frame image. There is no need to perform face detection on each frame image, which greatly reduces the time consumption of face detection. Since the initial face position of the next frame image is obtained through the high-definition face position of the previous frame image, it avoids the situation where the face is moving quickly and the initial face position obtained in the next frame does not contain a complete face or no face at all, thereby restarting the face detection or sending the image containing the incomplete face into the face contour model for further processing, avoiding single-frame jumps in the results or delays in key point effects, improving the speed and accuracy of face tracking, improving the face tracking effect, and solving the angle mutation caused by blinking, so that the face tracking algorithm of the present application can run smoothly on low-end machines with limited computing power, thereby expanding the scope of application of the face tracking algorithm.

[0120] A preferred embodiment of the present application further provides a computer device, comprising a memory and a processor; the memory stores a computer program that can be executed by the processor; when the processor executes the computer program, the steps of the face tracking method are executed.

[0121] A preferred embodiment of the present application further provides a computer-readable storage medium, which includes program code. When the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the face tracking method.

[0122] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Suitable nonvolatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0123] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0124] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0127] The foregoing descriptions of specific exemplary embodiments of the present invention are for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been selected and described for the purpose of explaining the specific principles of the invention and their practical application, thereby enabling those skilled in the art to realize and utilize a variety of exemplary embodiments of the invention and various options and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A face tracking algorithm, characterized in that: The following steps are involved: S1. Obtain the original image of the current frame to be tested containing the face; S2 detects and captures the face area in the original image to obtain an initial face image; S3 inputs the initial face image into the ultra-lightweight face key point model, aligns the initial face image to obtain an aligned face image; S4. The aligned face image is fed into the face recognition model to determine whether the aligned face image contains a face. If so, proceed to the next step, if not, return to step S2; S5. The aligned face image is input into the facial key point contour model, the aligned face image is processed in high definition to obtain the position coordinates of the high-definition face detection frame; S6 obtains the original image of the next adjacent frame, based on the position coordinates of the high-definition face detection frame of the current frame image, captures the initial face image of the next adjacent frame image; Repeat steps S3-S6 to sequentially output high-definition face detection frames in subsequent consecutive frame images; Wherein, the step S5 specifically includes the following contents: S51. The aligned face image is input into the facial key point contour model, and the output is a first high-precision facial key point coordinate set of the face; S52. Based on the inverse matrix of the similarity transformation matrix M1, the coordinates of each key point in the first high-precision facial key point coordinate set are inversely transformed to obtain a second high-precision facial key point coordinate set of the face on the original image; S53 by calculating the maximum and minimum values ​​of the horizontal and vertical coordinates of the second high-precision facial key point coordinate set, determining the position coordinates of the third face detection frame on the original image; S54. Enlarge the original image proportionally, and calculate the position coordinates of the high-definition face detection frame using the position coordinates of the third face detection frame.

2. A face tracking algorithm according to claim 1, characterized in that: The step S2 specifically includes the following contents: S21. Detect the original image and obtain the position coordinates [l, t, r, b] of a first face detection frame, where (l, t) are the horizontal and vertical coordinates of the upper left corner of the first face detection frame on the original image, and (r, b) are the horizontal and vertical coordinates of the lower right corner of the first face detection frame on the original image; S22. Proportionally enlarge the original image to obtain an enlarged image, detect the face region in the enlarged image, and obtain the position coordinates of a second face detection frame [l', t', r', b'], where (l', t') are the horizontal and vertical coordinates of the upper left corner of the second face detection frame on the enlarged image, and (r', b') are the horizontal and vertical coordinates of the lower right corner of the second face detection frame on the enlarged image; S23. Use the second face detection frame to capture the face portion in the enlarged image to obtain an initial face image, where the height of the initial face image is b'-t' and the width is r'-l'.

3. A face tracking algorithm according to claim 2, characterized in that: The step S3 specifically includes the following contents: S31. Input the initial face image into the ultra-lightweight face key point model, and output the first low-precision face key point coordinate set of the face. , where n represents the total number of facial key points, Indicates the horizontal and vertical coordinates of the first facial key point, Represents the horizontal and vertical coordinates of the nth facial key point; S32. Obtain a second low-precision facial key point coordinate set of the face on the original image by adding the horizontal coordinate and vertical coordinate of each key point in the first low-precision facial key point coordinate set to the horizontal coordinate and vertical coordinate of the upper left corner position of the second face detection frame on the original image. ； S33. The similarity transformation matrix M1 is obtained by operating the second low-precision face key point coordinate set with the preset standard face key point coordinate set, M1 = , in , represents the rotation and scaling coefficient of the similarity transformation matrix, Represents the offset coefficient of the similarity transformation matrix; S34. Based on the similarity transformation matrix M1 and the original image, align the initial face image to obtain an aligned face image.

4. A face tracking algorithm according to claim 3, characterized in that: The step S6 specifically includes the following contents: Obtain the original image of the adjacent next frame, use the position coordinates of the high-definition face detection frame in the current frame image as the position coordinates of the second detection frame of the adjacent next frame image, and intercept the initial face image of the adjacent next frame image according to the position coordinates of the second detection frame of the adjacent next frame image.

5. A face tracking device, using the face tracking method according to any one of claims 1 to 4, characterized in that: include: An acquisition module is used to obtain the original image of the current frame to be tested containing the face; A detection module, configured to detect a face region in the original image; An alignment module, configured to align the face region to obtain an aligned face image; A judgment module, configured to judge whether the aligned face image contains a face; A determination module is configured to input the aligned facial image into a facial key point contour model and output a first high-precision facial key point coordinate set of the face; further perform an inverse coordinate transformation on the coordinates of each key point in the first high-precision facial key point coordinate set based on the inverse matrix of the similarity transformation matrix M1 to obtain a second high-precision facial key point coordinate set of the face on the original image; and determine the position coordinates of a third face detection frame on the original image by calculating the maximum and minimum values ​​of the horizontal and vertical coordinates in the second high-precision facial key point coordinate set; scale the original image, and calculate the position coordinates of a high-definition face detection frame using the position coordinates of the third face detection frame to serve as the high-definition face position in the original image; The tracking module is used to obtain the next frame image of the current frame and obtain the initial face image in the next frame image according to the high-definition face position in the original image of the current frame.

6. The face tracking device according to claim 5, characterized in that: Also includes: an enlargement module, configured to enlarge the original image by a predetermined multiple to obtain a face region on the enlarged image; The interception module is used to intercept the face area from the enlarged image to obtain an initial face image.

7. The face tracking device according to claim 5, characterized in that: The alignment module includes: An extraction submodule, configured to extract facial key point information of the facial area; A calculation submodule, configured to calculate a similarity transformation matrix for achieving face alignment using the facial key point information; The alignment submodule is used to align the face region using a similarity transformation matrix.

8. A computer device comprising a memory and a processor; the memory stores a computer program executable by the processor; when the processor executes the computer program, the processor executes the steps of the face tracking method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that It includes program code, and when the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the face tracking method according to any one of claims 1 to 4.

Images