Face tracking method and device based on video image, equipment and medium

Abstract

The embodiment of the application provides a face tracking method, device and equipment based on video images and a medium. In the method, first, an effective detection frame is screened out, then a space matching cost function matrix is obtained according to the effective detection frame and a tracking target, and then a matching result is obtained by combining a Hungarian maximum matching algorithm. If the matching result has multiple groups, a face feature matching cost function matrix is further obtained, and a matching result is obtained by combining the Hungarian maximum matching algorithm again. Finally, tracking parameters are refreshed and a tracking queue is updated according to the matching result. The space matching cost function matrix and the face feature matching cost function matrix are obtained, and the Hungarian maximum matching algorithm is used to obtain the matching result, so that the failure rate of face tracking is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video processing, in particular to a face tracking method and device based on video images, equipment and medium. BACKGROUND

[0002] With the development of society and technology, the technology of face tracking is becoming more and more mature, and the application field is also more and more, which has been widely used in security, retail and other fields. For the video obtained by fixed camera, mobile camera and other ways, how to obtain and identify face, obtain face feature and track face is the problem people want to solve.

[0003] In the prior art, after the gateway obtains the video, a plurality of frame images are extracted, the current frame image is identified to obtain a detection box and a corresponding target. Then, the tracking target determined in the last frame is cascaded and matched. For the detection box and the tracking target that are not cascaded and matched, an Intersection over Union (IOU) matrix is calculated, and the Hungarian maximum matching algorithm is used based on the IOU matrix to match to obtain the final matching result. Finally, the tracking target and the tracking parameter of the tracking target are updated according to the matching result.

[0004] In summary, the existing face tracking method only performs Hungarian maximum matching algorithm matching based on the IOU matrix once, resulting in a high failure rate of face tracking. SUMMARY

[0005] The embodiments of the present application provide a face tracking method and device based on video images, equipment and medium, which are used to solve the problem that the existing face tracking method only performs Hungarian maximum matching algorithm matching based on the IOU matrix once, resulting in a high failure rate of face tracking.

[0006] In a first aspect, the embodiments of the present application provide a face tracking method based on video images, which comprises:

[0007] For at least one detection box in each frame of video image, according to the confidence of each detection box and the preset confidence threshold, the effective detection box in the video image is obtained;

[0008] According to each tracking target in the tracking queue and the effective detection box, a spatial matching cost function matrix is calculated, which is used to represent the value of each tracking target and each effective detection box. The tracking queue includes all tracking targets obtained by processing the last frame of the video image.

[0009] According to the spatial matching cost function matrix, the Hungarian maximum matching algorithm is used for matching to obtain a target matching result.

[0010] According to the target matching result, refresh the tracking parameters of each tracking target in the tracking queue, wherein the tracking parameters of each tracking target include: the coordinates of the detection box corresponding to the tracking target, the number of consecutive missed tracking of the tracking target, and the face feature vector set of the tracking target.

[0011] For each tracking target, if the number of consecutive missed tracking of the tracking target is greater than a preset number threshold, the tracking target is deleted from the tracking queue to obtain an intermediate tracking queue.

[0012] The targets corresponding to the remaining effective detection boxes in the effective detection boxes except the effective detection box corresponding to the target matching result are added to the intermediate tracking queue as new tracking targets to obtain a new tracking queue after processing the video image, and the initial tracking parameters of the new tracking targets are set.

[0013] In a specific embodiment, according to the spatial matching cost function matrix, the Hungarian maximum matching algorithm is used for matching to obtain a target matching result, which includes:

[0014] According to the spatial matching cost function matrix, the Hungarian maximum matching algorithm is used for matching to obtain a matching result.

[0015] If the matching result has only one group, the matching result is determined as the target matching result.

[0016] If the matching result includes at least two groups, according to the spatial matching cost function matrix, a face feature matching cost function matrix is calculated, which is used to represent the cost value between each tracking target and each effective detection box.

[0017] According to the face feature matching cost function matrix, the Hungarian maximum matching algorithm is used for matching again to obtain a new matching result.

[0018] If the new matching result has only one group, the new matching result is taken as the target matching result.

[0019] If the new matching result includes at least two groups, a group is randomly selected as the target matching result.

[0020] In a specific embodiment, according to each tracking target in the tracking queue and the effective detection box, a spatial matching cost function matrix is calculated, which includes:

[0021] For each tracking target in the tracking queue, according to the coordinates of the detection box in the tracking parameters of the tracking target, the perimeter of the detection box is calculated.

[0022] According to the perimeter, a radius of a circle with the same perimeter as the perimeter is calculated;

[0023] According to the number of consecutive missed tracking of the tracking parameter of the tracking target and the radius of the circle, a position radius is calculated;

[0024] For each valid detection frame, it is judged whether the center point of the valid detection frame is within a circle with the center of the detection frame in the tracking parameter of the tracking target as the center and the position radius as the radius;

[0025] If the center point of the valid detection frame is not within the circle, the value of the valid detection frame and the tracking target is set to positive infinity;

[0026] If the center point of the valid detection frame is within the circle, it is judged whether the perimeter of the valid detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy a preset condition, wherein the preset condition is Lm represents the perimeter of the valid detection frame, Ln represents the perimeter of the detection frame in the tracking parameter of the tracking target, Tn represents the number of consecutive missed tracking in the tracking parameter of the tracking target, and β represents a perimeter variation adjustment parameter;

[0027] If the preset condition is not satisfied, the value of the valid detection frame and the tracking target is set to positive infinity;

[0028] If the preset condition is satisfied, the value of the valid detection frame and the tracking target is set to 0;

[0029] According to the value of each valid detection frame and each tracking target, a spatial matching cost function matrix is generated.

[0030] In a specific embodiment, the face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, comprising:

[0031] The positions with a value of 0 in the spatial matching cost function matrix except the positions corresponding to the matching results are set to positive infinity to obtain a result matrix;

[0032] For the valid detection frame and the tracking target corresponding to any position with a value of 0 in the result matrix, the feature vector distance between the face feature vector in the valid detection frame and each face feature vector in the face feature vector set in the tracking parameter of the tracking target is calculated;

[0033] selecting a minimum feature vector distance from the feature vector distances as a target feature vector distance, and judging whether the target feature vector distance is less than a preset face similarity threshold value;

[0034] if the target feature vector distance is less than the face similarity threshold value, setting a generation value between the effective detection frame and the tracking target as positive infinity;

[0035] if the target feature vector distance is greater than or equal to the face similarity threshold value, setting the generation value between the effective detection frame and the tracking target as the target feature vector distance;

[0036] setting each 0 in the result matrix as a generation value between a corresponding effective detection frame and a tracking target, to obtain the face feature matching generation cost function matrix.

[0037] In a specific embodiment, the refreshing of the tracking parameter of each tracking target in the tracking queue according to the target matching result comprises:

[0038] for each tracking target in the tracking queue, if the target matching result indicates that the matching to the tracking target is unsuccessful, keeping the detection frame coordinates and the face feature set in the tracking parameter of the tracking target unchanged, and adding 1 to the continuous missing tracking number;

[0039] if the target matching result indicates that the matching to the tracking target is successful, setting the continuous missing tracking number in the tracking parameter of the tracking target as 0, and reacquiring the detection frame coordinates in the tracking parameter of the tracking target, and refreshing the face feature set in the tracking parameter of the tracking target.

[0040] In a specific embodiment, the refreshing of the face feature set in the tracking parameter of the tracking target comprises:

[0041] acquiring a face feature vector, a face angle and a definition value corresponding to the tracking target according to a face feature vector extraction model and a definition detection model;

[0042] acquiring a minimum value ε of an absolute value of a difference between the face angle of the tracking target and 0, 30, 60, 90, 120, 150 and 180, and establishing a mapping relationship with the face feature vector of the tracking target, the face angle of the tracking target and the definition value of the tracking target and storing the mapping relationship;

[0043] acquiring a target angle interval to which the face angle of the tracking target belongs, the target angle interval being any interval of [0, 30), [30, 60), [60, 90), [90, 120), [120, 150), [150, 180].

[0044] obtaining a face angle corresponding to each face feature vector in a face feature vector set in the tracking parameter of the tracking target, and determining whether there is a face angle belonging to the target angle interval;

[0045] If there is no face angle belonging to the target angle interval, a face feature vector corresponding to the face angle of the tracking target is added to the face feature vector set in the tracking parameter of the tracking target.

[0046] In a specific embodiment, the method further comprises:

[0047] If there is a face angle belonging to the target angle interval, it is determined whether the ε corresponding to the face angle of the tracking target is less than the ε corresponding to the face angle belonging to the target angle interval;

[0048] If the ε corresponding to the face angle of the tracking target is less than the ε corresponding to the face angle belonging to the target angle interval, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval;

[0049] If the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target;

[0050] If the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded.

[0051] In a specific embodiment, the method further comprises:

[0052] If the ε corresponding to the face angle of the tracking target is not less than the ε corresponding to the face angle belonging to the target angle interval, it is determined whether the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within a preset numerical range;

[0053] If the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within the preset numerical range, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval;

[0054] if the face angle corresponding to the tracking target corresponds to a sharpness value greater than the face angle corresponding to the target angle interval, then in the face feature vector set of the tracking parameter of the tracking target, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target;

[0055] if the difference between the epsilon corresponding to the face angle of the tracking target and the epsilon corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or if the face angle corresponding to the tracking target corresponds to a sharpness value less than or equal to the face angle corresponding to the target angle interval, then the face feature vector corresponding to the face angle of the tracking target is discarded.

[0056] In a second aspect, an embodiment of the present application provides a face tracking device based on video images, comprising:

[0057] The acquisition module is configured to, for each detection frame in each frame of video images, acquire an effective detection frame in the video images according to the confidence of each detection frame and a preset confidence threshold.

[0058] The processing module is configured to calculate a spatial matching cost function matrix according to each tracking target in a tracking queue and the effective detection frame, the spatial matching cost function matrix being used to represent the cost value between each tracking target and each effective detection frame, the tracking queue including all tracking targets obtained by processing the previous frame of the video images.

[0059] The processing module is further configured to acquire a target matching result by using a Hungarian maximum matching algorithm to match according to the spatial matching cost function matrix.

[0060] The processing module is further configured to refresh the tracking parameter of each tracking target in the tracking queue according to the target matching result, wherein the tracking parameter of each tracking target includes the coordinates of the detection frame corresponding to the tracking target, the continuous missing tracking number of the tracking target, and the face feature vector set of the tracking target.

[0061] The processing module is further configured to, for each tracking target, delete the tracking target from the tracking queue if the continuous missing tracking number of the tracking target is greater than a preset number threshold, to obtain an intermediate tracking queue.

[0062] The processing module is further configured to add a target corresponding to a remaining valid detection frame, except for the valid detection frame corresponding to the target matching result, into the intermediate tracking queue as a new tracking target, to obtain a new tracking queue after processing the video image, and set an initial tracking parameter of the new tracking target.

[0063] In a third aspect, an electronic device is provided, including:

[0064] a processor, a memory, and a communication interface;

[0065] The memory is configured to store executable instructions of the processor.

[0066] The processor is configured to execute the executable instructions to implement the face tracking method based on a video image according to any one of the first aspect.

[0067] In a fourth aspect, a readable storage medium is provided, and the readable storage medium stores a computer program, and the computer program is configured to implement the face tracking method based on a video image according to any one of the first aspect when executed by a processor.

[0068] In a fifth aspect, a computer program product is provided, and the computer program product includes a computer program, and the computer program is configured to implement the face tracking method based on a video image according to any one of the first aspect when executed by a processor.

[0069] The face tracking method based on a video image, the device, the electronic device, and the readable storage medium provided by the embodiments of the present application filter out valid detection frames, and then obtain a spatial matching cost function matrix according to the valid detection frames and tracking targets, and then combine a Hungarian maximum matching algorithm to obtain a matching result. If there are multiple groups of matching results, a face feature matching cost function matrix is further obtained, and the Hungarian maximum matching algorithm is combined again to obtain a matching result. Finally, the tracking parameters are refreshed and the tracking queue is updated according to the matching result. The present application obtains the spatial matching cost function matrix and the face feature matching cost function matrix, and uses the Hungarian maximum matching algorithm to obtain the matching result, thereby effectively reducing the failure rate of face tracking. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0071] Figure 1aThe scene schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0072] Figure 1b The architecture schematic diagram one of the face tracking method based on video images provided by the present application is shown in the figure.

[0073] Figure 1c The architecture schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure. Figure Two ;

[0074] Figure 2 The flow schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0075] Figure 3 The flow schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0076] Figure 4a The flow schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0077] Figure 4b The detection frame position diagram in the tracking parameter of the effective detection frame and the tracking target provided by the present application is shown in the figure.

[0078] Figure 5 The flow schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0079] Figure 6a The flow schematic diagram of the face tracking method based on video images provided by the present application is shown in the figure.

[0080] Figure 6b The face angle schematic diagram provided by the present application is shown in the figure.

[0081] Figure 7 The structure schematic diagram of the face tracking device provided by the present application is shown in the figure.

[0082] Figure 8 The structure schematic diagram of the electronic device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments made by those skilled in the art according to the inspiration of the present embodiments are within the scope of protection of the present application.

[0084] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms here is not meant to limit the scope of the application to the precise terms used. It should be further understood that the data used here to describe the embodiments of the present application is for illustrative purposes only and is not meant to limit the scope of the application to the specific embodiments presented. Furthermore, the terms "comprise" and "include", and their conjugates, are intended to cover non-exclusive inclusions, such that processes, methods, systems, products, or apparatuses that comprise, include, or are

[0085] With the development of society and technology, the technology of face tracking is becoming more and more mature, and the application around the face tracking technology is also increasing, for example, monitoring the crowd in public places, realizing people flow statistics, tracking specific characters, etc. The application field is also more and more, for example, security, retail and other fields. For the video obtained by fixed camera, mobile camera and other ways, how to obtain and identify the face, obtain the face feature, and track the face is the problem that people want to solve.

[0086] After the gateway obtains the video, a plurality of frame images are extracted, the current frame image is identified to obtain a detection box and a corresponding target. Then, the tracking target determined in the last frame is cascaded and matched. For the detection box and the tracking target that are not cascaded and matched, an Intersection over Union (IOU) matrix is calculated, the Hungarian maximum matching algorithm is used based on the IOU matrix to perform matching, and a final matching result is obtained. Finally, the tracking target and the tracking parameter of the tracking target are updated according to the matching result. The existing face tracking method only performs one Hungarian maximum matching algorithm based on the IOU matrix to perform matching, which leads to a high failure rate of face tracking.

[0087] In view of the problems in the prior art, the inventors found, in the process of researching a face tracking method based on video images, that after an electronic device for face tracking acquires a video, a number of frame images are extracted from the video, the images are processed to obtain a detection frame and a corresponding confidence, the detection frame is filtered using the confidence to obtain an effective detection frame, and a spatial matching cost function matrix can be obtained in combination of the effective detection frame and a tracking target. The spatial matching cost function matrix is processed using a Hungarian maximum matching algorithm to obtain a matching result. When there is only one matching result, the matching result is taken as a target matching result. When there are multiple matching results, a face feature matching cost function matrix can be obtained from the spatial matching cost function matrix, the face feature matching cost function matrix is processed using the Hungarian maximum matching algorithm again to obtain a new matching result. If the new matching result has only one group, the new matching result is taken as the target matching result. If the new matching result has multiple groups, one group is randomly selected from the new matching result as the target matching result. Finally, the tracking parameters of the tracking target are refreshed and the tracking queue is updated according to the matching result. Based on the above inventive concept, a face tracking scheme based on video images is designed in the present application.

[0088] Exemplarily, Figure 1a A scene schematic diagram of a face tracking method based on video images provided by the present application is shown in Figure 1a The application scenario can include a camera device 11, an electronic device 12, and a server 13.

[0089] Exemplarily, in Figure 1a The present scheme is applied to the electronic device 12, which is a gateway, in the application scenario shown in the figure. The camera device 11 is a camera, which can be a camera installed in a fixed position or a mobile camera.

[0090] The camera device 11 can take a video and send the video to the electronic device 12 for processing.

[0091] After receiving the video, the electronic device 12 processes the video, extracts a number of frame images, obtains an effective detection frame, obtains a spatial matching cost function matrix, performs matching using a Hungarian maximum matching algorithm, obtains a matching result, refreshes tracking parameters and a tracking queue, and finally sends the tracking queue and the tracking parameters to the server 13.

[0092] The server 13 can receive the tracking queue and the tracking parameters sent by the electronic device 12 and further process them as needed.

[0093] It should be noted that the electronic device 12 can acquire the video in the following manner: receiving the video sent by the camera device 11, or inputting the video into the electronic device 12 by a staff, and then the electronic device 12 can directly acquire the video. The embodiment of the present application does not limit the manner in which the electronic device 12 acquires the video, and the manner can be selected according to actual conditions.

[0094] It should be noted that the electronic device 12 obtains the new tracking queue and the tracking parameter, and then completes the face tracking of the frame image. Further processing of the new tracking queue and the tracking parameter is not limited to being sent to the server 13, but the electronic device 12 can directly use the new tracking queue and the tracking parameter to process according to requirements. The embodiment of the present application does not specifically limit the manner in which the new tracking queue and the tracking parameter are further processed, and the manner can be selected according to actual conditions.

[0095] It should be noted that the camera device 11 is an electronic device having a video shooting and sending function, which can be a camera having a sending function, or a smart phone. The embodiment of the present application does not specifically limit the camera device 11, and the camera device 11 can be selected according to actual conditions.

[0096] It should be noted that the electronic device 12 is a device capable of tracking a face according to a video, which can be a gateway or a server. The embodiment of the present application does not specifically limit the electronic device, and the electronic device can be selected according to actual conditions.

[0097] It should be noted that, Figure 1a The above is only a schematic diagram of an application scenario provided by the embodiment of the present application, and the embodiment of the present application does not limit the actual forms of various devices included in the Figure 1a The embodiment of the present application does not limit the actual forms of various devices included in the Figure 1a The embodiment of the present application does not limit the actual forms of various devices included in the

[0098] The technical solutions of the present application will be described in detail in the following specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments.

[0099] In the process of executing the present scheme by the electronic device, first, a video to be processed is acquired, and a plurality of frame video images are extracted from the video. After obtaining the video images, the video images are processed by using a neural network model, so that a detection frame and a confidence corresponding to the detection frame are obtained in the video images, and the face target is displayed in the detection frame.

[0100] For example, Figure 1b The architecture of the face tracking method based on a video image provided by the present application is shown in Fig. 1, Figure 1cThe architecture schematic of the face tracking method based on video images provided by the present application Figure Two .

[0101] As shown in Figure 1b , before the face tracking scheme of the present application is performed, face detection is performed on the video images by a neural network model to obtain a detection frame and a confidence corresponding to the detection frame, and then face tracking is performed. In the process of face tracking, a face feature vector extraction model is required to extract face feature vectors of the target in the effective detection frame or the tracking target.

[0102] As shown in Figure 1c , before the face tracking scheme of the present application is performed, face detection is performed on the video images by a neural network model to obtain a detection frame and a confidence corresponding to the detection frame, and then face tracking is performed. In the process of face tracking, after the spatial matching cost function matrix is obtained, the Hungarian matching algorithm is used for Hungarian matching. If the matching result is only one group, the face angle is extracted by using the face feature vector extraction model when refreshing the tracking parameters of the tracking target, and then the refreshing of the tracking parameters is completed. If the matching result is at least two groups, the face feature vectors of the targets in the effective detection frame are further extracted according to the face feature vector extraction model, and then the face feature matching cost function matrix is obtained. The Hungarian matching algorithm is used again for Hungarian matching, and the tracking parameters of the tracking target are refreshed according to the matching result. The face angle is extracted by using the face feature vector extraction model, and then the refreshing of the tracking parameters is completed.

[0103] It should be noted that the way of extracting a plurality of frames of video images from the video can be random extraction, equal interval extraction, or full extraction. The present application does not limit the way of extracting a plurality of frames of video images from the video, and the actual situation can be selected.

[0104] Figure 2 The flowchart of the first embodiment of the face tracking method based on video images provided by the present application is shown in Figure 2 , which specifically includes the following steps:

[0105] S201: For each detection frame in each frame of video images, the effective detection frame in the video images is obtained according to the confidence of each detection frame and the preset confidence threshold.

[0106] In this step, after the electronic device obtains the detection frame of the current frame video image, since each detection frame has a corresponding confidence, the confidence corresponding to each detection frame is compared with a preset confidence threshold in size, if the confidence of the detection frame is greater than the preset confidence threshold, the detection frame is retained and set as a valid detection frame; if the confidence of the detection frame is less than or equal to the preset confidence threshold, the detection frame is discarded.

[0107] It should be noted that the preset confidence threshold is set in the electronic device by the staff before the face tracking scheme is performed, and is used for size comparison with the confidence of each detection frame. The present application does not specifically limit the preset confidence threshold, which can be set according to the actual situation.

[0108] S202: Calculate a spatial matching cost function matrix according to each tracking target in the tracking queue and the valid detection frame.

[0109] In this step, after the electronic device obtains the valid detection frame of the current frame video image, the spatial matching cost function matrix can be calculated according to each tracking target in the tracking queue, the tracking parameter of each tracking target and the valid detection frame. The spatial matching cost function matrix is used to represent the cost value between each tracking target and each valid detection frame, and the tracking queue includes all tracking targets obtained by processing the previous frame image of the video image.

[0110] S203: Match using the Hungarian maximum matching algorithm according to the spatial matching cost function matrix to obtain a target matching result.

[0111] In this step, after the electronic device calculates and obtains the spatial matching cost function matrix, the Hungarian maximum matching algorithm can be used for matching to obtain a target matching result.

[0112] Specifically, the Hungarian maximum matching algorithm is used for matching based on the spatial matching cost function matrix to obtain a matching result. If the matching result has only one group, it is taken as the target matching result. If the matching result includes at least two groups, a face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, and the Hungarian maximum matching algorithm is used again for matching to obtain a new matching result. If the new matching result has only one group, the new matching result is taken as the target matching result. If the new matching result includes at least two groups, a group is randomly selected as the target matching result.

[0113] S204: Refresh the tracking parameter of each tracking target in the tracking queue according to the target matching result.

[0114] In this step, after the electronic device obtains the target matching result, the tracking parameters of each tracking target in the tracking queue can be refreshed according to the target matching result. The tracking parameters of each tracking target include the coordinates of the detection box corresponding to the tracking target, the number of consecutive missed tracking of the tracking target, and the face feature vector set of the tracking target.

[0115] Specifically, for each tracking target in the tracking queue, if the target matching result indicates that the tracking target is not successfully matched, the coordinates of the detection box and the face feature set in the tracking parameters of the tracking target are kept unchanged, and the number of consecutive missed tracking is increased by 1.

[0116] If the target matching result indicates that the tracking target is successfully matched, the number of consecutive missed tracking in the tracking parameters of the tracking target is set to 0; the coordinates of the detection box in the tracking parameters of the tracking target are re-acquired, and the re-acquired coordinates of the detection box are the coordinates of the valid detection box matched with the tracking target; the face feature set in the tracking parameters of the tracking target is refreshed, and the face feature set is refreshed according to the face angle of the tracking target and the face angle corresponding to each face feature vector in the face feature vector set in the tracking parameters of the tracking target.

[0117] It should be noted that the coordinates of the detection box in the tracking parameters of the tracking target can be the coordinates of the four corners of the detection box, or the coordinates of the upper left corner and the lower right corner. The present embodiment does not specifically limit the coordinates of the detection box, which can be selected according to actual conditions.

[0118] S205: For each tracking target, if the number of consecutive missed tracking of the tracking target is greater than a preset number threshold, the tracking target is deleted from the tracking queue, and an intermediate tracking queue is obtained.

[0119] In this step, after the electronic device refreshes the tracking parameters of each tracking target, for each tracking target, whether it is deleted from the tracking queue can be determined according to the number of consecutive missed tracking in the tracking parameters. If the number of consecutive missed tracking of the tracking target is greater than a preset number threshold, the tracking target is deleted from the tracking queue; if the number of consecutive missed tracking of the tracking target is less than or equal to the preset number threshold, the tracking target is retained in the tracking queue, thereby obtaining an intermediate tracking queue.

[0120] It should be noted that the preset number threshold is set in the electronic device by the staff before the face tracking scheme is performed, and is used for comparison with the number of consecutive missed tracking of the tracking target. The present embodiment does not specifically limit the preset number threshold, which can be set according to actual conditions.

[0121] S206: add the target corresponding to the remaining effective detection frame of the effective detection frame corresponding to the target matching result in the new tracking queue as a new tracking target, and set the initial tracking parameter of the new tracking target.

[0122] In this step, the electronic device further processes the intermediate tracking queue after obtaining the intermediate tracking queue. In all effective detection frames of the current frame video image, remove the remaining effective detection frame of the detection frame corresponding to the target matching result, add the target in the remaining detection frame as a new tracking target to the intermediate tracking queue, obtain the new tracking queue after processing the video image, and set the initial tracking parameter of the new tracking target.

[0123] For example, all effective detection frames of the current frame video image are M1, M2, M3, M4, M5, M6, and the detection frame corresponding to the target matching result is M2, M4, M5, M6. Then, the targets corresponding to M1 and M3 are added to the intermediate tracking queue as new tracking targets.

[0124] The way to set the initial tracking parameter of the new tracking target is to set the continuous missing tracking number in the tracking parameter of the new tracking target to 0; the detection frame coordinates in the tracking parameter of the new tracking target are the coordinates of the effective detection frame corresponding to the tracking target; an empty face feature vector set is established, and the face feature vector is obtained through the face feature vector extraction model and put into the empty face feature vector set to obtain the face feature vector set of the tracking target.

[0125] In addition, the face angle of the tracking target can also be obtained through the face feature vector extraction model, and the clarity value of the tracking target can be obtained according to the clarity detection model. The mapping relationship between the face feature vector of the tracking target, the face angle of the tracking target, and the clarity value of the tracking target is established and stored.

[0126] It should be noted that for the first frame image extracted from the video, the electronic device obtains the effective detection frame, establishes an empty tracking queue, adds the target in each effective detection frame as a tracking target to the tracking queue, and sets the initial tracking parameter for each tracking target.

[0127] The face tracking method based on video images provided in the embodiment can calculate and obtain a spatial matching cost function matrix by screening valid detection boxes and combining tracking parameters of tracking targets, match the target matching result by using the Hungarian maximum matching algorithm, and then update the tracking parameters of the tracking targets and the tracking queue according to the matching result. Compared with the prior art that matches by using the Hungarian maximum matching algorithm based on an IOU matrix, the present scheme matches by using the Hungarian maximum matching algorithm based on a spatial matching cost function matrix, effectively reducing the face tracking failure rate.

[0128] Figure 3 The flowchart of the second embodiment of the face tracking method based on video images provided in the present application is shown in FIG. 2, which is based on the first embodiment and the step S203 in the first embodiment specifically includes the following steps: Figure 3

[0129] S301: Match by using the Hungarian maximum matching algorithm based on the spatial matching cost function matrix to obtain a matching result.

[0130] In this step, after the electronic device obtains the spatial matching cost function matrix, the Hungarian maximum matching algorithm can be used to match to obtain a matching result.

[0131] For example, the matching result is in the form of (M1, N1), (M2, N3), (M4, N2), where M1, M2, and M4 represent different valid detection boxes, and N1, N3, and N2 represent different tracking targets. The number of valid detection boxes corresponding to the matching result is less than or equal to the number of valid detection boxes of the current frame of video images, and the number of tracking targets corresponding to the matching result is less than or equal to the number of tracking targets in the tracking queue.

[0132] S302: If the matching result is only one group, the matching result is determined as the target matching result.

[0133] In this step, after the electronic device obtains the matching result, it needs to determine whether the matching result is one group or multiple groups. If the matching result is only one group, the matching result is determined as the target matching result.

[0134] It should be noted that, to determine whether the matching result is one group or multiple groups, it is necessary to determine whether the valid detection boxes and the tracking targets in the matching result are in a one-to-one correspondence. If one valid detection box corresponds to only one tracking target, it is determined that the matching result is one group. If one valid detection box corresponds to two or more tracking targets, or two or more valid detection boxes correspond to one tracking target, it is determined that the matching result is multiple groups.

[0135] ​For example, if the matching result is (M1, N1), (M2, N3), (M4, N2), each valid detection frame corresponds to one tracking target, which indicates that the matching result is one group. If the matching result is (M1, N1), (M2, N3), (M4, N2), (M1, N5), the valid detection frame M1 corresponds to the tracking targets N1 and N5, which indicates that the matching result is multiple groups. If the matching result is (M1, N1), (M2, N3), (M4, N2), (M3, N2), the valid detection frames M4 and M3 correspond to the tracking target N2, which indicates that the matching result is multiple groups.

[0136] S303: If the matching result includes at least two groups, a face feature matching cost function matrix is calculated according to a spatial matching cost function matrix.

[0137] In this step, the electronic device determines that the matching result is multiple groups, i.e., includes at least two groups, and then sets the positions with a value of 0 in the spatial matching cost function matrix to positive infinity except for the positions corresponding to the matching result, to obtain a result matrix. Then, the cost value of each valid detection frame and tracking target corresponding to any position with a value of 0 in the result matrix is calculated and set in the original position, to obtain a face feature matching cost function matrix. The face feature matching cost function matrix is used to represent the cost value between each tracking target and each valid detection frame.

[0138] S304: According to the face feature matching cost function matrix, the Hungarian maximum matching algorithm is used for matching again to obtain a new matching result.

[0139] S305: If the new matching result is only one group, the new matching result is taken as the target matching result.

[0140] S306: If the new matching result includes at least two groups, a group is randomly selected as the target matching result.

[0141] In the above steps, after the electronic device obtains the face feature matching cost function matrix, the Hungarian maximum matching algorithm is used for matching again to obtain a new matching result. It is determined whether the new matching result is only one group or multiple groups. If the new matching result is only one group, the new matching result is taken as the target matching result. If the new matching result includes at least two groups, a group is randomly selected as the target matching result.

[0142] It should be noted that in the target matching result, each valid detection frame corresponds to one tracking target, and each tracking target corresponds to one valid detection frame.

[0143] For example, the new matching result is (M1, N1), (M2, N3), (M4, N2), (M1, N5), and it is indicated that the matching result is two groups, which are (M1, N1), (M2, N3), (M4, N2) and (M1, N5), (M2, N3), (M4, N2). The new matching result is (M1, N1), (M1, N5), (M2, N3), (M2, N1), (M4, N2), and it is indicated that the matching result is three groups, which are (M1, N1), (M2, N3), (M4, N2), (M1, N5), (M2, N3), (M4, N2) and (M1, N5), (M2, N1), (M4, N2).

[0144] The face tracking method based on video images provided in the embodiment is matched by using the Hungarian maximum matching algorithm based on the space matching cost function matrix, and after the matching result is obtained, if the matching result is multiple groups, the face feature matching cost function matrix needs to be obtained again, and the Hungarian maximum matching algorithm is used again to obtain a new matching result, and the target matching result is obtained according to the new matching result. The scheme matches twice by using the Hungarian maximum matching algorithm, and effectively reduces the face tracking failure rate.

[0145] Figure 4a The flowchart of the third face tracking method based on video images provided in the embodiment is shown in FIG. 6. Figure 4a The step S202 in the first embodiment specifically includes the following steps.

[0146] S401: For each tracking target in the tracking queue, the perimeter of the detection frame is calculated according to the coordinates of the detection frame in the tracking parameter of the tracking target.

[0147] After the electronic device obtains the valid detection frame, the space matching cost function matrix needs to be calculated further. The electronic device obtains a new tracking queue after processing the previous frame of video images, and the tracking queue includes all tracking targets obtained by processing the previous frame of images. Each tracking target has a tracking parameter, and the tracking parameter includes: the coordinates of the detection frame corresponding to the tracking target, the number of consecutive missed tracking of the tracking target, and the face feature vector set of the tracking target.

[0148] In this step, for each tracking target in the tracking queue, the coordinates of the detection frame of the tracking target are obtained from the tracking parameter of the tracking target, and the perimeter of the detection frame can be obtained according to the coordinates.

[0149] S402: The radius of the circle with the same perimeter as the perimeter is calculated according to the perimeter.

[0150] S403: Calculate the position radius according to the consecutive missing tracking times in the tracking parameters of the tracking target and the radius of the circle.

[0151] In the above step, after the electronic device obtains the perimeter of the detection frame in the tracking parameters of the tracking target, the perimeter is divided by 2π to obtain the radius of the circle with the same perimeter as the detection frame. Then the formula is used to calculate the position radius, where RR represents the position radius, Tn represents the consecutive missing tracking times in the tracking parameters of the tracking target, and R represents the radius of the circle with the same perimeter. represents the motion speed adjustment parameter.

[0152] It should be noted that the motion speed adjustment parameter depends on the lateral motion speed of the tracking target and the video frame extraction rate. The motion speed adjustment parameter can be set in the electronic device by the staff before the face tracking scheme is performed, and the electronic device can directly obtain the motion speed adjustment parameter. The motion speed adjustment parameter can also be stored in the electronic device by the staff before the face tracking scheme is performed, and the electronic device can obtain the motion speed adjustment parameter by directly obtaining the lateral motion speed of the tracking target and the video frame extraction rate. The embodiments of the present application do not limit the obtaining method of the motion speed adjustment parameter, and the obtaining method can be selected according to the actual situation.

[0153] S404: For each valid detection frame, it is judged whether the center point of the valid detection frame is in the circle with the center of the detection frame in the tracking parameters of the tracking target as the center and the position radius as the radius.

[0154] S405: If the center point of the valid detection frame is not in the circle, the value of the valid detection frame and the tracking target is set to positive infinity.

[0155] In the above step, after the electronic device obtains the position radius, for each valid detection frame, it is judged whether the center point of the valid detection frame is in the circle with the center of the detection frame in the tracking parameters of the tracking target as the center and the position radius as the radius. If the center point of the valid detection frame is not in the circle, the value of the valid detection frame and the tracking target is set to positive infinity.

[0156] For example, Figure 4b The present application provides a detection frame and tracking target tracking parameter position diagram. As shown in Figure 4b The diagram includes: the detection frame 41 in the tracking parameters of the tracking target, the valid detection frame 42, the valid detection frame 43, the valid detection frame 44, and the circle 45 with the center of the detection frame in the tracking parameters of the tracking target as the center and the position radius as the radius. From Figure 4bAs can be seen, the center points of the effective detection frame 42 and the effective detection frame 43 are in the circle 45, and the center point of the effective detection frame 44 is not in the circle 45. At this time, the penalty value between the effective detection frame 44 and the tracking target can be set to positive infinity.

[0157] S406: If the center point of the effective detection frame is in the circle, it is determined whether the perimeter of the effective detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy a preset condition.

[0158] S407: If the preset condition is not satisfied, the penalty value between the effective detection frame and the tracking target is set to positive infinity.

[0159] In the above steps, if the electronic device determines that the center point of the effective detection frame is in the circle, it is further determined whether the perimeter of the effective detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy a preset condition. If the preset condition is not satisfied, the penalty value between the effective detection frame and the tracking target is set to positive infinity.

[0160] The preset condition is Lm represents the perimeter of the effective detection frame, Ln represents the perimeter of the detection frame in the tracking parameter of the tracking target, Tn represents the number of consecutive missed tracking times in the tracking parameter of the tracking target, and β represents a perimeter variation adjustment parameter.

[0161] It should be noted that the perimeter variation adjustment parameter depends on the front and rear motion speed of the tracking target and the video frame extraction rate. The perimeter variation adjustment parameter can be set in the electronic device by the staff before the face tracking scheme is performed, and the electronic device can directly obtain the perimeter variation adjustment parameter. The perimeter variation adjustment parameter can also be stored in the electronic device by the staff before the face tracking scheme is performed, and the perimeter variation adjustment parameter can be obtained by the electronic device through the directly obtained front and rear motion speed of the tracking target and the video frame extraction rate. The embodiment of the present application does not limit the acquisition method of the perimeter variation adjustment parameter, and the acquisition method of the perimeter variation adjustment parameter can be selected according to the actual situation.

[0162] S408: If the preset condition is satisfied, the penalty value between the effective detection frame and the tracking target is set to 0.

[0163] S409: A spatial matching cost function matrix is generated according to the penalty value between each effective detection frame and each tracking target.

[0164] In the above step, the electronic device judges that the perimeter of the effective detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy the preset condition, sets the value of the generation between the effective detection frame and the tracking target to 0, and generates the spatial matching cost function matrix according to the value of the generation between each effective detection frame and each tracking target.

[0165] For example, the generated spatial matching cost function matrix is shown in Table 1:

[0166] N1 N2 N3 N4 M1 +∞ 0 +∞ 0 M2 0 +∞ +∞ 0 M3 0 0 0 +∞

[0167] As shown in Table 1, the value of the generation between the effective detection frame M1 and the tracking target N1 is positive infinity, and the value of the generation between the tracking target N2 is 0; the value of the generation between the effective detection frame M3 and the tracking target N3 is 0, and the value of the generation between the tracking target N4 is positive infinity.

[0168] It should be noted that Table 1 is only an example of the generated spatial matching cost function matrix, and does not limit the spatial matching cost function matrix, which is generated according to the actual situation in the actual process.

[0169] The face tracking method based on video images provided in the embodiment determines the value of the generation between each effective detection frame and each tracking target by judging whether the center point of the effective detection frame is in the circle with the center of the detection frame in the tracking parameter of the tracking target as the center and the position radius as the radius, and whether the perimeter of the effective detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy the preset condition, and further generates the spatial matching cost function matrix. The matching result is obtained by matching using the matrix, which effectively reduces the failure rate of face tracking.

[0170] Figure 5 The flowchart of the face tracking method based on video images provided in the fourth embodiment of the application is shown in Figure 5 The step S303 in the second embodiment specifically includes the following steps:

[0171] S501: Set the positions with values of 0 in the spatial matching cost function matrix to positive infinity except the positions corresponding to the matching result, to obtain a result matrix.

[0172] When the matching result obtained by the electronic device by matching based on the spatial matching cost function matrix using the Hungarian maximum matching algorithm has multiple groups, the face feature matching cost function matrix needs to be calculated according to the spatial matching cost function matrix.

[0173] In this step, the spatial matching cost function matrix is first processed, and the positions with values of 0 in the spatial matching cost function matrix are set to positive infinity except the positions corresponding to the matching result, to obtain a result matrix.

[0174] For example, the space matching cost function matrix is as shown in Table 1 in Embodiment Three, the matching results are (M1, N2), (M2, N4), (M3, N1), (M3, N3), and the result matrix is as shown in Table 2:

[0175] N1 N2 N3 N4 M1 +∞ 0 +∞ +∞ M2 +∞ +∞ +∞ 0 M3 0 +∞ 0 +∞

[0176] It should be noted that Table 2 is only an example of the obtained result matrix, and does not limit the result matrix, which is obtained according to actual conditions in actual process.

[0177] S502: For the effective detection frame and the tracking target corresponding to any value of 0 in the result matrix, calculating a feature vector distance between a face feature vector of a target in the effective detection frame and each face feature vector in a face feature vector set in a tracking parameter of the tracking target.

[0178] In this step, after the electronic device obtains the result matrix, for the effective detection frame and the tracking target corresponding to any value of 0 in the result matrix, the face feature vector of the target in the effective detection frame can be calculated according to the face feature vector extraction model. Then the feature vector distance between the face feature vector of the target in the effective detection frame and each face feature vector in the face feature vector set in the tracking parameter of the tracking target can be calculated. In order to subsequently select the smallest feature vector distance from them, and then determine whether it is less than the preset face similarity threshold.

[0179] S503: Selecting the smallest feature vector distance as a target feature vector distance from the feature vector distances, and determining whether the target feature vector distance is less than the preset face similarity threshold.

[0180] S504: If the target feature vector distance is less than the face similarity threshold, setting the cost value between the effective detection frame and the tracking target as positive infinity.

[0181] S505: If the target feature vector distance is greater than or equal to the face similarity threshold, setting the cost value between the effective detection frame and the tracking target as the target feature vector distance.

[0182] In the above steps, after the electronic device obtains a plurality of feature vector distances, the smallest feature vector distance can be selected as the target feature vector distance, and then the target feature vector distance and the preset face similarity threshold are compared in size. If the target feature vector distance is less than the face similarity threshold, the cost value between the effective detection frame and the tracking target is set as positive infinity; if the target feature vector distance is greater than or equal to the face similarity threshold, the cost value between the effective detection frame and the tracking target is set as the target feature vector distance.

[0183] It should be noted that the preset face similarity threshold is related to the accuracy of the face feature extraction model, and is set in the electronic device by the staff before the face tracking scheme is performed, and is used for comparing the size of the target feature vector distance. The preset face similarity threshold is not specifically limited in the embodiment of the application, and can be set according to actual conditions.

[0184] S506: Set each 0 in the result matrix as the generation value between the corresponding effective detection frame and the tracking target, to obtain a face feature matching generation function matrix.

[0185] In this step, after the electronic device sets each 0 in the result matrix as the generation value between the corresponding effective detection frame and the tracking target, the 0 can be changed to the corresponding generation value, to obtain a face feature matching generation function matrix.

[0186] For example, the result matrix is as shown in Table 2 in Embodiment Four, the generation value is reacquired for the effective detection frame and the tracking target corresponding to the position with a value of 0, the generation value between M1 and N2 is Fmin1, the generation value between M2 and N4 is Fmin2, the generation value between M3 and N1 is positive infinity, and the generation value between M3 and N3 is Fmin3, and the face feature matching generation function matrix is as shown in Table 3:

[0187] N1 N2 N3 N4 M1 +∞ Fmin1 +∞ +∞ M2 +∞ +∞ +∞ Fmin2 M3 +∞ +∞ Fmin3 +∞

[0188] It should be noted that Table 3 is only an example of the obtained face feature matching generation function matrix, and does not limit the face feature matching generation function matrix, which is obtained according to actual conditions in the actual process.

[0189] The face tracking method based on video images provided in the embodiment selects the minimum value in the feature vector distance between the face feature vector of the target in the effective detection frame and each face feature vector in the face feature vector set in the tracking parameter of the tracking target as the target feature vector distance. Then, the generation value between the effective detection frame and the tracking target is determined according to the size of the target feature vector distance and the preset face similarity threshold, and then a face feature matching generation function matrix is obtained. The face feature matching generation function matrix is used for matching, which effectively reduces the failure rate of face tracking.

[0190] Figure 6a The flowchart of the face tracking method based on video images provided in Embodiment Five of the application is as shown in Figure 6a After the electronic device obtains the target matching result, the tracking parameter of the tracking target needs to be refreshed according to the target matching result. For the tracking target with a successful matching result, the face feature set in the tracking parameter is refreshed on the basis of the above-mentioned embodiments, and the specific steps include the following steps:

[0191] S601: Obtain the face feature vector, the face angle and the definition value of the tracking target according to the face feature vector extraction model and the definition detection model.

[0192] S602: Obtain the minimum value ε of the absolute value of the difference between the face angle of the tracking target and 0, 30, 60, 90, 120, 150 and 180, and establish a mapping relationship with the face feature vector of the tracking target, the face angle of the tracking target and the definition value of the tracking target and store them.

[0193] In the above steps, for the tracking target that matches successfully, when refreshing the face feature vector set in the tracking parameter of the tracking target, the face feature vector extraction model and the definition detection model are used to process it, the face feature vector, the face angle and the definition value of the tracking target can be obtained. Then the absolute value of the difference between the face angle of the tracking target and 0, 30, 60, 90, 120, 150 and 180 is calculated, and the minimum value ε is selected, and the face feature vector of the tracking target, the face angle of the tracking target and the definition value of the tracking target are established a mapping relationship and stored, so that in the process of processing the next frame of video image, the corresponding face angle can be found through the feature vector in the face feature vector set in the tracking parameter of the tracking target.

[0194] S603: Obtain the target angle interval to which the face angle of the tracking target belongs.

[0195] In this step, after the electronic device obtains the face angle of the tracking target, the target angle interval to which the face angle of the tracking target belongs is further obtained. The target angle interval is any interval in [0, 30), [30, 60), [60, 90), [90, 120), [120, 150), [150, 180].

[0196] S604: Obtain the face angle corresponding to each face feature vector in the face feature vector set in the tracking parameter of the tracking target, and determine whether there is a face angle belonging to the target angle interval.

[0197] S605: If there is no face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is added to the face feature vector set in the tracking parameter of the tracking target.

[0198] In the above step, after the electronic device obtains the target face angle to which the face angle of the tracking target belongs, the electronic device further needs to obtain the face angle corresponding to each face feature vector in the face feature vector set in the tracking parameter of the tracking target, and determine whether there is a face angle belonging to the target angle interval. If there is no face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is added to the face feature vector set in the tracking parameter of the tracking target, and the refreshing of the face feature vector set is completed.

[0199] S606: If there is a face angle belonging to the target angle interval, it is determined whether the epsilon corresponding to the face angle of the tracking target is less than the epsilon corresponding to the face angle belonging to the target angle interval.

[0200] In this step, if the electronic device determines that there is a face angle belonging to the target angle interval in the face angle corresponding to each face feature vector in the face feature vector set in the tracking parameter of the tracking target, it can continue to determine whether the epsilon corresponding to the face angle of the tracking target is less than the epsilon corresponding to the face angle belonging to the target angle interval, so that the face feature vector set can be refreshed according to the determination result in the subsequent process. In a specific implementation, at least two implementation manners are included: one manner is shown in steps S607 to S610, and the other manner is shown in steps S611 to S613.

[0201] S607: If the epsilon corresponding to the face angle of the tracking target is less than the epsilon corresponding to the face angle belonging to the target angle interval, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval.

[0202] S608: If the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target.

[0203] In the above step, if the electronic device determines that the epsilon corresponding to the face angle of the tracking target is less than the epsilon corresponding to the face angle belonging to the target angle interval, the electronic device further needs to determine whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval. When the electronic device processes the previous frame of video image, the mapping relationship between the face angle in the target angle interval and the sharpness value corresponding thereto is established and stored, so that the corresponding sharpness value can be found through the face angle in the target angle interval.

[0204] If the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target, and refreshing of the face feature vector set is completed.

[0205] For example, the sharpness value corresponding to the face angle of the tracking target is 8, the sharpness value corresponding to the face angle belonging to the target angle interval is 6, the face feature vector set before refreshing is {a, b, c, d, e}, the face feature vector corresponding to the face angle belonging to the target angle interval is d, and the face feature vector corresponding to the face angle of the tracking target is f, and then the face feature vector set after refreshing is {a, b, f, d, e}.

[0206] S609: If the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded.

[0207] In this step, if the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded, that is, the face feature vector set before refreshing is not modified, and is used as the face feature vector set after refreshing.

[0208] S610: If the ε corresponding to the face angle of the tracking target is not less than the ε corresponding to the face angle belonging to the target angle interval, it is determined whether the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within a preset numerical range.

[0209] S611: If the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within the preset numerical range, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval.

[0210] S612: If the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target.

[0211] In the above step, the electronic device judges that the ε corresponding to the face angle of the tracking target is not less than the ε corresponding to the face angle belonging to the target angle interval, and needs to further judge whether the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within a preset numerical range. If the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within the preset numerical range, it is further judged whether the clarity value corresponding to the face angle of the tracking target is greater than the clarity value corresponding to the face angle belonging to the target angle interval. When the clarity value corresponding to the face angle of the tracking target is greater than the clarity value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target, and the refreshing of the face feature vector set is completed.

[0212] It should be noted that the preset numerical range is set in the electronic device by the staff before the face tracking scheme is performed, and is used for size comparison of the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval. The preset numerical range is not specifically limited in the embodiment of the application, and can be set according to actual conditions.

[0213] For example, the ε corresponding to the face angle of the tracking target is 20, the ε corresponding to the face angle belonging to the target angle interval is 16, the preset numerical range is [-5, 5], the clarity value corresponding to the face angle of the tracking target is 8, the clarity value corresponding to the face angle belonging to the target angle interval is 6, the face feature vector set before refreshing is {a, b, c, d, e}, the face feature vector corresponding to the face angle belonging to the target angle interval is d, and the face feature vector corresponding to the face angle of the tracking target is f. The difference between 20 and 16 is 4, which is within [-5, 5], and because 8 is greater than 6, the face feature vector set after refreshing is {a, b, f, d, e}.

[0214] S613: If the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or if the clarity value corresponding to the face angle of the tracking target is less than or equal to the clarity value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded.

[0215] In this step, the electronic device determines that the difference between the and corresponding to the face angle of the tracking target and the corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, and then discards the face feature vector corresponding to the face angle of the tracking target, that is, does not modify the face feature vector set before refreshing as the face feature vector set after refreshing.

[0216] Figure 6b The face angle schematic diagram provided by the present application is shown in FIG. 1. Figure 6b As shown, when continuously tracking the face by using the present application, the face angle corresponding to each face feature vector in the face feature vector set in the tracking parameter of the tracking target will continuously approach 0, 30, 60, 90, 120, 150, and 180, and the corresponding sharpness value will also become higher and higher.

[0217] The face tracking method based on video images provided by the present embodiment determines whether to replace the face feature vector corresponding to the face angle belonging to the target angle interval with the face feature vector corresponding to the face angle of the tracking target or directly uses the face feature vector set before refreshing as the face feature vector set after refreshing by using the face angle size, the difference size of, and the sharpness value. The present application obtains the face feature vector set, and then can use the set for data processing, etc. Compared with the prior art which only obtains a face feature vector for data processing, the accuracy of the data processing result is effectively improved.

[0218] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the method embodiment of the present application.

[0219] Figure 7 The structure schematic diagram of the face tracking device based on video images provided by the present application is shown in FIG. 2. Figure 7 As shown, the face tracking device 70 based on video images includes:

[0220] The obtaining module 71 is configured to obtain the effective detection frame in the video image according to the confidence of each detection frame and the preset confidence threshold for at least one detection frame in each frame of video image.

[0221] The processing module 72 is configured to calculate a spatial matching cost function matrix according to each tracking target in the tracking queue and the effective detection frame, the spatial matching cost function matrix being used to represent the cost value between each tracking target and each effective detection frame, and the tracking queue including all tracking targets obtained by processing the last frame of the video image.

[0222] The processing module 72 is further configured to perform matching by using a Hungarian maximum matching algorithm according to the spatial matching cost function matrix, to obtain a target matching result.

[0223] The processing module 72 is further configured to update tracking parameters of each tracking target in the tracking queue according to the target matching result, wherein the tracking parameters of each tracking target include coordinates of a detection frame corresponding to the tracking target, a continuous missing tracking number of the tracking target, and a face feature vector set of the tracking target.

[0224] The processing module 72 is further configured to, for each tracking target, if the continuous missing tracking number of the tracking target is greater than a preset number threshold, delete the tracking target from the tracking queue, to obtain an intermediate tracking queue.

[0225] The processing module 72 is further configured to add targets corresponding to remaining effective detection frames in the effective detection frames except for the target matching result to the intermediate tracking queue as new tracking targets, to obtain a new tracking queue after processing of the video image, and set initial tracking parameters of the new tracking targets.

[0226] Further, the processing module 72 is specifically configured to:

[0227] perform matching by using a Hungarian maximum matching algorithm according to the spatial matching cost function matrix, to obtain a matching result.

[0228] If the matching result has only one group, the matching result is determined as the target matching result.

[0229] If the matching result includes at least two groups, a face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, the face feature matching cost function matrix being used to represent a cost value between each tracking target and each effective detection frame.

[0230] The face feature matching cost function matrix is used to perform matching again by using the Hungarian maximum matching algorithm, to obtain a new matching result.

[0231] If the new matching result has only one group, the new matching result is taken as the target matching result.

[0232] If the new matching result includes at least two groups, one group is randomly selected as the target matching result.

[0233] Further, the processing module 72 is specifically further configured to:

[0234] For each tracking target in the tracking queue, a perimeter of the detection box in the tracking parameter of the tracking target is calculated according to coordinates of the detection box;

[0235] According to the perimeter, a radius of a circle with the same perimeter as the perimeter is calculated;

[0236] According to the consecutive missing tracking times in the tracking parameter of the tracking target and the radius of the circle, a position radius is calculated;

[0237] For each valid detection box, it is judged whether the center point of the valid detection box is within a circle with the center of the detection box in the tracking parameter of the tracking target as the center and the position radius as the radius;

[0238] If the center point of the valid detection box is not within the circle, a value of the valid detection box and the tracking target is set to positive infinity;

[0239] If the center point of the valid detection box is within the circle, it is judged whether the perimeter of the valid detection box and the perimeter of the detection box in the tracking parameter of the tracking target satisfy a preset condition, wherein the preset condition is Lm represents the perimeter of the valid detection box, Ln represents the perimeter of the detection box in the tracking parameter of the tracking target, Tn represents the consecutive missing tracking times in the tracking parameter of the tracking target, and β represents a perimeter variation adjustment parameter;

[0240] If the preset condition is not satisfied, the value of the valid detection box and the tracking target is set to positive infinity;

[0241] If the preset condition is satisfied, the value of the valid detection box and the tracking target is set to 0;

[0242] According to the value between each valid detection box and each tracking target, the spatial matching cost function matrix is generated.

[0243] Further, the processing module 72 is specifically further used for:

[0244] The positions with values of 0 in the spatial matching cost function matrix except the positions corresponding to the matching results are set to positive infinity to obtain a result matrix;

[0245] For the valid detection box and the tracking target corresponding to any position with a value of 0 in the result matrix, a feature vector distance between a face feature vector in the valid detection box and each face feature vector in a face feature vector set in the tracking parameter of the tracking target is calculated;

[0246] selecting a minimum feature vector distance from the feature vector distances as a target feature vector distance, and judging whether the target feature vector distance is less than a preset face similarity threshold value;

[0247] if the target feature vector distance is less than the face similarity threshold value, setting a generation value between the effective detection frame and the tracking target as positive infinity;

[0248] if the target feature vector distance is greater than or equal to the face similarity threshold value, setting the generation value between the effective detection frame and the tracking target as the target feature vector distance;

[0249] setting each 0 in the result matrix as a generation value between a corresponding effective detection frame and a tracking target, to obtain the face feature matching generation cost function matrix.

[0250] Further, the processing module 72 is specifically further configured to:

[0251] for each tracking target in the tracking queue, if the target matching result indicates that matching of the tracking target is unsuccessful, keeping the detection frame coordinates and the face feature set in the tracking parameter of the tracking target unchanged, and adding 1 to the continuous missing tracking number;

[0252] if the target matching result indicates that matching of the tracking target is successful, setting the continuous missing tracking number in the tracking parameter of the tracking target as 0, and reacquiring the detection frame coordinates in the tracking parameter of the tracking target, and refreshing the face feature set in the tracking parameter of the tracking target.

[0253] Further, the processing module 72 is specifically further configured to:

[0254] obtaining a face feature vector, a face angle and a definition value corresponding to the tracking target according to a face feature vector extraction model and a definition detection model;

[0255] obtaining a minimum value ε of an absolute value of a difference between the face angle of the tracking target and 0, 30, 60, 90, 120, 150 and 180, and establishing a mapping relationship with the face feature vector of the tracking target, the face angle of the tracking target and the definition value of the tracking target and storing the mapping relationship;

[0256] obtaining a target angle interval to which the face angle of the tracking target belongs, the target angle interval being any interval of [0, 30), [30, 60), [60, 90), [90, 120), [120, 150), [150, 180];

[0257] obtaining a face angle corresponding to each face feature vector in a face feature vector set in the tracking parameter of the tracking target, and determining whether there is a face angle belonging to the target angle interval;

[0258] If there is no face angle belonging to the target angle interval, a face feature vector corresponding to the face angle of the tracking target is added to the face feature vector set in the tracking parameter of the tracking target.

[0259] Further, the processing module 72 is specifically further configured to:

[0260] If there is a face angle belonging to the target angle interval, it is determined whether the ε corresponding to the face angle of the tracking target is less than the ε corresponding to the face angle belonging to the target angle interval;

[0261] If the ε corresponding to the face angle of the tracking target is less than the ε corresponding to the face angle belonging to the target angle interval, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval;

[0262] If the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target in the face feature vector set in the tracking parameter of the tracking target;

[0263] If the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded.

[0264] Further, the processing module 72 is specifically further configured to:

[0265] If the ε corresponding to the face angle of the tracking target is not less than the ε corresponding to the face angle belonging to the target angle interval, it is determined whether the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within a preset numerical range;

[0266] If the difference between the ε corresponding to the face angle of the tracking target and the ε corresponding to the face angle belonging to the target angle interval is within the preset numerical range, it is determined whether the sharpness value corresponding to the face angle of the tracking target is greater than the sharpness value corresponding to the face angle belonging to the target angle interval;

[0267] if the face angle corresponding to the tracking target corresponds to a sharpness value greater than the face angle corresponding to the target angle interval, then in the face feature vector set in the tracking parameter of the tracking target, the face feature vector corresponding to the face angle belonging to the target angle interval is replaced by the face feature vector corresponding to the face angle of the tracking target;

[0268] if the difference between the epsilon corresponding to the face angle of the tracking target and the epsilon corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or if the sharpness value corresponding to the face angle of the tracking target is less than or equal to the sharpness value corresponding to the face angle belonging to the target angle interval, then the face feature vector corresponding to the face angle of the tracking target is discarded.

[0269] The face tracking device based on a video image provided by the embodiment is used to execute the technical solutions in any of the preceding method embodiments, and has similar implementation principles and technical effects, which will not be repeated here.

[0270] Figure 8 A structural schematic diagram of an electronic device is provided in the present application. Figure 8 As shown in the figure, the electronic device 80 includes:

[0271] a processor 81, a memory 82, and a communication interface 83;

[0272] The memory 82 is used to store executable instructions of the processor 81.

[0273] The processor 81 is configured to execute the technical solutions in any of the preceding method embodiments by executing the executable instructions.

[0274] Optionally, the memory 82 can be independent or integrated with the processor 81.

[0275] Optionally, when the memory 82 is independent of the processor 81, the electronic device 80 can further include:

[0276] a bus for connecting the above-mentioned devices.

[0277] The server is used to execute the technical solutions in any of the preceding method embodiments, and has similar implementation principles and technical effects, which will not be repeated here.

[0278] The embodiment of the present application further provides a readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the technical solutions provided by any of the preceding embodiments.

[0279] The embodiment of the present application further provides a computer program product comprising a computer program, which is used for implementing the technical solutions provided by any of the preceding method embodiments when executed by a processor.

[0280] Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by a program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program, when executed, executes the steps of the foregoing method embodiments; and the foregoing storage medium includes various media capable of storing program codes, such as ROM, RAM, magnetic disk, or optical disk.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A face tracking method based on video images, characterized in that, The method comprises: For each detection frame in each frame of video image, according to the confidence of each detection frame and the preset confidence threshold, the effective detection frame in the video image is obtained; According to each tracking target in the tracking queue and the effective detection frame, a spatial matching cost function matrix is calculated, the spatial matching cost function matrix is used to represent the cost value between each tracking target and each effective detection frame, and the tracking queue includes all tracking targets obtained by processing the last frame of the video image; According to the spatial matching cost function matrix, the Hungarian maximum matching algorithm is used for matching to obtain a matching result; If the matching result is only one group, the matching result is determined as a target matching result; If the matching result includes at least two groups, a face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, the face feature matching cost function matrix is used to represent the cost value between each tracking target and each effective detection frame; the Hungarian maximum matching algorithm is used again for matching according to the face feature matching cost function matrix to obtain a new matching result; if the new matching result is only one group, the new matching result is taken as the target matching result; if the new matching result includes at least two groups, a group is randomly selected as the target matching result; According to the target matching result, the tracking parameters of each tracking target in the tracking queue are refreshed, wherein the tracking parameters of each tracking target include: the coordinates of the detection frame corresponding to the tracking target, the continuous missing tracking number of the tracking target, and the face feature vector set of the tracking target; For each tracking target, if the continuous missing tracking number of the tracking target is greater than a preset number threshold, the tracking target is deleted from the tracking queue to obtain an intermediate tracking queue; The targets corresponding to the remaining effective detection frames obtained by removing the effective detection frame corresponding to the target matching result from the effective detection frame are added to the intermediate tracking queue as new tracking targets to obtain a new tracking queue after processing the video image, and the initial tracking parameters of the new tracking targets are set.

2. The method of claim 1, wherein, According to each tracking target in the tracking queue and the effective detection frame, the spatial matching cost function matrix is calculated, which comprises: For each tracking target in the tracking queue, the perimeter of the detection frame in the tracking parameters of the tracking target is calculated and obtained according to the coordinates of the detection frame; According to the perimeter, the radius of a circle with the same perimeter as the perimeter is calculated and obtained; According to the continuous missing tracking number in the tracking parameters of the tracking target and the radius of the circle, a position radius is calculated and obtained; For each effective detection frame, it is judged whether the center point of the effective detection frame is in a circle with the center of the detection frame in the tracking parameters of the tracking target as the center and the position radius as the radius; If the center point of the effective detection frame is not in the circle, the cost value between the effective detection frame and the tracking target is set to positive infinity; If the center point of the effective detection frame is in the circle, it is determined whether the perimeter of the effective detection frame and the perimeter of the detection frame in the tracking parameter of the tracking target satisfy a preset condition, wherein the preset condition is , denotes the perimeter of the effective detection frame, denotes the perimeter of the detection frame in the tracking parameter of the tracking target, denotes the number of consecutive missing tracking times in the tracking parameter of the tracking target, denotes the perimeter variation adjustment parameter; If the preset condition is not met, the cost value between the effective detection frame and the tracking target is set to positive infinity; If the preset condition is met, a cost value between the effective detection frame and the tracking target is set to 0; According to the cost value between each effective detection frame and each tracking target, the spatial matching cost function matrix is generated.

3. The method of claim 1, wherein, The face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, including: In the spatial matching cost function matrix, positions except a position corresponding to the matching result are set to positive infinity to obtain a result matrix; For an effective detection frame and a tracking target corresponding to any position with a value of 0 in the result matrix, a face feature vector of a target in the effective detection frame is calculated, and a feature vector distance between the face feature vector and each face feature vector in a face feature vector set in the tracking parameter of the tracking target is calculated; The smallest feature vector distance is selected as a target feature vector distance from the feature vector distances, and whether the target feature vector distance is smaller than a preset face similarity threshold is determined; If the target feature vector distance is smaller than the face similarity threshold, the cost value between the effective detection frame and the tracking target is set to positive infinity; If the target feature vector distance is greater than or equal to the face similarity threshold, the cost value between the effective detection frame and the tracking target is set to the target feature vector distance; Each 0 in the result matrix is set to a cost value between corresponding effective detection frames and tracking targets to obtain the face feature matching cost function matrix.

4. The method according to any one of claims 1 to 3, characterized in that, According to the target matching result, the tracking parameter of each tracking target in the tracking queue is refreshed, including: For each tracking target in the tracking queue, if the target matching result indicates that matching of the tracking target is unsuccessful, the detection frame coordinate and the face feature set in the tracking parameter of the tracking target are kept unchanged, and a continuous missing tracking number is increased by 1; If the target matching result indicates that matching of the tracking target is successful, the continuous missing tracking number in the tracking parameter of the tracking target is set to 0, and the detection frame coordinate in the tracking parameter of the tracking target is reacquired, and the face feature set in the tracking parameter of the tracking target is refreshed.

5. The method of claim 4, wherein, The face feature set in the tracking parameter of the tracking target is refreshed, including: According to a face feature vector extraction model and a definition detection model, a face feature vector, a face angle and a definition value corresponding to the tracking target are acquired; Obtaining the minimum value of the absolute value of the difference between the face angle of the tracking target and 0, 30, 60, 90, 120, 150, and 180 And a mapping relationship is established with the face feature vector of the tracking target, the face angle of the tracking target, and the clarity value of the tracking target and is stored. A target angle interval to which the face angle of the tracking target belongs is acquired, and the target angle interval is any interval in [0, 30), [30, 60), [60, 90), [90, 120), [120, 150), [150, 180]; A face angle corresponding to each face feature vector in the face feature vector set in the tracking parameter of the tracking target is acquired, and whether there is a face angle belonging to the target angle interval is determined; If there is no face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is added to the face feature vector set in the tracking parameter of the tracking target.

6. The method of claim 5, wherein, The method further comprises: If there is a face angle belonging to the target angle interval, it is judged whether the face angle corresponding to the tracking target is less than the face angle corresponding to the target angle interval ; if not, the face angle corresponding to the tracking target is greater than the face angle corresponding to the target angle interval . If the face angle corresponding to the tracking target corresponds to a target angle interval If the face angle corresponding to the tracking target corresponds to a target angle interval If the face angle corresponding to the tracking target corresponds to a target angle interval if the facial angle of the tracking target corresponds to a sharpness value greater than the facial angle belonging to the target angle interval, replacing the facial feature vector corresponding to the facial angle belonging to the target angle interval with the facial feature vector corresponding to the facial angle of the tracking target in the facial feature vector set of the tracking target in the tracking parameter of the tracking target; if the facial angle of the tracking target corresponds to a sharpness value less than or equal to the facial angle belonging to the target angle interval, discarding the facial feature vector corresponding to the facial angle of the tracking target.

7. The method of claim 6, wherein, The method further comprises: If the angle of the face of the tracked target corresponds to Not less than the face angle corresponding to the target angle range Then determine the angle of the face of the tracked target. The face angle corresponding to the target angle range Is the difference within the preset value range? If the face angle of the tracking target corresponds to If the face angle of the tracking target corresponds to If the difference between the face angle of the tracking target and the face angle of the target angle interval is within the preset numerical range, it is determined whether the definition value corresponding to the face angle of the tracking target is greater than the definition value corresponding to the face angle of the target angle interval. if the facial angle of the tracking target corresponds to a sharpness value greater than the facial angle belonging to the target angle interval, replacing the facial feature vector corresponding to the facial angle belonging to the target angle interval with the facial feature vector corresponding to the facial angle of the tracking target in the facial feature vector set of the tracking target in the tracking parameter of the tracking target; If the face angle corresponding to the tracking target is not within the target angle interval, or if the face angle corresponding to the tracking target corresponds to a face feature vector whose difference from the face feature vector corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or if the face angle corresponding to the tracking target corresponds to a face feature vector whose definition value is less than or equal to the definition value of the face feature vector corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded. If the face angle corresponding to the tracking target is not within the target angle interval, or if the face angle corresponding to the tracking target corresponds to a face feature vector whose difference from the face feature vector corresponding to the face angle belonging to the target angle interval is not within the preset numerical range, or if the face angle corresponding to the tracking target corresponds to a face feature vector whose definition value is less than or equal to the definition value of the face feature vector corresponding to the face angle belonging to the target angle interval, the face feature vector corresponding to the face angle of the tracking target is discarded. If the face angle corresponding to the tracking target is 8. A face tracking apparatus based on video images, characterized by comprise: The acquisition module is configured to, for each detection frame in each frame of video image, acquire an effective detection frame in the video image according to a confidence of each detection frame and a preset confidence threshold. The processing module is configured to calculate a spatial matching cost function matrix according to each tracking target in a tracking queue and the effective detection frame, the spatial matching cost function matrix being used to represent a cost value between each tracking target and each effective detection frame, the tracking queue including all tracking targets obtained by processing a previous frame of image of the video image. The processing module is further configured to perform matching by using a Hungarian maximum matching algorithm according to the spatial matching cost function matrix to obtain a matching result. If the matching result only has one group, the matching result is determined as a target matching result. If the matching result includes at least two groups, a face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, the face feature matching cost function matrix being used to represent a cost value between each tracking target and each effective detection frame. The face feature matching cost function matrix is calculated according to the spatial matching cost function matrix, and the Hungarian maximum matching algorithm is used to perform matching again to obtain a new matching result. If the new matching result only has one group, the new matching result is taken as the target matching result. If the new matching result includes at least two groups, one group is randomly selected as the target matching result. The processing module is further configured to refresh tracking parameters of each tracking target in the tracking queue according to the target matching result, wherein the tracking parameters of each tracking target include coordinates of a detection frame corresponding to the tracking target, a continuous missing tracking number of the tracking target, and a facial feature vector set of the tracking target. The processing module is further configured to, for each tracking target, if the continuous missing tracking number of the tracking target is greater than a preset number threshold, delete the tracking target from the tracking queue to obtain an intermediate tracking queue. The processing module is further configured to add a target corresponding to a remaining effective detection frame, which is obtained by removing the effective detection frame corresponding to the target matching result, as a new tracking target into the intermediate tracking queue, to obtain a new tracking queue after processing the video image, and set an initial tracking parameter of the new tracking target.

9. An electronic device, comprising: The method comprises: a processor, a memory, and a communication interface; the memory is configured to store executable instructions of the processor; the processor is configured to execute the executable instructions to perform the face tracking method based on the video image according to any one of claims 1 to 7.

10. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the face tracking method based on the video image according to any one of claims 1 to 7.

11. A computer program product, characterised in that, The computer program is executed by the processor to implement the face tracking method based on the video image according to any one of claims 1 to 7.

Images