Face tracking method and device, electronic equipment and storage medium

Abstract

The face tracking method and device, the electronic device, and the storage medium provided in the application comprise the following steps: obtaining face images and position information in a current frame image and a previous frame image of a video to be tracked; determining distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image; obtaining image features of first face images in the current frame image and the previous frame image for which the distances do not exceed a first preset value, determining similarities between each first face image in the current frame image and each first face image in the previous frame image, and determining that the first face images in the current frame image and the previous frame image for which the similarities satisfy a second preset value are the same face. Through the distance determination, the number of times and the time of feature extraction are reduced, and the demand for the computing power of the running device is also reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a face tracking method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of image technology, face tracking technology is increasingly widely used in life. For example, face tracking technology can be used in security supervision, intelligent beauty, and other fields.

[0003] Currently, when tracking a face, in order to determine whether a face in a current frame image in video information is the same face as a face appearing in a previous frame image, image features of the face and all faces in the previous frame image are usually extracted, and the two faces with the most similar features are determined as the same face.

[0004] However, in the above method, the face feature extraction algorithm consumes a large amount of calculation, and if there are many faces in the previous frame image, the running device will consume a large amount of computing resources and take a long time. SUMMARY

[0005] The present application provides a face tracking method, device, electronic device, and storage medium to solve the problem that the face feature extraction algorithm in the prior art consumes a large amount of calculation, and if there are many faces to be matched, the running device will consume a large amount of computing resources and take a long time.

[0006] In a first aspect, the present application provides a face tracking method, which comprises:

[0007] obtaining face images and position information in a current frame image and a previous frame image of a video to be tracked;

[0008] determining distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image;

[0009] obtaining image features of a first face image in the current frame image and the previous frame image, which has a distance not exceeding a first preset value;

[0010] determining similarities between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, and determining that the first face images in the current frame image and the previous frame image are the same face when the similarities satisfy a second preset value.

[0011] In some embodiments, before the determining the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, the method further comprises:

[0012] determining the intersection over union of each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, and determining the first face images in the current frame image and the previous frame image as the same face image when the intersection over union meets a preset threshold value;

[0013] The determining the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image comprises:

[0014] For the second face images in the current frame image and the previous frame image which do not meet the threshold value, determining the distance between each second face image in the current frame image and each second face image in the previous frame image according to the position information of the second face images.

[0015] In some embodiments, after the determining the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, the method further comprises:

[0016] determining the face images in the current frame image and the previous frame image as different face images when the distance exceeds the first preset value.

[0017] In some embodiments, after the determining the similarity between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, the method further comprises:

[0018] storing the face images in the current frame image and the previous frame image as new face images into the tracking list when the similarity does not reach the second preset value.

[0019] In some embodiments, the obtaining the face images and the position information in the previous frame image comprises:

[0020] obtaining the face images in the previous frame image of the current frame image and the position information of the face images in the previous frame image from the tracking list, the tracking list storing the position information of the face images in the frame images before the current frame image;

[0021] The tracking list includes face data of multiple faces, and the face data of each face includes an identifier of the face, information of each frame image, a face image of the face in the each frame image, and position information of the face image in the each frame image.

[0022] In some embodiments, the face data of each face in the tracking list further includes image features of the face in at least one frame image of the each frame image; and the obtaining of the image features of the first face image includes:

[0023] The image features of the first face image in the current frame image are extracted;

[0024] For the first face image in the previous frame image, the following processing is performed: according to the identifier of the first face corresponding to the first face image in the previous frame image, the face data in which the identifier of the first face is located in the current tracking list is searched; if the image features corresponding to the previous frame image exist in the face data in which the identifier of the first face is located, the image features are obtained as the image features of the first face image; and if the image features corresponding to the previous frame image do not exist in the face data in which the identifier of the first face is located, the image features of the first face image are extracted based on the face image of the previous frame image in the face data.

[0025] In some embodiments, the image features of the face image in the tracking list are extracted from the frame images every predetermined number of frames.

[0026] In a second aspect, the present application provides a face tracking device, and the device includes:

[0027] A first obtaining unit is configured to obtain face images and position information of the face images in a current frame image and a previous frame image of a video to be tracked.

[0028] A first determining unit is configured to determine distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image.

[0029] A second obtaining unit is configured to obtain image features of a first face image in the current frame image and the previous frame image, which have a distance not more than a first preset value.

[0030] The second determining unit is configured to determine the similarity between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, and determine the first face images in the current frame image and the previous frame image as the same face when the similarity satisfies a second preset value.

[0031] In some embodiments, the apparatus further includes:

[0032] The third determining unit is configured to, before the first determining unit determines the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, determine the intersection-over-union of each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, and determine the first face images in the current frame image and the previous frame image as the same face when the intersection-over-union satisfies a preset threshold.

[0033] The first determining is specifically configured to:

[0034] For the second face images in the current frame image and the previous frame image that do not satisfy the threshold, the distance between each second face image in the current frame image and each second face image in the previous frame image is determined according to the position information of the second face images.

[0035] In some embodiments, the apparatus further includes:

[0036] The fourth determining unit is configured to, after the first determining unit determines the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, determine the face images in the current frame image and the previous frame image as different faces when the distance exceeds the first preset value.

[0037] In some embodiments, the apparatus further includes:

[0038] The storage unit is configured to, after the second determining unit determines the similarity between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, store the face images in the current frame image and the previous frame image that do not reach the second preset value as new faces to a tracking list.

[0039] In some embodiments, the first obtaining unit includes:

[0040] The first obtaining module is configured to obtain, from a tracking list, a face image in a previous frame image of the current frame image and an identifier of a face corresponding to the face image, and position information of the face image in the previous frame image, wherein the tracking list stores position information of face images in frame images before the current frame image.

[0041] The tracking list comprises face data of a plurality of faces, wherein the face data of each face comprises an identifier of the face, information of each frame image, a face image of the face in the each frame image, and position information of the face image of the face in the each frame image.

[0042] In some embodiments, the face data of each face in the tracking list further comprises image features of the face in at least one frame image of the each frame image, and the second obtaining unit comprises:

[0043] The first extracting module is configured to extract image features of a first face image in the current frame image.

[0044] The second extracting module is configured to perform the following processing on the first face image in the previous frame image: searching, according to an identifier of a first face corresponding to the first face image in the previous frame image, face data in which the identifier of the first face is located in the current tracking list; if the image features corresponding to the previous frame image exist in the face data in which the identifier of the first face is located, obtaining the image features as the image features of the first face image; if the image features corresponding to the previous frame image do not exist in the face data in which the identifier of the first face is located, extracting the image features of the first face image based on a face image of the previous frame image in the face data.

[0045] In some embodiments, the image features of the face image in the tracking list are extracted from frame images every predetermined number of frames.

[0046] In a third aspect, the present application provides an electronic device, comprising: a memory, a processor.

[0047] The memory is configured to store instructions executable by the processor.

[0048] The processor is configured to execute the method according to any one of the first aspect.

[0049] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method according to any one of the first aspect.

[0050] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of the first aspect.

[0051] The face tracking method, device, electronic device and storage medium provided by the present application comprise the following steps: obtaining face images and position information in a current frame image and a previous frame image of a video to be tracked; determining distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image; obtaining image features of first face images in the current frame image and the previous frame image, determining similarities between each first face image in the current frame image and each first face image in the previous frame image, and determining that the first face images in the current frame image and the previous frame image are the same face when the similarities satisfy a second preset value, for the first face images in the current frame image and the previous frame image whose distances do not exceed a first preset value. Through the distance determination, the number of times and the time of feature extraction are reduced, and the demand for the computing power of the running device is also reduced. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0053] Figure 1 A flowchart of a face tracking method provided by an embodiment of the present application;

[0054] Figure 2 A flowchart of another face tracking method provided by an embodiment of the present application;

[0055] Figure 3 A flowchart of a method for obtaining image features provided by an embodiment of the present application;

[0056] Figure 4 A structural diagram of a face tracking device provided by an embodiment of the present application;

[0057] Figure 5 A structural diagram of another face tracking device provided by an embodiment of the present application;

[0058] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present application.

[0059] The specific embodiments of the present application have been shown in the above-described drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application concept in any way, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0060] The exemplary embodiments will be described in detail below with reference to the drawings. In the following description, unless otherwise indicated, like numbers in the different drawings represent the same or similar elements. The following exemplary embodiments are not representative of all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0061] With the development of image technology, face tracking technology is increasingly widely used in life. For example, face tracking technology can be used in security supervision, intelligent beauty, and other fields.

[0062] Currently, when tracking a face, in order to determine whether a certain face in a current frame image in video information is a face that has appeared before the current frame image, image features of the face in the current frame image and all faces in previous frame images are usually extracted, and through feature matching, the two faces corresponding to the face features with higher matching degree are regarded as the same face, otherwise, they are new faces. Specifically, when extracting features, deep learning methods such as convolutional neural networks can be used to extract features of face images.

[0063] When the above method is executed, in the process of extracting features of the face, the deep learning method consumes a large amount of calculation and a large amount of time, and as the number of faces to be matched increases, the time-consuming of the above feature extraction process also increases, and the requirement for device computing power also increases.

[0064] The face tracking method, device, electronic equipment and storage medium provided by the present application aim to solve the above technical problems of the prior art.

[0065] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0066] The device provided in the present embodiment is used to implement the technical solutions provided by the above method, and has similar implementation principles and technical effects, which will not be described again.

[0067] Figure 1 A flowchart of a face tracking method provided by an embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 1 As shown in FIG. 1, the method includes the following steps:

[0068] S101, obtaining face images and position information in a current frame image and a previous frame image of a video to be tracked.

[0069] Exemplarily, when analyzing the video to be tracked, each frame in the video can be analyzed, or some frame images in the video to be tracked can be specified for analysis.

[0070] In one example, when obtaining the face images in the current frame image and the previous frame image of the video to be tracked, each face detection frame in which each face in the frame image is located can be identified according to a commonly used face detection method.

[0071] In one example, when determining the position information of the face images in the current frame image and the previous frame image of the video to be tracked, the position information of the face detection frame can be taken as the position information of the face image. For example, the position information can include the coordinates of the four vertices of the face detection frame in the frame image and the coordinates of the center point of the face detection frame.

[0072] S102, according to the position information of the face images in the current frame image and the previous frame image, determine the distance between each face image in the current frame image and each face image in the previous frame image.

[0073] Exemplarily, after the position information of the face images in the current frame image and the previous frame image is determined, the distance between each face in the current frame image and each face in the previous frame image can be calculated according to the position information of the faces in the current frame image and the previous frame image.

[0074] In one example, the distance between each face in the current frame image and each face in the previous frame image can be determined by the distance between the center points of the face detection frames corresponding to the two faces respectively.

[0075] S103, for the first face images of the current frame image and the previous frame image whose distance does not exceed a first preset value, obtaining the image features of the first face images.

[0076] Exemplarily, after the distance between each face in the current frame image and each face in the previous frame image is calculated, it is considered that when the position information of the face in the current frame image and the face in the previous frame image is far away, the two faces are not the same face. Therefore, for the first face images of the current frame image and the previous frame image whose distance is less than or equal to the first preset value, it is considered that the first face in the current frame image may be the same face as the first face in the previous frame image. Therefore, in order to further determine whether the above first face is the same face, the feature extraction of the above first face is continued. The first preset value can be specifically set by relevant personnel.

[0077] S104, determine the similarity between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, and determine the first face images in the current frame image and the previous frame image as the same face if the similarity satisfies a second preset value.

[0078] Exemplarily, the similarity between the image features of the first face in the current frame image and the image features of the first face in the previous frame image is calculated, that is, for any first face in the current frame image, the image feature similarity between the first face and the face in the previous frame image with a distance less than a first preset value is calculated, and if the similarity satisfies a second preset value (i.e., the similarity is greater than the second preset value), the first face in the current frame image and the first face in the previous frame image are considered as the same face.

[0079] In this embodiment, when performing face tracking, the distance between the face in the current frame image and the face in the previous frame image is obtained through the face position information in the current frame image and the face position information in the previous frame image, and for each face in the current frame image, the first face in the previous frame image with a distance less than or equal to a first preset value from the face is determined (that is, the face in the previous frame image with a distance greater than the first preset value from the face is considered not to be the same face as the face, and the face with a distance not greater than the first preset value is considered to be the same face), and then whether it is the same face is further determined through the feature similarity method. That is, in the above face tracking method, the image of the face that needs to be extracted through features in the subsequent process can be reduced through the judgment of the distance, thereby reducing the number of feature extraction, improving the processing speed of face tracking, and reducing the demand for device computing power.

[0080] Figure 2 Another flowchart of a face tracking method provided by the embodiment of the present application is shown in FIG. 4, which includes the following steps: Figure 2

[0081] S201, obtain the face images and position information in the current frame image and the previous frame image of a video to be tracked.

[0082] Exemplarily, the principle of this step is similar to that of step S101 in the embodiment of the present application, which will not be described herein again. Figure 1

[0083] S202, determine the intersection-over-union of each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image.

[0084] S203, determine the first face images in the current frame image and the previous frame image as the same face if the intersection-over-union satisfies a preset threshold.​​

[0085] Exemplarily, before the distance determination, the intersection-over-union of each face image in the current frame image and each face image in the previous frame image can be determined first. The intersection-over-union of two face images is the area of the overlapping part of the two face images divided by the total area of the two face images after merging. In an example, the intersection-over-union of each face image in the current frame image and each face image in the previous frame image can be determined by the ratio between the area of the overlapping part of the face bounding box in the current frame image and the face bounding box in the previous frame image and the total area of the two face bounding boxes.

[0086] If the intersection-over-union between a face image in the current frame and a face image in the previous frame satisfies a preset threshold (for example, when the intersection-over-union is greater than or equal to the preset threshold), the two faces are considered to be the same face.

[0087] S204, for the second face images in the current frame image and the previous frame image whose intersection-over-union does not satisfy the threshold, the distance between each second face image in the current frame image and each second face image in the previous frame image is determined according to the position information of the second face images.

[0088] Exemplarily, after step S203, after removing the face images that have been confirmed in the current frame image, the distance between each second face image in the current frame and each second face image in the previous frame is further determined among the remaining face images in the current frame (i.e., the second faces in the current frame image) and the remaining face images in the previous frame (i.e., the second faces in the previous frame image).

[0089] S205, for the first face images in the current frame image and the previous frame image whose distance does not exceed a first preset value, the image features of the first face images are obtained.

[0090] Exemplarily, among the second face images in the current frame and the second face images in the previous frame, the current frame image and the previous frame image whose distance does not exceed the first preset value are determined as the first face images in the current frame image and the previous frame image, and the image features of the first face images are further extracted to determine whether the first face images are the same face through the image feature similarity.

[0091] In an example, the face images in the current frame image and the previous frame image whose distance exceeds the first preset value can be determined as different faces.

[0092] S206, the similarity between each first face image in the current frame image and each first face image in the previous frame image is determined according to the image features of the first face images in the current frame image and the previous frame image, and the first face images in the current frame image and the previous frame image whose similarity satisfies a second preset value are determined as the same face.

[0093] Exemplarily, the step is combined with Figure 1 The principle of step S104 is similar, and thus is not described herein.

[0094] In this embodiment, before the images are filtered according to the distance, the images of the same person can be first determined by the IoU of the face image in the current frame and the face image in the previous frame. Then, the face images are further filtered according to the distance. By combining the IoU judgment and the distance judgment, the number of times of image feature extraction is further reduced. Compared with directly filtering according to the distance, the IoU judgment can avoid the possibility of misjudgment caused by the distance between the center points of the face in the current frame and the face in the previous frame being far when the face image in the video is large.

[0095] In the above Figure 1 On the basis of the embodiments shown in the above Figure 2 If the size of the calculated similarity does not reach the second preset value, that is, the similarity is less than the second preset value, it is considered that the face in the current frame is a newly appearing face, and then the face can be stored in the tracking list.

[0096] In some embodiments, when tracking the face, a tracking list can be set. The tracking list includes face data corresponding to multiple faces. In the face data corresponding to each face, an identifier of the face is included, which is used to mark and distinguish the faces in the tracking list. The tracking list further includes information of each frame image (for example, the frame image information can be a frame number corresponding to a video frame in which the face image is located or time information corresponding to the frame number). The tracking list further includes a face image of the face in each frame image and position information of the face image in each frame image (that is, when the face image is detected, the position of the face image in the frame image).

[0097] For example, the face data of four faces is included in the tracking list (i.e., four different persons are included in the tracking list). In the face data of each group of faces, the face identifier of the face is included, for example, the face identifiers of the four groups of faces are ID1, ID2, ID3, and ID4, respectively. For each group of face data, the frame image information of the face and the face image of the face in each frame image are included. Taking one group of face data as an example, the identifier of the face is ID1, and the face appears in the 5th frame, the 6th frame, and the 7th frame of the video, respectively, so the group of face data includes the face image corresponding to ID1-5, the face image corresponding to ID1-6, and the face image corresponding to ID1-7. ID1-5 indicates the face image of the face with the identifier ID1 when the face appears in the 5th frame of the video, that is, ID1-5 also represents the frame image information of the face. That is, one ID can be used to represent a person. Under the ID, the face images of the face in different frames are included, and the face images can be distinguished by the respective frame image information under the group of IDs.

[0098] When the face image in the previous frame image and the position information of the face image need to be obtained (for example, when step S103 in the embodiment shown in Figure 1 or step S205 in the embodiment shown in Figure 2 is performed), all the stored face images, position information, and the face identifier corresponding to each face image corresponding to the frame image information of the previous frame of the current frame can be directly found in the tracking list from the frame image information of the previous frame of the current frame in the tracking list.

[0099] In some implementations, on the basis of the above embodiment, the face data of each face in the tracking list further includes: image features of the face in at least one frame image of the frame images. For example, in the face data of the face with the identifier ID1, the image features of the face images corresponding to ID1-5 and ID1-7 are stored, respectively. The face image corresponding to ID1-6 does not store the image features corresponding to the face image.

[0100] When step S103 in Figure 1 or step S205 in the embodiment shown in Figure 2 is performed (i.e., when the image features of the first face image are obtained), as Figure 3 shown, Figure 3A flowchart of a method for obtaining image features is provided in the embodiments of the present application. The method comprises the following steps: S301, for a first face image in a current frame image, obtaining image features of the first face image by extraction; S302, for the first face image in a previous frame image, searching for face data in which an identifier of the first face is located in a current tracking list according to the identifier of the first face corresponding to the first face image in the previous frame image; S303, if the image features corresponding to the previous frame image exist in the face data in which the identifier of the first face is located, obtaining the image features as the image features of the first face image; and S304, if the image features corresponding to the previous frame image do not exist in the face data in which the identifier of the first face is located, obtaining the image features of the first face image by extraction based on a face image corresponding to the previous frame image in the face data.

[0101] That is, for the first face image in the current frame image, the image features of the first face image can be directly obtained by using a feature extraction method. For the face image corresponding to the previous frame of the current frame, the face data in which the identifier of the face corresponding to the face image is located can be searched in the tracking list according to the identifier of the face, and if the image features corresponding to the previous frame image exist, the image features are taken as the face features corresponding to the face image of the previous frame of the current frame. If the image features of the face do not exist in the current tracking list, the image features of the face image in the previous frame image can be further obtained by using a feature extraction algorithm. For example, if the previous frame of the current frame is the 7th frame of a video, for the face with an identifier of ID1, the image features of the face image corresponding to ID1-7 are stored, and thus the image features can be directly obtained from the tracking list.

[0102] In some embodiments, when the tracking list is updated, the face data of each face in the tracking list comprises image features corresponding to at least one face image corresponding to the face. Generally, for each face, all the face images corresponding to the face can be extracted and stored, and the image features of all the face images are taken as the image features of the face. In an example, in order to avoid the problem that the number of times of feature extraction is large due to too many face images corresponding to a face, in the embodiments, for a certain face, the image features of the face can be obtained by extraction from frame images every predetermined number of frames, that is, the face images of the frame images corresponding to the face are extracted every predetermined number of frames to reduce the number of times of image extraction. When the predetermined number of frames is set, the frame rate of the video can be considered for setting. Compared with a video with a small frame rate, the face images of adjacent two frames in a video with a large frame rate change less, and thus the predetermined number of frames can be set to be larger, that is, the frame rate of the video and the predetermined number of frames are in a positive correlation.

[0103] Figure 4A structure diagram of a face tracking device is provided for an embodiment of the present application, and the device comprises:

[0104] The first acquisition unit 51 is configured to acquire face images and position information in a current frame image and a previous frame image of a video to be tracked.

[0105] The first determination unit 52 is configured to determine distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image.

[0106] The second acquisition unit 53 is configured to acquire image features of first face images in the current frame image and the previous frame image, which have distances not exceeding a first preset value.

[0107] The second determination unit 54 is configured to determine similarities between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, and determine the first face images in the current frame image and the previous frame image as the same face when the similarities satisfy a second preset value.

[0108] The device provided in the embodiment is used to implement the technical solution provided by the above method, and has similar implementation principles and technical effects, which will not be described herein.

[0109] Figure 5 A structure diagram of another face tracking device is provided for an embodiment of the present application, and in Figure 4 the structure shown, the device further comprises:

[0110] The third determination unit 55 is configured to determine intersection-over-union ratios of each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image before the first determination unit 52 determines the distances between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, and determine first face images in the current frame image and the previous frame image as the same face when the intersection-over-union ratios satisfy a preset threshold.

[0111] The first determination 52 is specifically configured to:

[0112] For second face images in the current frame image and the previous frame image, which do not satisfy the threshold, distances between each second face image in the current frame image and each second face image in the previous frame image are determined according to the position information of the second face images.

[0113] In some embodiments, the device further comprises:

[0114] The fourth determining unit 56 is configured to, after the first determining unit 52 determines the distance between each face image in the current frame image and each face image in the previous frame image according to the position information of the face images in the current frame image and the previous frame image, determine the face images in the current frame image and the previous frame image, whose distance exceeds a first preset value, as different faces.

[0115] In some embodiments, the apparatus further includes:

[0116] The storage unit 57 is configured to, after the second determining unit 54 determines the similarity between each first face image in the current frame image and each first face image in the previous frame image according to the image features of the first face images in the current frame image and the previous frame image, store the face images in the current frame image and the previous frame image, whose similarity does not reach a second preset value, as new faces into the tracking list.

[0117] In some embodiments, the first obtaining unit 51 includes:

[0118] The first obtaining module 511 is configured to obtain, from the tracking list, the face images in the previous frame image of the current frame image and the identities of the faces corresponding to the face images, and the position information of the face images in the previous frame image, the tracking list storing the position information of the face images in the frame images before the current frame image.

[0119] The tracking list includes face data of multiple faces, and the face data of each face includes an identity of the face, information of each frame image, a face image of the face in each frame image, and position information of the face image of the face in each frame image.

[0120] In some embodiments, the face data of each face in the tracking list further includes image features of the face in at least one frame image of each frame image, and the second obtaining unit 53 includes:

[0121] The first extracting module 531 is configured to, for each first face image in the current frame image, extract the image features of the first face image.

[0122] The second extracting module 532 is configured to, for each first face image in the previous frame image, perform the following processing: according to the identity of the first face corresponding to the first face image in the previous frame image, search for face data in which the identity of the first face is located in the current tracking list; if the image features corresponding to the previous frame image exist in the face data in which the identity of the first face is located, obtain the image features as the image features of the first face image; if the image features corresponding to the previous frame image do not exist in the face data in which the identity of the first face is located, extract the image features of the first face image based on the face image of the face corresponding to the previous frame image in the face data.

[0123] In some embodiments, the image features of the face images in the tracking list are obtained by extracting from the frame images every predetermined number of frames.

[0124] The device provided by the embodiment has the same implementation principle and technical effects as the technical solutions provided by the above method, and thus will not be described here.

[0125] Figure 6 A structural diagram of an electronic device provided in the embodiment is shown in FIG. 1. Figure 6 The electronic device includes:

[0126] The processor 291, the electronic device further includes a memory 292; and can further include a communication interface 293 and a bus 294. The processor 291, the memory 292, the communication interface 293 can communicate with each other through the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can call the logical instructions in the memory 294 to execute the method of the above embodiment.

[0127] In addition, the logical instructions in the memory 292 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0128] The memory 292 as a computer readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiment. The processor 291 executes the functions and data processing by running the software programs, instructions and modules stored in the memory 292, that is, implements the method in the above method embodiment.

[0129] The memory 292 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created by the terminal device in use, etc. In addition, the memory 292 can include a high-speed random access memory, and can also include a non-volatile memory.

[0130] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the method provided by the above embodiment.

[0131] The embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the computer program implements the method provided by the above embodiment.

[0132] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0133] It is to be understood that the application is not limited to the precise construction herein disclosed and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A face tracking method, characterized in that, include: Obtain the face image and location information in the current frame and the previous frame of the video to be tracked; Based on the position information of the face images in the current frame image and the previous frame image, the intersection-union ratio (IUU) of each face image in the current frame image and each face image in the previous frame image is determined. The first face image in the current frame image and the previous frame image whose IUU satisfies the preset threshold is determined to be the same face. For the second face image in the current frame image and the previous frame image where the intersection-union ratio does not meet the threshold, the distance between each second face image in the current frame image and each second face image in the previous frame image is determined based on the position information of the second face image. For the second face image in the current frame image and the previous frame image, which are within a distance of no more than a first preset value, the image features of the second face image are obtained; Based on the image features of the second face images in the current frame and the previous frame, the similarity between each second face image in the current frame and each second face image in the previous frame is determined, and the second face images in the current frame and the previous frame that have a similarity that meets a second preset value are determined to be the same face.

2. The method according to claim 1, characterized in that, After determining the distance between each face image in the current frame and each face image in the previous frame based on the position information of the face images in the current frame and the previous frame, the method further includes: Faces in the current frame and the previous frame that are more than a certain distance from the first preset value are identified as different faces.

3. The method according to claim 1, characterized in that, After determining the similarity between each first face image in the current frame and each first face image in the previous frame based on the image features of the first face images in the current frame and the previous frame, the method further includes: If the similarity does not reach the second preset value, the face images in the current frame and the previous frame are stored as new faces in the tracking list.

4. The method according to claim 1, characterized in that, The step of obtaining the face image and location information in the previous frame includes: The tracking list retrieves the face image and its corresponding face identifier from the previous frame image of the current frame image, as well as the position information of the face image in the previous frame image. The tracking list stores the position information of the face images in the frames before the current frame image. The tracking list includes facial data of multiple faces; wherein the facial data of each face includes the face's identifier, information of each frame image, the face image of the face in each frame image, and the position information of the face image of the face in each frame image.

5. The method according to claim 4, characterized in that, The facial data for each face in the tracking list also includes the image features of that face in at least one frame of each frame image; The step of obtaining image features of the second face image from the current frame image and the previous frame image, which are at a distance not exceeding a first preset value, includes: For the second face image in the current frame image, extract the image features of the second face image; For the second face image in the previous frame, the following processing is performed: based on the identifier of the second face image in the previous frame, the face data where the identifier of the second face is located in the current tracking list is searched; if the face data where the identifier of the second face is located contains the image feature corresponding to the previous frame image, then the image feature is obtained as the image feature of the second face image; if the face data where the identifier of the second face is located does not contain the image feature corresponding to the previous frame image, then the image feature of the second face image is extracted based on the face image corresponding to the previous frame image in the face data.

6. The method according to claim 5, characterized in that, The image features of the face images in the tracking list are extracted from frame images at predetermined intervals.

7. A face tracking device, characterized in that, include: The first acquisition unit is used to acquire the face image and position information in the current frame image and the previous frame image of the video to be tracked. The third determining unit is used to determine the intersection-union ratio of each face image in the current frame image and each face image in the previous frame image based on the position information of the face images in the current frame image and the previous frame image, and to determine the first face image of the current frame image and the previous frame image that satisfy the intersection-union ratio of the preset threshold as the same face. The first determining unit is used to determine the distance between each second face image in the current frame image and each second face image in the previous frame image based on the position information of the second face image, for the current frame image and the second face image of the previous frame image where the intersection-union ratio does not meet the threshold. The second acquisition unit is used to acquire image features of the second face image based on the current frame image and the second face image of the previous frame image, which are at a distance not exceeding a first preset value. The second determining unit is used to determine the similarity between each second face image in the current frame image and each second face image in the previous frame image based on the image features of the second face images in the current frame image and the previous frame image, and to determine the second face images in the current frame image and the previous frame image that satisfy the second preset value as the same face.

8. The apparatus according to claim 7, characterized in that, The device further includes: The fourth determining unit is used to determine, after the first determining unit determines the distance between each face image in the current frame image and each face image in the previous frame image based on the position information of the face images in the current frame image and the previous frame image, the face images in the current frame image and the previous frame image whose distance exceeds the first preset value as different faces.

9. The apparatus according to claim 7, characterized in that, The device further includes: The storage unit is used to store, after the second determining unit determines the similarity between each first face image in the current frame image and each first face image in the previous frame image based on the image features of the first face images in the current frame image and the previous frame image, the face images in the current frame image and the previous frame image whose similarity does not reach the second preset value as new faces in the tracking list.

10. The apparatus according to claim 7, characterized in that, The first acquisition unit includes: The first acquisition module is used to acquire the face image and its corresponding face identifier in the previous frame image of the current frame image from the tracking list, as well as the position information of the face image in the previous frame image. The tracking list stores the position information of the face images in the frame images before the current frame image. The tracking list includes facial data of multiple faces; wherein the facial data of each face includes the face's identifier, information of each frame image, the face image of the face in each frame image, and the position information of the face image of the face in each frame image.

11. The apparatus according to claim 10, characterized in that, The facial data for each face in the tracking list further includes: image features of the face in at least one frame of each frame image; the second acquisition unit includes: The first extraction module is used to extract the image features of the second face image from the second face image in the current frame image; The second extraction module is used to perform the following processing on the second face image in the previous frame: based on the identifier of the second face image corresponding to the second face image in the previous frame image, search for the face data where the identifier of the second face is located in the current tracking list; if the face data where the identifier of the second face is located contains the image feature corresponding to the previous frame image, then obtain the image feature as the image feature of the second face image; if the face data where the identifier of the second face is located does not contain the image feature corresponding to the previous frame image, then extract the image feature of the second face image based on the face image corresponding to the previous frame image in the face data.

12. The apparatus according to claim 11, characterized in that, The image features of the face images in the tracking list are extracted from frame images at predetermined intervals.

13. An electronic device, characterized in that, include: Memory, processor; Memory; Memory used to store the processor's executable instructions; The processor is configured to execute the method as described in any one of claims 1-6 according to the executable instructions.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images