Optimal face image extraction method and device based on multi-face video

Abstract

The invention relates to an optimal face image extraction method and device based on a multi-face video, and the method comprises the steps: obtaining a multi-face video, and carrying out the detection of video frames in the multi-face video, and obtaining a detection result; detecting a first face feature vector in the detection result and a preset face database, and determining whether a new face is detected in the video frame; when a new face is detected in the video frame, extracting an optimal face image corresponding to the new face according to the confidence coefficient and the key points in the detection result; the optimal face image is updated to a face database; the key points are used for indicating shielding of key parts in the human face. According to the method and the device, the problems that the real-time requirement is not met and the method and the device are not suitable for real-time video analysis due to the fact that a large amount of iterative computation is needed in the prior art are solved, the identification requirement of processing a new person in real time is realized, and the adaptability is enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of audio and video processing, and in particular to an optimal facial image extraction method and device based on multi-face videos. Background Art

[0002] With the development of face recognition technology, face recognition technology has been widely used in security, transportation, education and other fields. In the application of face recognition technology, a hierarchical clustering algorithm is used to extract the best face image from multi-face videos; however, the hierarchical clustering algorithm is a method of clustering similar face features together. Although it can effectively process multi-angle features and select the best face image, it cannot meet the real-time requirements. This is because the hierarchical clustering algorithm needs to iteratively calculate a large amount of data, resulting in a long processing time, which is not suitable for application in real-time video analysis.

[0003] There is no effective solution to the problem that related technologies require a large amount of iterative calculations, which leads to failure to meet real-time requirements and are not suitable for real-time video analysis. Summary of the invention

[0004] In this embodiment, a method and device for extracting an optimal face image based on multi-face video are provided to solve the problem in the related art that a large amount of iterative calculation is required, resulting in failure to meet real-time requirements and unsuitability for real-time video analysis.

[0005] In a first aspect, in this embodiment, a method for extracting an optimal face image based on a multi-face video is provided, comprising: Acquire a multi-face video, and detect video frames in the multi-face video to obtain a detection result; Detecting the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; When a new face is detected in the video frame, the best face image corresponding to the new face is extracted according to the confidence and key points in the detection result; and the best face image is updated to the face database; the key points are used to indicate the occlusion of key parts in the face.

[0006] In some of the embodiments, obtaining a multi-face video, detecting video frames in the multi-face video, and obtaining a detection result include: Acquire a multi-face video, and pre-process the multi-face video to obtain corresponding video frames; The video frame is input into a pre-trained face detection model for recommendation to obtain a detection result.

[0007] In some embodiments, the method further comprises: After obtaining the detection result, L2 normalization processing is performed on the first facial feature vector in the detection result.

[0008] In some embodiments, the first face feature vector in the detection result is detected with a preset face database to determine whether a new face is detected in the video frame, including: Calculate the similarity between the first face feature vector and the second face feature vector in the face database to obtain a similarity result; According to the similarity result and a preset similarity threshold, an identity ID is assigned to the face in the video frame, and it is determined whether a new face is detected in the video frame.

[0009] In some of the embodiments, assigning an identity ID to a face in the video frame according to the similarity result and a preset similarity threshold, and determining whether a new face is detected in the video frame includes: When the similarity result is greater than a first threshold in the similarity thresholds, the face to be detected in the video frame is an old face, and an existing identity ID is assigned to the face to be detected; When the similarity result is less than or equal to the first threshold, and the similarity result is greater than a second threshold in the similarity thresholds, the face to be detected in the video frame is an old face, an existing identity ID is assigned to the face to be detected, and the first face feature vector is stored in the face database; When the similarity result is less than or equal to the second threshold, it is determined that a new face is detected in the video frame, and a new identity ID is assigned to the new face.

[0010] In some embodiments, similarity calculation is performed between the first face feature vector and the second face feature vector in the face database to obtain a similarity result, including: The similarity between the first face feature vector and the second face feature vector in the face database is calculated by using a cosine similarity formula to obtain a similarity result.

[0011] In some embodiments, the method further comprises: When the first facial feature vector is stored in the face database, the face database is dynamically updated based on a preset update strategy.

[0012] In a second aspect, in this embodiment, an optimal face image extraction device based on multi-face video is provided, including: a processing module, a detection module and an extraction module; The processing module is used to obtain a multi-face video, detect video frames in the multi-face video, and obtain a detection result; The detection module is used to detect the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; The extraction module is used to extract the best face image corresponding to the new face according to the confidence and key points in the detection results when a new face is detected in the video frame; and update the best face image to the face database; the key points are used to indicate the occlusion of key parts in the face.

[0013] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the optimal face image extraction method based on multi-face video described in the first aspect is implemented.

[0014] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the optimal face image extraction method based on multi-face video described in the first aspect above is implemented.

[0015] Compared with the related art, the best face image extraction method and device based on multi-face video provided in this embodiment obtains multi-face video, detects the video frame in the multi-face video, and obtains the detection result; detects the first face feature vector in the detection result with the preset face database to determine whether a new face is detected in the video frame; when a new face is detected in the video frame, extracts the best face image corresponding to the new face according to the confidence and key points in the detection result; and updates the best face image to the face database; the key points are used to indicate the occlusion of key parts in the face, which solves the problem in the related art that a large number of iterative calculations are required, resulting in failure to meet the real-time requirements and being unsuitable for real-time video analysis. It can automatically detect the newly appeared face in the multi-face video and extract the best face image of the face, realizing the real-time processing of the recognition needs of new characters and enhancing the adaptability.

[0016] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 This is a hardware structure block diagram of a terminal device for an optimal face image extraction method based on multi-face video provided by an embodiment of the present application; Figure 2 is a flow chart of an optimal face image extraction method based on multi-face video provided by an embodiment of the present application; Figure 3 is a flowchart of step S210; Figure 4 is a flowchart of step S220; Figure 5 It is a structural block diagram of an optimal face image extraction device based on multi-face video provided by an embodiment of the present application.

[0018] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, processing module; 220, detection module; 230, extraction module. DETAILED DESCRIPTION

[0019] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the optimal face image extraction method based on multi-face video of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0022] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the optimal face image extraction method based on multi-face video in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0024] In this embodiment, a method for extracting an optimal face image based on a multi-face video is provided. Figure 2 is a flow chart of the optimal face image extraction method based on multi-face video of this embodiment. Figure 2As shown, the process includes the following steps: Step S210, obtaining a multi-face video, detecting video frames in the multi-face video, and obtaining a detection result; Step S220, detecting the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; Step S230, when a new face is detected in the video frame, the best face image corresponding to the new face is extracted according to the confidence and key points in the detection results; and the best face image is updated to the face database; the key points are used to indicate the occlusion of key parts in the face.

[0025] Specifically, a multi-face video refers to a video containing multiple faces, such as a video of a company's face clock-in point, a video of an attendance office, a video of a security checkpoint, etc. In actual application, the method of obtaining a multi-face video in the embodiment of the present application includes, but is not limited to, obtaining a multi-face video that meets the above requirements from a database pre-stored therein; it can also be downloading a multi-face video that meets the requirements from a network platform; it can also be generating a corresponding multi-face video according to demand, etc. The embodiment of the present application does not limit the method of obtaining a multi-face video.

[0026] After obtaining a multi-face video, the multi-face video can be decoded to obtain the video frames in the multi-face video; it can be considered that each video frame also has multiple faces, and one video frame is a face image. Then, a well-trained face detection model or face detection algorithm is used to detect the video frames in the multi-face video to obtain the detection results. Among them, the detection results at least include the first face feature vector (embedding1), confidence, and key points (kps); the first face feature vector is a vector composed of a set of representative numerical features obtained after the video frame is detected. It can be regarded as a mathematical representation of a face image in a high-dimensional feature space, which can capture the key feature information of the face, such as the angle, shape, position, texture, etc. of the facial organs, and can identify multiple angles or states of the face. Key points are used to indicate the occlusion of key parts in the face; for example, if key parts such as the eyes, nose, and mouth are obviously occluded, the priority of the video frame will be reduced. The confidence is the score of the face in the video frame, with the frontal photo having the highest confidence; the higher the score, the higher the credibility of the face photo. In other embodiments, the detection result may also include a bounding box; each bounding box represents a face image.

[0027] Next, the first face feature vector in the detection result is used as the detection basis to traverse the preset face database. If the second face feature vector similar to the first face feature vector is not detected, it can be considered that a new face is detected in the video frame. The face database is pre-set and stores the identity ID, second face feature vector, and confidence level corresponding to each face.

[0028] Finally, when a new face is detected in a video frame, since the confidence is the score of the face in the video frame, and the key points are used to indicate the occlusion of key parts in the face; then according to the confidence and key points in the detection results, the best face image corresponding to the new face is extracted from each video frame; that is, this embodiment can automatically select the best clear and unobstructed face image in the video frames (face images) at different times, and update the best face image to the face database to ensure that the stored image has high quality and easy recognition. In this embodiment, the relevant data in the detection results are used to automatically detect the new face in the multi-face video, and extract the best face image of the face, so as to realize the real-time processing of the recognition needs of new people and enhance the adaptability.

[0029] In related technologies, a hierarchical clustering algorithm is used to extract the best face image from a multi-face video; however, the hierarchical clustering algorithm is a method of clustering similar face features together. Although it can effectively process multi-angle features and select the best face image, it cannot meet the real-time requirements. This is because the hierarchical clustering algorithm needs to iteratively calculate a large amount of data, resulting in a long processing time, and is not suitable for application in real-time video analysis. In the present embodiment, a multi-face video is acquired, and the video frames in the multi-face video are detected to obtain a detection result; the first face feature vector in the detection result is detected with a preset face database to determine whether a new face is detected in the video frame; when a new face is detected in the video frame, the best face image corresponding to the new face is extracted according to the confidence and key points in the detection result; and the best face image is updated to the face database; the key points are used to indicate the occlusion of key parts in the face, which solves the problem in the related technology that a large number of iterative calculations are required, resulting in failure to meet the real-time requirements and being unsuitable for real-time video analysis. The present invention can automatically detect a new face in a multi-face video and extract the best face image of the face, thereby realizing real-time processing of the recognition needs of new characters and enhancing the adaptability.

[0030] The following is a detailed description of each of the above steps: In some of the embodiments, Figure 3 As shown, the step S210 of acquiring a multi-face video, detecting video frames in the multi-face video, and obtaining a detection result includes the following steps: Step S211, obtaining a multi-face video, pre-processing the multi-face video, and obtaining corresponding video frames; Step S212: input the video frame into a pre-trained face detection model for recommendation to obtain a detection result.

[0031] Specifically, since multi-face videos are usually stored after encoding and compression, decoding operations are required to convert them into image frame formats that can be directly processed by computers (such as pixel matrices in RGB or BGR formats). This process can be automatically completed by calling a video codec plug-in. Then read the video frames in the multi-face video frame by frame in sequence. Specific frame rate extraction rules can be set, such as extracting one frame every certain number of frames, to reduce the amount of data processed and improve processing efficiency. In practical applications, the appropriate extraction strategy is determined based on specific needs and content characteristics of the multi-face video, and there is no restriction on this.

[0032] Among them, the face detection model is pre-trained, and it can adopt the YOLO series model, SSD (Single Shot MultiBox Detector) model, Mask R-CNN, etc. In this embodiment, the face detection model can be MTCNN (Multi-task Cascaded Convolutional Networks); it adopts a cascade structure and consists of three convolutional neural networks (P-Net, R-Net and O-Net). P-Net is used to generate the bounding box of the candidate face area, R-Net screens and refines the candidate area, and O-Net further improves the detection accuracy and outputs the key points, face feature vectors and confidence of the face. Then the video frame is input into the pre-trained face detection model for recommendation, and the detection result can be obtained. The detection result includes the bounding box, key points, face feature vectors and confidence.

[0033] Through this embodiment, the processing of multi-face videos is completed efficiently and accurately, which can be applied to real-time video analysis while preparing for subsequent processing tasks.

[0034] In some embodiments, the optimal face image extraction method based on multi-face video further includes: After the detection result is obtained, L2 normalization processing is performed on the first face feature vector in the detection result.

[0035] Specifically, the first facial feature vector in the detection result is subjected to L2 normalization processing; L2 normalization processing will scale the L2 norm (i.e., the Euclidean length of the vector) of the first facial feature vector to 1, so that the modulus length of each vector is equal, and L2 normalization ensures that the directional information of the feature vector remains unchanged; so that in the subsequent calculation of cosine similarity, it will not be affected by the size of the vector, and only the directional consistency is concerned, thereby improving the accuracy of the cosine similarity calculation. In other embodiments, other normalization methods may also be used, and this is not limited.

[0036] It should be noted that the first face feature vector and the second face feature vector in the face database are obtained by the same processing method. For example, if the first face feature vector is processed by L2 normalization, the second face feature vector is also processed by L2 normalization.

[0037] In some of the embodiments, Figure 4 As shown, step S220 detects the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame, including the following steps: Step S221, calculating the similarity between the first face feature vector and the second face feature vector in the face database to obtain a similarity result; Step S222, assigning an identity ID to the face in the video frame according to the similarity result and a preset similarity threshold, and determining whether a new face is detected in the video frame.

[0038] In practical applications, the first facial feature vector and the second facial feature vector are the same type of data, but they are located in different positions. The first facial feature vector is obtained by processing the video frame of the current frame, and the second facial feature vector is stored in the face database. The similarity between the first facial feature vector and the second facial feature vector can be calculated using similarity calculation formulas such as cosine similarity, Pearson correlation coefficient, Euclidean distance or Manhattan distance. Specifically, the corresponding similarity calculation formula can be selected according to different application scenarios. For example, in the fields of data mining, machine learning, etc., the Euclidean distance similarity formula can be used. In fields such as face recognition, the cosine similarity formula can be used, and its expression can be: Cosine Similarity(A,B)=A×B; Where A represents the first face feature vector; B represents the second face feature vector.

[0039] Among them, the cosine similarity formula can be considered that cosine similarity is equal to the dot product of two feature vectors (the sum of the products of the corresponding elements). The greater the cosine similarity of two faces, the more similar they are. Then, after obtaining the similarity result, the similarity result is compared with the preset similarity threshold to obtain the comparison result of whether a new face is detected in the video frame; for the comparison result of detecting a new face, a new identity ID is created and assigned to the new face; for the comparison result of detecting an old face, the old identity ID is assigned to the old face.

[0040] Through this embodiment, the similarity between the first facial feature vector and the second facial feature vector is first calculated to obtain a similarity result; then, based on the similarity result and a preset similarity threshold, an identity ID is assigned to the face in the video frame, and it is determined whether a new face is detected in the video frame; thereby achieving automatic identity matching and assignment of an identity ID.

[0041] In some embodiments, the step S222 assigns an identity ID to a face in a video frame according to the similarity result and a preset similarity threshold, and determines whether a new face is detected in the video frame, including the following steps: When the similarity result is greater than the first threshold in the similarity threshold, the face to be detected in the video frame is an old face, and an existing identity ID is assigned to the face to be detected; When the similarity result is less than or equal to the first threshold value, and the similarity result is greater than the second threshold value in the similarity threshold value, the face to be detected in the video frame is an old face, an existing identity ID is assigned to the face to be detected, and the first face feature vector is stored in the face database; When the similarity result is less than or equal to the second threshold, it is determined that a new face is detected in the video frame, and a new identity ID is assigned to the new face.

[0042] Specifically, the similarity threshold is pre-set. In this embodiment, there are two similarity thresholds, including a first threshold and a second threshold. The first threshold is greater than the second threshold. The specific parameters of the first threshold and the second threshold can be set according to the usage scenario, and there is no limitation on this.

[0043] For example: the first threshold is 0.8; the second threshold is 0.5; when the similarity result of the face to be detected in the video frame is greater than 0.8, it is considered that the face to be detected in the video frame is an old face, and is the same face as the face corresponding to the second face feature vector in the face data volume, so the identity ID of the old face is assigned to the face to be detected. When the similarity result is less than or equal to 0.8, and the similarity result is greater than 0.5, it is considered that the face to be detected in the video frame is an old face, and is considered that the face to be detected in the video frame is an old face, and is the same face as the face corresponding to the second face feature vector in the face data volume, so the identity ID of the old face is assigned to the face to be detected; however, it is considered that the face image of the face to be detected is different from the face image in the face database (possibly different angles or expression changes of known faces), so the first face feature vector is stored in the face database to further improve the face image in the face database, and continuously optimize the face database to improve the recognition effect. When the similarity result is less than or equal to 0.5, the face to be detected in the video frame is considered to be a new face, that is, it is determined that a new face is detected in the video frame. At this time, a new identity ID will be created for the new face and the created identity ID will be assigned to the new face.

[0044] Through this embodiment, facial feature vectors from multiple angles can be integrated to improve recognition accuracy across scenarios, and the first facial feature vector is stored in a face database, so that multi-angle and multi-expression features of each identity are provided, effectively reducing the occurrence of misrecognition.

[0045] In some of the embodiments, the optimal face image extraction method based on multi-face video further includes the following steps: When the first facial feature vector is stored in the face database, the face database is dynamically updated based on a preset update strategy.

[0046] Specifically, the second face feature vector list of each identity ID in the face database is limited to no more than N; N represents the second face feature vectors of N different angles or states. In this embodiment, N can be 10.

[0047] When the similarity result is less than or equal to the first threshold, and the similarity result is greater than the second threshold in the similarity threshold, the first facial feature vector corresponding to the new face is considered to be highly representative. At this time, the first facial feature vector will be stored in the feature list of the face database (each face corresponds to a feature list and an identity ID; the feature list stores N second facial feature vectors).

[0048] If the feature list has reached the upper limit, the corresponding feature vector is removed according to the predetermined update strategy. The update strategy is that in the feature list of each identity ID, when a new first face feature vector is stored, the length of the feature list of the identity ID is checked. If the feature list has reached the maximum number, the second face feature vector is removed according to the first-in-first-out principle or the confidence level, and then the new first face feature vector is stored and becomes the second face feature vector.

[0049] Through this embodiment, the multi-angle and multi-expression facial feature vectors of each identity can be dynamically updated, effectively reducing the occurrence of misidentification; at the same time, the comparison speed and efficiency can be improved.

[0050] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] In this embodiment, an optimal face image extraction device based on multi-face video is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0052] Figure 5 is a structural block diagram of the optimal face image extraction device based on multi-face video of this embodiment, as shown in Figure 5 As shown, the device includes: a processing module 210, a detection module 220 and an extraction module 230; The processing module 210 is used to obtain a multi-face video, detect video frames in the multi-face video, and obtain a detection result; A detection module 220, configured to detect the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; The extraction module 230 is used to extract the best face image corresponding to the new face according to the confidence and key points in the detection results when a new face is detected in the video frame; and update the best face image to the face database; the key points are used to indicate the occlusion of key parts in the face.

[0053] Through the above-mentioned device, the problem in the related technology that a large amount of iterative calculations are required, resulting in failure to meet real-time requirements and unsuitability for real-time video analysis is solved. The device can automatically detect newly appeared faces in multi-face videos and extract the best face image of the face, thereby realizing real-time processing of new character recognition needs and enhancing adaptability.

[0054] In some of the embodiments, the processing module 210 is further used to obtain a multi-face video, pre-process the multi-face video, and obtain corresponding video frames; Input the video frame into the pre-trained face detection model for recommendation and obtain the detection result.

[0055] In some of these embodiments, the apparatus further comprises: a normalization module; The normalization module is used to perform L2 normalization processing on the first face feature vector in the detection result after obtaining the detection result.

[0056] In some of the embodiments, the detection module 220 is further used to calculate the similarity between the first face feature vector and the second face feature vector in the face database to obtain a similarity result; According to the similarity result and the preset similarity threshold, an identity ID is assigned to the face in the video frame, and it is determined whether a new face is detected in the video frame.

[0057] In some of the embodiments, the detection module 220 is further configured to, when the similarity result is greater than a first threshold in the similarity threshold, determine that the face to be detected in the video frame is an old face, and assign an existing identity ID to the face to be detected; When the similarity result is less than or equal to the first threshold value, and the similarity result is greater than the second threshold value in the similarity threshold value, the face to be detected in the video frame is an old face, an existing identity ID is assigned to the face to be detected, and the first face feature vector is stored in the face database; When the similarity result is less than or equal to the second threshold, it is determined that a new face is detected in the video frame, and a new identity ID is assigned to the new face.

[0058] In some of the embodiments, the detection module 220 is further configured to calculate the similarity between the first facial feature vector and the second facial feature vector in the face database by using a cosine similarity formula to obtain a similarity result.

[0059] In some of these embodiments, the apparatus further comprises: an update module; The updating module is used to dynamically update the face database based on a preset updating strategy when storing the first face feature vector in the face database.

[0060] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0061] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0062] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0063] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining a multi-face video, detecting video frames in the multi-face video, and obtaining a detection result; S2, detecting the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; S3, when a new face is detected in a video frame, the best face image corresponding to the new face is extracted according to the confidence and key points in the detection results; and the best face image is updated to the face database; the key points are used to indicate the occlusion of key parts of the face.

[0064] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0065] In addition, in combination with the optimal face image extraction method based on multi-face video provided in the above embodiment, a storage medium can also be provided in this embodiment to implement. The storage medium stores a computer program; when the computer program is executed by the processor, any optimal face image extraction method based on multi-face video in the above embodiment is implemented.

[0066] It should be noted that the information and data involved in this application are all authorized by the user or fully authorized by all parties and will be used legally.

[0067] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0068] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0069] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0070] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An optimal face image extraction method based on multi-face video, characterized in that: include: Acquire a multi-face video, and detect video frames in the multi-face video to obtain a detection result; Detecting the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; When a new face is detected in the video frame, the best face image corresponding to the new face is extracted according to the confidence and key points in the detection result; and the best face image is updated to the face database; the key points are used to indicate the occlusion of key parts in the face.

2. The optimal face image extraction method based on multi-face video according to claim 1 is characterized in that: Acquire a multi-face video, detect video frames in the multi-face video, and obtain a detection result, including: Acquire a multi-face video, and pre-process the multi-face video to obtain corresponding video frames; The video frame is input into a pre-trained face detection model for recommendation to obtain a detection result.

3. The optimal face image extraction method based on multi-face video according to claim 1 is characterized in that: The method further comprises: After obtaining the detection result, L2 normalization processing is performed on the first facial feature vector in the detection result.

4. The optimal face image extraction method based on multi-face video according to any one of claims 1 to 3, characterized in that: Detecting the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame includes: Calculate the similarity between the first face feature vector and the second face feature vector in the face database to obtain a similarity result; According to the similarity result and a preset similarity threshold, an identity ID is assigned to the face in the video frame, and it is determined whether a new face is detected in the video frame.

5. The optimal face image extraction method based on multi-face video according to claim 4 is characterized in that: According to the similarity result and a preset similarity threshold, an identity ID is assigned to the face in the video frame, and it is determined whether a new face is detected in the video frame, including: When the similarity result is greater than a first threshold in the similarity thresholds, the face to be detected in the video frame is an old face, and an existing identity ID is assigned to the face to be detected; When the similarity result is less than or equal to the first threshold, and the similarity result is greater than a second threshold in the similarity thresholds, the face to be detected in the video frame is an old face, an existing identity ID is assigned to the face to be detected, and the first face feature vector is stored in the face database; When the similarity result is less than or equal to the second threshold, it is determined that a new face is detected in the video frame, and a new identity ID is assigned to the new face.

6. The optimal face image extraction method based on multi-face video according to claim 4 is characterized in that: Calculating the similarity between the first face feature vector and the second face feature vector in the face database to obtain a similarity result includes: The similarity between the first face feature vector and the second face feature vector in the face database is calculated by using a cosine similarity formula to obtain a similarity result.

7. The optimal face image extraction method based on multi-face video according to claim 5, characterized in that: The method further comprises: When the first facial feature vector is stored in the face database, the face database is dynamically updated based on a preset update strategy.

8. An optimal face image extraction device based on multi-face video, characterized in that: include: A processing module, a detection module, and an extraction module; The processing module is used to obtain a multi-face video, detect video frames in the multi-face video, and obtain a detection result; The detection module is used to detect the first face feature vector in the detection result with a preset face database to determine whether a new face is detected in the video frame; The extraction module is used to extract the best face image corresponding to the new face according to the confidence and key points in the detection results when a new face is detected in the video frame; and update the best face image to the face database; the key points are used to indicate the occlusion of key parts in the face.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the optimal face image extraction method based on multi-face video according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the optimal face image extraction method based on multi-face video according to any one of claims 1 to 7 are implemented.

Images