A face recognition method and recognition system for video stream

Abstract

The present invention discloses a method and system for face recognition in a video stream, comprising: an image extraction step, obtaining video stream data, and extracting a plurality of continuous video image information from the video stream data; a face structure similarity recognition step, performing face plane structure processing on face information in the video image information to obtain structured face information, and performing similarity judgment on structured face information at adjacent positions in the continuous video image information; a deep learning network recognition step, selectively performing deep learning network recognition processing on part of the face information according to the judgment result of the face structure similarity recognition step; an output step, outputting and utilizing the processing result of the deep learning network recognition step to realize dynamic tracking, processing the matched part of the face information through a deep learning network model, reducing repeated calculations, and having higher accuracy and operation efficiency.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a face recognition method and a recognition system for a video stream. Background Art

[0002] Face recognition is a biometric technology that identifies a person based on his or her facial features. It uses a computer to analyze facial images, extract valid information from the images and automatically identify the person. Nowadays, face recognition technology has been widely used in security systems and human-computer interaction, and has become one of the important research topics in the field of computer vision and pattern recognition.

[0003] Among the face recognition methods, the existing deep learning network recognition methods will collect a large amount of face information as a training library, establish a deep learning neural network model, and then input the acquired face information. It can perform calculations and recognition based on the trained neural network model to determine the degree of matching of the face information.

[0004] In order to realize face recognition in video stream, a video stream containing faces is collected by a camera or a camcorder. The video stream is actually composed of multiple image information, and each image information may contain multiple different face information. If it is necessary to use a deep learning network recognition method to process each face information in each image information, there will be a large amount of calculation and the processing efficiency is low, which cannot meet the requirements of practical application. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a face recognition method for video streams, dynamic tracking, and recognition based on a deep learning network, with high accuracy and computational efficiency.

[0006] The present invention also proposes a recognition system, which can realize rapid recognition of face information in a video stream with high accuracy.

[0007] According to the first aspect of the present invention, a face recognition method for a video stream includes: an image extraction step, obtaining video stream data and extracting multiple continuous video image information from the video stream data; a face structure similarity recognition step, performing face plane structure processing on the face information in the video image information to obtain structured face information, and performing similarity judgment on the structured face information at adjacent positions in the continuous video image information; a deep learning network recognition step, selectively performing deep learning network recognition processing on part of the face information according to the judgment result of the face structure similarity recognition step; and an output step, outputting the processing result of the deep learning network recognition step for use.

[0008] A face recognition method for a video stream according to an embodiment of the present invention has at least the following beneficial effects:

[0009] The face recognition method of the present invention extracts continuous video image information from video stream data, and analyzes the structure of the face information therein, and then makes a similarity judgment on the face information at adjacent positions in the continuous video image information. If the similarity meets the requirements, the face information with high similarity can be pre-matched to achieve dynamic tracking, and then the matched part of the face information is processed through a deep learning network model to reduce repeated calculations and have higher accuracy and computing efficiency.

[0010] According to some embodiments of the present invention, in the image extraction step, a plurality of continuous video image information is extracted from the video stream data based on the data frame.

[0011] According to some embodiments of the present invention, the facial structure similarity recognition step includes: a key point locating step, locating facial information based on facial key points in video image information; a facial structure locating step, dividing the facial information into facial plane structures to obtain structured facial information; and a similarity judgment step, performing similarity judgment on structured facial information at adjacent positions in continuous video image information.

[0012] According to some embodiments of the present invention, in the face structure location step, a Voronoi diagram algorithm is used to divide the face information into plane structures to obtain structured face information.

[0013] According to some embodiments of the present invention, in the similarity judgment step, similarity judgment is performed on structured face information at adjacent positions in continuous video image information according to the nearest neighbor principle.

[0014] According to some embodiments of the present invention, in the output step, the face information processed by the deep learning network recognition step is classified and a face recognition library is established.

[0015] According to some embodiments of the present invention, in the deep learning network recognition step, the acquired facial information is compared with the facial information in the face recognition library for deep learning network recognition judgment, and the judgment result is output in the output step.

[0016] According to some embodiments of the present invention, in the output step, the facial information is reconstructed into 3D facial information.

[0017] According to the second aspect of the present invention, the recognition system includes a processor and a memory, and the memory is used to store at least one program. When the at least one program is executed by the processor, the processor implements a face recognition method for a video stream disclosed in any of the above embodiments.

[0018] The recognition system according to the embodiment of the present invention has at least the following beneficial effects:

[0019] The recognition system of the present invention dynamically tracks faces in continuous image information in a video stream, and quickly recognizes multiple face information with high similarity and that can match each other based on a deep learning network model, with high accuracy.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0022] Figure 1 This is a main flow chart of one embodiment of the face recognition method of the present invention;

[0023] Figure 2 The present invention is a flowchart of a facial structure similarity recognition step of one embodiment of a facial recognition method. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0025] In the description of the present invention, it should be understood that descriptions involving orientation, such as the orientations or positional relationships indicated by terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside” and “outside” are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0026] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0027] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] like Figure 1 , 2 As shown, a face recognition method for a video stream according to an embodiment of the present invention includes: an image extraction step, obtaining video stream data, and extracting multiple continuous video image information from the video stream data; a face structure similarity recognition step, performing face plane structure processing on the face information in the video image information to obtain structured face information, and performing similarity judgment on the structured face information at adjacent positions in the continuous video image information; a deep learning network recognition step, selectively performing deep learning network recognition processing on part of the face information according to the judgment result of the face structure similarity recognition step; and an output step, outputting the processing result of the deep learning network recognition step for use.

[0029] The face recognition method of the present invention extracts continuous video image information from video stream data, and analyzes the structure of the face information therein, and then makes a similarity judgment on the face information at adjacent positions in the continuous video image information. If the similarity meets the requirements, the face information with high similarity can be pre-matched to achieve dynamic tracking, and then the matched part of the face information is processed through a deep learning network model to reduce repeated calculations and have higher accuracy and computing efficiency.

[0030] In some embodiments of the present invention, in the image extraction step, a plurality of continuous video image information is extracted from the video stream data based on the data frame.

[0031] In some embodiments of the present invention, Figure 2 As shown, the facial structure similarity recognition step includes a key point positioning step, a facial structure positioning step and a similarity judgment step.

[0032] Among them, in the key point positioning step, the face information is positioned based on the face key points in the video image information, and the 512-dimensional key feature points can be located. When constructing the deep learning network model, a detection deep learning network model MTCNN composed of three sub-networks of P, R, and O can be built, and the network parameters can be set.

[0033] In the face structure location step, the face information is divided into face plane structures to obtain structured face information. Specifically, the face information can be divided into face plane structures using a Voronoi diagram algorithm to obtain structured face information.

[0034] In the similarity judgment step, the similarity of structured face information at adjacent positions in the continuous video image information is judged. Specifically, the similarity of structured face information at adjacent positions in the continuous video image information is judged according to the nearest neighbor principle.

[0035] The Voronoi unit of each point in the Voronoi diagram is the area composed of the points closest to the point, which can be used to quickly process facial features. The structured face obtained in each frame of video stream data can be quickly located, thereby quickly matching the similarity of adjacent faces.

[0036] In some embodiments of the present invention, in the output step, the face information processed by the deep learning network recognition step is classified and a face recognition library is established.

[0037] It should be noted that the face recognition method of the present design can be used when constructing a deep learning network model. It can collect a large amount of face information from the video stream to form a training library. The face recognition method of the present design can speed up the construction of the deep learning network model. Similarly, after the deep learning network model is built, the present design can also be used in practical applications to recognize faces captured by the camera.

[0038] Specifically, when establishing a face recognition library, data of facial information is obtained, and for data within a face information collection angle of 180 degrees, the collected images are organized into a test set for a deep learning network model, and the deep learning network model performs feature extraction to obtain a feature vector for learning. The face area corresponding to each image is then input into the trained deep learning network model, and a 512-dimensional face feature vector for each face image is output. A deep learning network model for face recognition is obtained through iteration, and a black and white list library can be established to perform face classification through the deep learning network model.

[0039] Among them, in the black and white list library, the white list personnel are marked and classified. The same person needs to take multiple images with facial information within 180 degrees for modeling, and build classification folders. Different individuals are in the same folder, and different people are distinguished by folder names. In the subsequent deep learning network model, people who do not belong to this white list can be considered as black list personnel and stored in the black list library.

[0040] In some embodiments of the present invention, in the deep learning network recognition step, the acquired facial information is compared with the facial information in the face recognition library for deep learning network recognition judgment, and the judgment result is output in the output step.

[0041] Furthermore, in the output step, the facial information is reconstructed into 3D facial information, which can be achieved by using a conventional model for converting a flat facial image to a 3D facial image.

[0042] Specifically, in practical applications, this design locates facial information from the video stream captured by the camera, and quickly obtains the face planar structure through the Voronoi diagram algorithm. After feature extraction, the feature vector is obtained. In the deep learning network recognition step, the real-time features are compared with the trained face recognition library, and face classification is performed. Through intra-class aggregation and inter-class separation algorithms, it is determined whether the face is on the whitelist in the library. If the face is not on the whitelist in the library, or the similarity is very low, a blacklist library is constructed, and an early warning can be issued at the same time.

[0043] Furthermore, the blacklist image is reconstructed in 3D to obtain the 3D face model of the person and provide it to relevant management personnel.

[0044] According to the second aspect of the present invention, the recognition system includes a processor and a memory, and the memory is used to store at least one program. When the at least one program is executed by the processor, the processor implements a face recognition method for a video stream disclosed in any of the above embodiments.

[0045] The recognition system of the present invention dynamically tracks faces in continuous image information in a video stream, and quickly recognizes multiple face information with high similarity and that can match each other based on a deep learning network model, with high accuracy.

[0046] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0047] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A face recognition method for a video stream, characterized in that: include: An image extraction step, acquiring video stream data, and extracting a plurality of continuous video image information from the video stream data; The face structure similarity recognition step performs face plane structure processing on the face information in the video image information to obtain structured face information, and performs similarity judgment on the structured face information at adjacent positions in the continuous video image information. If the similarity meets the requirements, the face information with high similarity is pre-matched; A deep learning network recognition step, selectively performing deep learning network recognition processing on the matching partial face information according to the judgment result of the face structure similarity recognition step; The output step outputs the processing results of the deep learning network recognition step for use.

2. The face recognition method of a video stream according to claim 1, characterized in that: In the image extraction step, a plurality of continuous video image information is extracted from the video stream data based on the data frame.

3. The face recognition method of a video stream according to claim 1, characterized in that: The facial structure similarity recognition step includes: A key point positioning step, locating face information based on face key points in the video image information; The face structure location step divides the face information into plane structures to obtain structured face information; The similarity judgment step performs similarity judgment on structured face information at adjacent positions in continuous video image information.

4. The face recognition method of a video stream according to claim 3, characterized in that: In the face structure location step, the Voronoi diagram algorithm is used to divide the face information into plane structures to obtain structured face information.

5. The face recognition method of a video stream according to claim 3, characterized in that: In the similarity judgment step, the similarity judgment is performed on the structured face information at adjacent positions in the continuous video image information according to the nearest neighbor principle.

6. The method for face recognition in a video stream according to claim 1, characterized in that: In the output step, the face information processed by the deep learning network recognition step is classified and a face recognition library is established.

7. The method for face recognition in a video stream according to claim 6, characterized in that: In the deep learning network recognition step, the acquired face information is compared with the face information in the face recognition library for deep learning network recognition judgment, and the judgment result is output in the output step.

8. The face recognition method of a video stream according to claim 1, characterized in that: In the output step, the face information is reconstructed into 3D face information.

9. An identification system, characterized in that It includes a processor and a memory, the memory is used to store at least one program, when the at least one program is executed by the processor, the processor implements a face recognition method for a video stream as described in any one of claims 1-8.

Images