A personnel identification method, device and medium based on real-time video stream

Abstract

The application provides a personnel identification method and device based on real-time video stream and a medium, the method comprising: obtaining a first video stream corresponding to a first target area in which a to-be-identified person is located in real time; obtaining a plurality of first to-be-detected images; extracting feature information from each first to-be-detected image to obtain a plurality of to-be-identified personnel feature information corresponding to the to-be-identified person; determining the to-be-identified person as a target person if the first database stores first identity information corresponding to the to-be-identified face feature information; obtaining a plurality of target images; and performing encapsulation processing on each target image to obtain a target video stream. By using computer vision, machine learning and multimedia processing technology, the application can quickly and accurately extract and identify personnel features in real-time video stream, and mark the relevant information of the target person on the first to-be-detected image, so that the personnel identification result is more explicit and visualized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a personnel identification method based on real-time video stream, device and medium. BACKGROUND

[0002] In the current field of security monitoring and personnel management, the demand for real-time video stream processing and personnel feature identification is becoming increasingly important. Traditional security monitoring systems usually rely on manual operation and analysis, which is time-consuming and prone to errors. With the development of computer vision and machine learning technology, automated real-time video analysis systems have become a more efficient and reliable solution.

[0003] Current face recognition and image analysis technology is limited to processing static images, and has limited processing capability for real-time video streams. In addition, existing technologies still face some challenges in the accuracy and efficiency of personnel feature identification. For example, it is still challenging to simultaneously identify and extract features of multiple personnel in complex scenarios, as well as accurately match personnel information. Existing technologies often require a large amount of computing resources and time to process real-time video streams, and are prone to feature extraction errors or identification errors in complex scenarios. In high-density personnel scenarios, simultaneously identifying and extracting features of multiple personnel is even more complex, and existing technologies are difficult to accurately distinguish and extract features of multiple personnel, and are prone to mis-matching and information confusion. Existing technologies also have limitations and shortcomings in matching real-time extracted personnel features with personnel databases, obtaining names and ID numbers, and labeling related information on images. In real-time video stream processing, the real-time stream and file generation methods in existing technologies are also inconvenient and inefficient. SUMMARY

[0004] Therefore, the present application provides a personnel identification method based on real-time video stream, device and medium, which at least partially solves the technical problems existing in the prior art. The technical solution adopted by the present application is as follows:

[0005] According to one aspect of the present application, a personnel identification method based on real-time video stream is provided, which is applied to a personnel identification system connected with a first database and a target display interface, and the first database stores a plurality of first personnel corresponding first face feature vectors and first identity information.

[0006] The personnel identification method based on real-time video stream comprises the following steps:

[0007] S100, real-time acquiring a first video stream corresponding to a to-be-identified personnel in a first target area;

[0008] S200, pre-processing the first video stream to obtain a plurality of first to-be-detected images;

[0009] S300, feature information extraction is performed on each first to-be-detected image to obtain a plurality of to-be-recognized personnel feature information corresponding to the to-be-recognized personnel; the to-be-recognized personnel feature information includes to-be-recognized face feature information and to-be-recognized appearance feature information corresponding to the to-be-recognized personnel;

[0010] S400, if the first database stores first identity information corresponding to the to-be-recognized face feature information, the to-be-recognized personnel is determined as the target personnel;

[0011] S500, a plurality of target appearance feature information and target identity information corresponding to the target personnel are marked in a first marking box corresponding to each first to-be-detected image to obtain a plurality of target images;

[0012] S600, encapsulation processing is performed on each target image to obtain a target video stream;

[0013] S700, the target video stream is displayed in a target display interface.

[0014] In an exemplary embodiment of the present application, step S200 includes:

[0015] S210, decoding processing is performed on the first video stream to obtain a plurality of first video frames;

[0016] S220, frame extraction processing is performed on each first video frame to obtain a plurality of first to-be-detected images.

[0017] In an exemplary embodiment of the present application, step S300 further includes:

[0018] S301, according to a machine learning algorithm, each to-be-recognized personnel feature information corresponding to the to-be-recognized personnel is classified to obtain a plurality of to-be-recognized feature information classes, and each to-be-recognized feature information class includes a plurality of to-be-recognized personnel feature information.

[0019] In an exemplary embodiment of the present application, the to-be-recognized face feature information is determined by the following steps:

[0020] S310, according to a face detection algorithm, a face region in each first to-be-detected image is determined;

[0021] S320, according to a face key point positioning algorithm, each face region in each first to-be-detected image is detected to determine a key feature information point in each face region;

[0022] S330, according to a face feature information extraction algorithm and each key feature information point, to-be-recognized face feature information in each first to-be-detected image is extracted.

[0023] In one exemplary embodiment of this application, step S400 includes:

[0024] S410. Based on n facial feature information to be identified, obtain the facial feature information vector Q = (Q1, Q2, ..., Qn) corresponding to the person to be identified. i ,...,Q n ); where i = 1, 2, ..., n; Q i This refers to the facial feature information of the i-th person to be identified;

[0025] S420. Obtain the first facial feature vectors R1, R2, ..., R1 corresponding to the h first persons stored in the first database. g ,...,R h ;R g =(R g1 ,R g2 ,...,R gi ,...,R gn ); where g = 1, 2, ..., h; R g R is the first face feature vector corresponding to the g-th first person; gi R represents the i-th first face feature information in the first face feature vector corresponding to the g-th first person; gi The corresponding key feature information points and Q i The corresponding key feature information points are the same;

[0026] S430, Connect Q to R1,R2,...,R g ,...,R h The matching degree is calculated separately to obtain the corresponding first matching degree M1, M2, ..., M g ,...,M h ;

[0027] S440, If MAX(M1,M2,...,M... g ,...,M h If the match rate is greater than or equal to M0, then the person to be identified is identified as the target person; where M0 is the preset matching threshold; and MAX() is the preset maximum value determination function.

[0028] S450, set MAX(M1,M2,...,M) g ,...,M h The first identity information of the first person corresponding to the target identity information is determined.

[0029] S460. The facial features to be identified are determined as the target facial features.

[0030] In one exemplary embodiment of this application, the first identifier frame is determined through the following steps:

[0031] S510. According to the bounding box algorithm, the human body region in each first image to be detected is selected to obtain the initial label box corresponding to each first image to be detected.

[0032] S520. Obtain the field lengths of several target appearance feature information corresponding to the target person in each first image to be detected, and obtain the field length set F = (F1, F2, ..., F...). j ,...,F m );F j =(F j1 ,F j2 ,...,F jd ,...,F jf(j) ); where j = 1, 2, ..., m; d = 1, 2, ..., f(j); m is the number of the first images to be detected; f(j) is the number of target appearance feature information corresponding to the target person in the j-th first image to be detected; F j F is a list of field lengths for the target person's appearance feature information in the j-th first image to be detected; jd The length of the field of the target person's appearance feature information corresponding to the dth target in the jth first image to be detected;

[0033] S530, Obtain the field length G of the target identity information;

[0034] S540, If G≥MAX(F) j If the initial bounding box is G units long, then the boundary of the initial bounding box is expanded outwards, and this expansion is used to define the first bounding box corresponding to the j-th first image to be detected; otherwise, the boundary of the initial bounding box is expanded outwards by MAX(F). j The length is determined as the first bounding box corresponding to the j-th first image to be detected.

[0035] In one exemplary embodiment of this application, after step S540, the method further includes:

[0036] S550. If the first identifier box contains the boundary of the corresponding first image to be detected, then the boundary is taken as the boundary of the corresponding first identifier box.

[0037] In one exemplary embodiment of this application, step S600 includes:

[0038] S610. According to the audio and video coding algorithm, encode the m target images to obtain the corresponding second video frame;

[0039] S620, encapsulate each second video frame as a real-time rtmp stream to obtain a target video stream;

[0040] S630, store each second video frame in an flv file format.

[0041] According to an aspect of the present application, a non-transitory computer readable storage medium is provided, the storage medium storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the aforementioned personnel identification method based on real-time video stream.

[0042] According to an aspect of the present application, an electronic device is provided, comprising a processor and the aforementioned non-transitory computer readable storage medium.

[0043] The present application has at least the following beneficial effects:

[0044] The present application provides a high-efficiency and accurate personnel feature extraction, identification and labeling method based on real-time video stream, which can quickly and accurately extract and identify personnel features in real-time video stream by using computer vision, machine learning and multimedia processing technology, realize real-time information acquisition and labeling, greatly improve the processing speed and efficiency, and compare the first face feature vector obtained according to the personnel feature information to be identified with the first face feature vector in the first database, realize the acquisition of the first identity information of the first personnel, provide a convenient and practical means for security monitoring and personnel management, and label the target personnel corresponding to the target appearance feature information and target identity information on the first to-be-detected image, which can provide intuitive visual information, make the personnel identification result more clear and visualized, facilitate further analysis and application, at the same time, generate a new real-time rtmp stream and an flv file, so that the processed video stream can be conveniently transmitted and saved in real time, provide convenience for subsequent data analysis, archiving and playback, and meet the needs of the field of security monitoring and personnel management. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 The flowchart of the personnel identification method based on real-time video stream provided by the embodiments of the present application. DETAILED DESCRIPTION

[0047] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the protection scope of the present application.

[0048] A personnel identification method based on real-time video stream, applied to a personnel identification system, the personnel identification system being connected with a first database and a target display interface, the first database storing a first face feature vector and a first identity information corresponding to a plurality of first personnel.

[0049] The personnel identification system is used for face identification and identity verification of the to-be-identified personnel, the first database is used for storing the first face feature vector and the first identity information of the first personnel, the first personnel is a personnel whose identity information has been stored, the first face feature vector is a feature vector obtained according to face feature information of the corresponding first personnel, the first identity information is identity information of the corresponding first personnel, and the identity information includes a name and an identity number of the corresponding first personnel, the target display interface is an interface for displaying corresponding identification information of the identified personnel, and the target display interface is used for displaying real-time video stream, that is, when a first video stream of the to-be-identified personnel is received, identity verification and marking of corresponding identification information of the to-be-identified personnel are performed, and the to-be-identified personnel is displayed in the target display interface.

[0050] An srs (Simple Realtime Server) service is built in the personnel identification system and used for receiving an rtmp video stream, an ffmpeg environment is installed in the personnel identification system and used for generating the rtmp video stream, and the corresponding first face feature vector and the first identity information are obtained through photo information, face feature information and personnel basic information of each first personnel.

[0051] As shown in Figure 1 The personnel identification method based on real-time video stream includes the following steps:

[0052] S100, a first video stream corresponding to a to-be-identified personnel in a first target area is acquired in real time;

[0053] The first target area is a designated area in which identity identification and verification of a visitor are required, the visitor is the to-be-identified personnel, the first video stream is a real-time video stream of the first target area collected through an image collection device, the first video stream is an rtmp video stream, and the image collection device can be a camera placed in the first target area.

[0054] S200, the first video stream is preprocessed to obtain a plurality of first to-be-detected images;

[0055] The preprocessing is processed by a software library established in the personnel identification system, and the software library can be OpenCV (cross-platform computer vision library). The preprocessing includes decoding processing and frame extraction processing in sequence, for decoding and extracting image frames of the received first video stream in real time to obtain a plurality of first detection images corresponding to the first video stream. Each first detection image corresponds to an acquisition time, that is, a shooting time. The plurality of first detection images are sorted according to the corresponding acquisition time, so as to facilitate the generation of the video stream after identity recognition verification according to the acquisition time, to prevent the occurrence of garbled codes.

[0056] Further, step S200 includes:

[0057] S210, decoding processing is performed on the first video stream to obtain a plurality of first video frames;

[0058] S220, frame extraction processing is performed on each first video frame to obtain a plurality of first detection images.

[0059] S300, feature information of each first detection image is extracted to obtain a plurality of to-be-identified personnel feature information corresponding to the to-be-identified personnel; the to-be-identified personnel feature information includes to-be-identified face feature information and to-be-identified appearance feature information of the to-be-identified personnel;

[0060] After obtaining a plurality of first detection images, since identity recognition is to be performed on the to-be-identified personnel in each first detection image and information labeling is to be performed on the corresponding first detection image, feature information of the to-be-identified personnel in each first detection image is extracted to obtain to-be-identified personnel feature information. The to-be-identified personnel feature information is feature information of the to-be-identified personnel in the corresponding first detection image, including to-be-identified face feature information and to-be-identified appearance feature information of the to-be-identified personnel. The to-be-identified face feature information is face feature information of the to-be-identified personnel. The to-be-identified appearance feature information includes age information, upper garment color, lower garment color, shoe color, and whether there is a beard of the to-be-identified personnel. Each first detection image corresponds to a plurality of to-be-identified personnel feature information. For example, only the upper body position of the to-be-identified personnel can be obtained in one of the first detection images. Therefore, the to-be-identified personnel feature information obtained by the first detection image does not include corresponding lower garment color and shoe color information. The to-be-identified face feature information is used for identity verification of the to-be-identified personnel. The to-be-identified appearance feature information is used for more accurate identification of the to-be-identified personnel, which helps to avoid misidentification and information confusion, and provides a reliable basis for subsequent personnel comparison.

[0061] The to-be-identified face feature information is determined by the following steps:

[0062] S310, according to a face detection algorithm, a face region in each first detection image is determined.

[0063] The face detection algorithm can be a detection algorithm performed by a deep learning-based face detection model, used to locate the face region in the first to-be-detected image.

[0064] S320, detecting the face region in each first to-be-detected image according to the face key point positioning algorithm to determine the key feature information points in each face region;

[0065] The face key point positioning algorithm can be a method based on a regression model or deep learning, used to accurately locate the key points of the face, such as the eye, nose, mouth, and other feature points.

[0066] S330, extracting the to-be-recognized face feature information in each first to-be-detected image according to the face feature information extraction algorithm and each key feature information point.

[0067] The face feature information extraction algorithm can be an extraction algorithm performed by a deep learning-based feature extraction model, used to extract the feature information of the face.

[0068] Further, step S300 further includes:

[0069] S301, classifying each to-be-recognized personnel feature information corresponding to the to-be-recognized personnel according to the machine learning algorithm to obtain a plurality of to-be-recognized feature information groups, and each to-be-recognized feature information group includes a plurality of to-be-recognized personnel feature information.

[0070] The machine learning algorithm can be a support vector machine (SVM) or a convolutional neural network (CNN) algorithm, used to classify and identify personnel features, such as age information class, upper garment color and style class, lower garment color and style class, shoe color and style class, etc., to facilitate the statistics of the obtained to-be-recognized personnel feature information.

[0071] S400, if the first database stores the first identity information corresponding to the to-be-recognized face feature information, the to-be-recognized personnel is determined as the target personnel;

[0072] Further, step S400 includes:

[0073] S410, obtaining the face feature information vector Q=(Q1, Q2,..., Qn) corresponding to the to-be-recognized personnel according to the n to-be-recognized face feature information; wherein i=1, 2,..., n; Q i is the i-th to-be-recognized face feature information corresponding to the to-be-recognized personnel; n i

[0074] ​​The face feature information to be identified is face feature information corresponding to the to-be-identified person, and the face feature information corresponding to the to-be-identified person is formed into a face feature information vector of the to-be-identified person according to the face feature information of the to-be-identified person.

[0075] S420, obtain the first face feature vectors R1, R2,..., R g ,...,R h of the h first persons stored in the first database. g g1 g2 gi gn ; wherein g = 1, 2,..., h; R g is the first face feature vector of the gth first person; R gi is the ith first face feature information in the first face feature vector of the gth first person; R gi corresponds to the same key feature information point as Q i corresponds to the same key feature information point as Q

[0076] In order to facilitate subsequent comparison of the first face feature vector and the face feature information vector, the key feature information point corresponding to each first face feature information in each first face feature vector is the same as the key feature information point corresponding to each to-be-identified face feature information in the face feature information vector.

[0077] S430, respectively calculate the matching degrees of Q and R1, R2,..., R g ,...,R h , and obtain the corresponding first matching degrees M1, M2,..., M g ,...,M h .

[0078] The matching degree calculation can be implemented by a matching degree measurement algorithm, such as a cosine matching degree algorithm or a Euclidean distance algorithm. Through the matching degree measurement algorithm, the first matching degree between the face feature information vector and each first face feature vector is obtained. The greater the first matching degree, the more similar the face feature information vector and the corresponding first face feature vector.

[0079] S440, if MAX(M1, M2,..., M g ,...,M h ) ≥ M0, then the to-be-identified person is determined as the target person; wherein M0 is a preset matching degree threshold; MAX() is a preset maximum value determination function.

[0080] If MAX(M1, M2,..., M g ,...,M​​​​h ) is greater than or equal to M0, it indicates that the to-be-identified person is the first person corresponding to the first matching degree, and the first person is determined as the target person; otherwise, if MAX(M1, M2,..., M g ,...,M h ) is less than M0, it indicates that the to-be-identified person is a person other than the first person, and only the target appearance feature information needs to be displayed.

[0081] S450, the first identity information of the first person corresponding to MAX(M1, M2,..., M g ,...,M h ) is determined as the target identity information.

[0082] S460, the to-be-identified appearance feature information is determined as the target appearance feature information.

[0083] S500, a plurality of target appearance feature information and target identity information corresponding to the target person are labeled in the first identification frame corresponding to each first to-be-detected image, to obtain a plurality of target images.

[0084] Labeling the target appearance feature information and the target identity information in the first identification frame corresponding to the first to-be-detected image can provide intuitive visual information, make the person identification result more clear and visualized, and facilitate further analysis and application.

[0085] Wherein, the first identification frame is determined by the following steps:

[0086] S510, according to the bounding box algorithm, the human body region in each first to-be-detected image is framed, to obtain the initial identification frame corresponding to each first to-be-detected image.

[0087] The bounding box algorithm is a method based on sliding window or region proposal, which determines the initial identification frame in each first to-be-detected image.

[0088] S520, the field length of the plurality of target appearance feature information corresponding to the target person in each first to-be-detected image is obtained, to obtain a field length set F=(F1, F2,..., F j ,...,F m ); F j =(F j1 ,F j2 ,...,F jd ,...,F jf(j) ); Wherein, j=1, 2,..., m; d=1, 2,..., f(j); m is the number of first to-be-detected images; f(j) is the number of target appearance feature information corresponding to the target person in the jth first to-be-detected image; F jA field length list of the target appearance feature information corresponding to the target person in the jth first to-be-detected image; F jd A field length of the dth target appearance feature information corresponding to the target person in the jth first to-be-detected image;

[0089] S530, acquire the field length G of the target identity information;

[0090] S540, if G≥MAX(F j ), then expand the boundary of the initial identification frame by G lengths, and determine it as the first identification frame corresponding to the jth first to-be-detected image; otherwise, expand the boundary of the initial identification frame by MAX(F j ) lengths, and determine it as the first identification frame corresponding to the jth first to-be-detected image;

[0091] Expand the initial identification frame outward by the longest field length in each first to-be-detected image as the expansion length, to obtain the corresponding first identification frame.

[0092] S550, if the boundary of the corresponding first to-be-detected image is contained in the first identification frame, then take the boundary as the boundary of the corresponding first identification frame;

[0093] If the boundary of the first to-be-detected image is located in the first identification frame, in order to prevent the information field from being unable to be completely displayed in the first to-be-detected image, the boundary of the first to-be-detected image having an intersection relationship is taken as the boundary of the corresponding first identification frame, and the boundary of the first identification frame which has no intersection with the boundary of the first to-be-detected image does not need to be changed.

[0094] S600, perform packaging processing on each target image to obtain a target video stream;

[0095] Further, step S600 comprises:

[0096] S610, encode m target images according to an audio-video encoding algorithm to obtain corresponding second video frames;

[0097] The audio-video encoding algorithm is, for example, H.264 or H.265.

[0098] S620, package each second video frame into a real-time rtmp stream to obtain a target video stream;

[0099] The target video stream is a video stream for real-time transmission, to meet the demand of real-time streaming.

[0100] S630, store each second video frame in an flv file format.

[0101] Each second video frame is stored in an flv file format, for subsequent data analysis, archiving and playback, etc.

[0102] S700, the target video stream is displayed in the target display interface.

[0103] The present application provides a kind of personnel feature extraction, identification and labeling method based on real-time video stream, by utilizing computer vision, machine learning and multimedia processing technology, personnel feature can be extracted and identified quickly and accurately in real-time video stream, realize real-time information acquisition and labeling, greatly improve processing speed and efficiency, and the first face feature vector obtained according to the personnel feature information to be identified is compared with the first face feature vector in the first database, the acquisition of the first identity information of the first personnel is realized, which provides a convenient and practical means for security monitoring and personnel management, and a plurality of target appearance feature information and target identity information corresponding to target personnel are labeled on the first to be detected image, which can provide intuitive visual information, make personnel identification result more explicit and visualized, facilitate further analysis and application, while generating new real-time rtmp stream and flv file, so that the processed video stream can be conveniently transmitted and saved in real time, which provides convenience for subsequent data analysis, archiving and playback, and meets the needs of security monitoring and personnel management field.

[0104] Embodiments of the present application also provide a non-transitory computer-readable storage medium, which can be arranged in an electronic device to save at least one instruction or at least one program related to a method in the method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to realize the method provided by the above-mentioned embodiments.

[0105] Embodiments of the present application also provide an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0106] Embodiments of the present application also provide a computer program product, which includes program code, when the program product is running on an electronic device, the program code is used to make the electronic device execute the steps in the method according to various exemplary embodiments of the present application described in the specification.

[0107] In addition, although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0108] Those skilled in the art can clearly understand, through the description of the above embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0109] In the example embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0110] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied in the form of a complete hardware, a complete software (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0111] The electronic device according to this embodiment of the present disclosure. The electronic device is only an example, and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0112] The electronic device is in the form of a general computing device. The components of the electronic device can include but are not limited to the above-mentioned at least one processor, the above-mentioned at least one storage, and a bus connecting different system components (including storage and processor).

[0113] The storage stores program codes which can be executed by the processor, so that the processor executes the steps according to various example embodiments of the present disclosure described in the above "example method" section of the present specification.

[0114] The storage can include a readable medium in the form of a volatile storage, such as a random access memory (RAM) and / or a cache memory, and can further include a read-only memory (ROM).

[0115] The storage can also include programs / utilities with a set of (at least one) program modules, such as operating systems, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0116] The bus can be one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures.

[0117] The electronic device can also communicate with one or more external devices such as a keyboard or a pointing device, through an I / O interface. The I / O interface can also include devices such as a Bluetooth device, a universal serial bus (USB) device, a serial device, a parallel device, or a game port. The electronic device can communicate with one or more devices that enable a user to interact with the electronic device through the I / O interface. The electronic device can also include a communication interface that can enable the electronic device to communicate with one or more other electronic devices. The communication interface can include a modem, a network interface card, a communication port, or a wireless communication device, among other possibilities. The electronic device can communicate with one or more networks, such as a local area network (LAN), a general area network (GAN), a wide area network (WAN), or the Internet, among other possibilities, through the communication interface. The communication interface can include logic encoded in software and / or hardware in a dedicated processing device for enabling communications between the electronic device and one or more networks. In some embodiments, the communication interface might manage a variety of communications protocols or components used in the

[0118] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB, a mobile hard disk, or the like) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0119] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which stores a program product capable of implementing the method described above. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps described in the “example method” section of the present disclosure according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0120] The program product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] The computer-readable signal medium can include a computer-readable storage medium that is communicated, propagated, or transported, for example, over a communication link, a wireless link, or a hard-wired link. The computer-readable signal medium can also be a computer-readable storage medium that is embodied into a computer-readable storage medium or used to manufacture a computer-readable storage medium. The computer-readable storage medium can be any appropriate medium (including the one or more computer-readable media described above) that participates in providing instructions to an instruction execution system, apparatus, or device such that the instructions, which properly render the instruction-execution system, apparatus, or device into a

[0122] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0123] The program code can be executed by one or more programmable processors, which can be individual or grouped processors, to perform the methods described above and illustrated in the flow charts. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). These network connections are

[0124] In addition, the above-described flowcharts are merely illustrative of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to limit the purpose. It is easily understood that the processes shown in the above-described flowcharts do not indicate or limit the time sequence of the processes. In addition, it is easily understood that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0125] It should be noted that, although several modules or units of the devices for action execution are mentioned in the foregoing detailed description, such division into modules or units is not mandatory. Indeed, according to an embodiment of the present disclosure, features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, features and functions of one module or unit described above can be further divided into plural modules or units.

[0126] The above merely shows the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed by the present application can be easily conceived by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for people identification based on real-time video streams, characterized in that, The system is applied to a personnel identification system, which is connected to a first database and a target display interface. The first database stores first facial feature vectors and first identity information corresponding to several first persons. The method includes the following steps: S100: Real-time acquisition of the first video stream corresponding to the person to be identified within the first target area; S200: Preprocess the first video stream to obtain several first images to be detected; S300. Extract feature information from each of the first images to be detected to obtain several feature information of the person to be identified; the feature information of the person to be identified includes the facial feature information and the appearance feature information of the person to be identified. S400. If the first database stores the first identity information corresponding to the facial feature information to be identified, then the person to be identified is identified as the target person. S500: Mark several target appearance feature information and target identity information corresponding to the target personnel in the first identification box corresponding to each first image to be detected, to obtain several target images; S600: Encapsulate each target image to obtain a target video stream; S700: Display the target video stream in the target display interface; The first identifier frame is determined through the following steps: S510. According to the bounding box algorithm, the human body region in each of the first images to be detected is selected to obtain the initial label box corresponding to each of the first images to be detected. S520. Obtain the field lengths of several target appearance feature information corresponding to the target person in each of the first images to be detected, and obtain the field length set F=(F1,F2,...,F...). j ,...,F m );F j =(F j1 ,F j2 ,...,F jd ,...,F jf(j) ); where j=1,2,...,m; d=1,2,...,f(j); m is the number of the first images to be detected; f(j) is the number of target appearance feature information corresponding to the target person in the j-th first image to be detected; F j F is a list of field lengths for the target person's appearance feature information in the j-th first image to be detected; jd The length of the field of the target person's appearance feature information corresponding to the dth target in the jth first image to be detected; S530. Obtain the field length G of the target identity information; S540, If G≥MAX(F) j If the initial bounding box is G units long, then the boundary of the initial bounding box is expanded outward by G units, and this expansion is determined as the first bounding box corresponding to the j-th first image to be detected; otherwise, the boundary of the initial bounding box is expanded outward by MAX(F). j ) lengths, and determine them as the first identifier box corresponding to the j-th first image to be detected; S550. If the first identifier box contains the boundary of the corresponding first image to be detected, then the boundary is taken as the boundary of the corresponding first identifier box.

2. The method according to claim 1, characterized in that, Step S200 includes: S210. Decode the first video stream to obtain a number of first video frames; S220. Perform frame extraction processing on each of the first video frames to obtain several first images to be detected.

3. The method according to claim 1, characterized in that, Step S300 further includes: S301. According to the machine learning algorithm, classify the feature information of each person to be identified to obtain several groups of feature information to be identified, and each group of feature information to be identified includes several feature information of the person to be identified.

4. The method according to claim 1, characterized in that, The facial feature information to be identified is determined through the following steps: S310. Determine the face region in each of the first images to be detected according to the face detection algorithm; S320. Based on the facial key point localization algorithm, detect the facial region in each of the first images to be detected, and determine the key feature information points in each facial region. S330. Based on the facial feature information extraction algorithm and each of the key feature information points, extract the facial feature information to be identified in each of the first images to be detected.

5. The method according to claim 4, characterized in that, Step S400 includes: S410. Based on the n facial feature information to be identified, obtain the facial feature information vector Q=(Q1,Q2,...,Q1) corresponding to the person to be identified. i ,...,Q n ); where i = 1, 2, ..., n; Q i This refers to the facial feature information of the i-th person to be identified. S420. Obtain the first facial feature vectors R1, R2, ..., R corresponding to the h first persons stored in the first database. g ,...,R h ;R g =(R g1 ,R g2 ,...,R gi ,...,R gn ); where g = 1, 2, ..., h; R g R is the first face feature vector corresponding to the g-th first person; gi R represents the i-th first face feature information in the first face feature vector corresponding to the g-th first person; gi The corresponding key feature information points and Q i The corresponding key feature information points are the same; S430, Connect Q to R1,R2,...,R g ,...,R h The matching degree is calculated separately to obtain the corresponding first matching degree M1, M2, ..., M g ,...,M h ; S440, If MAX(M1,M2,...,M... g ,...,M h If the match rate is greater than or equal to M0, then the person to be identified is determined as the target person; where M0 is a preset matching threshold; and MAX() is a preset maximum value determination function. S450, set MAX(M1,M2,...,M) g ,...,M h The first identity information of the first person corresponding to the target identity information is determined. S460. The facial feature information to be identified is determined as the target facial feature information.

6. The method according to claim 1, characterized in that, Step S600 includes: S610. According to the audio and video coding algorithm, the m target images are encoded to obtain the corresponding second video frames; S620. Encapsulate each of the second video frames into a real-time RTMP stream to obtain the target video stream; S630. Store each of the second video frames in FLV file format.

7. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method as described in any one of claims 1-6.

8. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 7.

Images