A visitor identification method based on face recognition, a storage medium and an electronic device

Abstract

The present application relates to the technical field of intelligent security, in particular to a visitor identification method based on face recognition, a storage medium and an electronic device. The method comprises: processing a first video stream to generate corresponding first characteristic values; processing a second video stream to generate second characteristic values of leaving visitors; calculating the similarity between the second characteristic values and the generated first characteristic values; and if the similarity between the second characteristic values and the generated first characteristic values is greater than a first similarity threshold, determining that the leaving visitor and the corresponding entering visitor are the same visitor. The present application compares the similarity between the leaving visitor and the entered visitor, and then matches the leaving visitor with the entered visitor according to the similarity. If the matching is successful, the visitor can be released; if the matching fails, an alarm is given. Thus, it can be ensured that the leaving visitor is necessarily one of the entered visitors, and the problem of mistaken entry and exit of visitors can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent security and protection, and in particular to a visitor identification method based on face recognition, a storage medium and an electronic device. BACKGROUND

[0002] In the current field of security monitoring and personnel management, there is an increasing demand for real-time video stream processing and personnel feature identification. Traditional security monitoring systems usually rely on manual operation and analysis, which is time-consuming and prone to errors. With the development of computer vision technology, some access control systems based on face recognition technology have also emerged, which are used to check the access rights of visitors and release or intercept them according to the access rights. However, the existing access control systems cannot effectively identify the wrong exit problems of a large number of visitors during the access process. SUMMARY

[0003] To solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0004] According to one aspect of the present application, a visitor identification method based on face recognition is provided, which comprises the following steps:

[0005] Obtaining a first video stream generated when an entering visitor is located in a first target area, the first video stream including a plurality of first to-be-detected images;

[0006] Performing visitor feature processing on each frame of the first to-be-detected image in the first video stream to generate a first feature value corresponding to the entering visitor;

[0007] Obtaining a second video stream generated when a leaving visitor is located in a second target area, the second video stream including a plurality of second to-be-detected images;

[0008] Performing visitor feature processing on each frame of the second to-be-detected image in the second video stream to generate a second feature value corresponding to the leaving visitor;

[0009] Calculating the similarity between the second feature value corresponding to the leaving visitor and each first feature value that has been generated;

[0010] If the similarity between the second feature value corresponding to the leaving visitor and any first feature value that has been generated is greater than a first similarity threshold, it is determined that the leaving visitor and the entering visitor corresponding to any first feature value are the same visitor;

[0011] The visitor feature processing includes:

[0012] Inputting the to-be-detected image into the SCRFD model to identify whether there is a face image in the to-be-detected image; the to-be-detected image includes the first to-be-detected image and the second to-be-detected image;

[0013] If there is a face image in the to-be-detected image, the to-be-detected image is input into the RepVGG model, the IResNet model and the DeepSORT model respectively, to generate a face image corresponding positive position parameter, face feature value and face tracking identifier;

[0014] The RepVGG model is used to generate the face position parameter of the face image, and the face position parameter is used to represent the closeness between the face image and the target front face image; the IResNet model is used to generate the face feature value of the face image; each face image corresponds to a unique face feature value; the DeepSORT model is used to generate the face tracking identifier of the face image; the face images of the same visitor correspond to the same face tracking identifier;

[0015] From all the to-be-detected images corresponding to the same video stream, the target feature value corresponding to each visitor is screened out, and the target feature value is the face feature value corresponding to the maximum positive position parameter in the multiple face feature values corresponding to the same face tracking identifier.

[0016] Further, after generating the first feature value corresponding to the entering visitor, the method further comprises:

[0017] configuring an entering label for the first feature value;

[0018] storing the first feature value and the corresponding entering label in the entering visitor database.

[0019] Further, after generating the second feature value corresponding to the leaving visitor, the method further comprises:

[0020] configuring a leaving label for the second feature value;

[0021] storing the second feature value and the corresponding leaving label in the leaving visitor database.

[0022] Further, after storing the second feature value and the corresponding leaving label in the leaving visitor database, the method further comprises:

[0023] when the preset time is reached, calculating the similarity between each first feature value already in the entering visitor database and each second feature value already in the leaving visitor database, to generate a similarity sequence corresponding to each first feature value and second feature value; wherein, A J i the similarity sequence corresponding to the ith first feature value already in the entering visitor database is A J i =(A J i1 ,A J i2 ,…,A J in ,…,A Jif(L) ), i = 1, 2, …, f(J), f(J) is the total number of the first feature values already in the visitor database; A J in is the similarity between the i-th first feature value already in the visitor database and the n-th second feature value already in the visitor database, n = 1, 2, …, f(L), f(L) is the total number of the second feature values already in the visitor database;

[0024] For each similarity sequence corresponding to the first feature value and the second feature value, respectively, an anomaly detection is performed, and if Max(A J i1 , A J i2 , …, A J in , …, A J if(L) ) < Y1; then A J i the corresponding first feature value is configured with an anomaly label; wherein Y1 is a first similarity threshold.

[0025] Further, after screening out the target feature value corresponding to each visitor from all the to-be-detected images corresponding to the same video stream, the visitor feature processing further includes:

[0026] using a Base64 encryption algorithm to encrypt the target feature value to generate a first encrypted feature value.

[0027] Further, before calculating the similarity between the second feature value corresponding to the leaving visitor and each first feature value already generated, the method further includes:

[0028] using a Base64 decryption algorithm to decrypt the first encrypted feature value to generate a first decrypted feature value corresponding to the first encrypted feature value, the first decrypted feature value being a binary string with a fixed length.

[0029] Further, the first feature value and the second feature value are both binary strings with a fixed length;

[0030] calculating the similarity between the second feature value corresponding to the leaving visitor and each first feature value already generated, including:

[0031] converting the second feature value to be used for similarity calculation into a corresponding second feature vector;

[0032] converting the first feature value to be used for similarity calculation into a corresponding first feature vector; the first feature vector and the second feature vector have the same length;

[0033] calculate a cosine value Cosβ between the second feature vector and the first feature vector using cosine similarity;

[0034] perform similarity conversion processing on Cosβ to generate similarity Pmeβ between the corresponding second feature value and the corresponding first feature value; Pmeβ∈[0, 1].

[0035] Further, Pmeβ satisfies the following condition:

[0036] Pmeβ=(Cosβ+1)*50%.

[0037] According to a second aspect of the present application, a non-transitory computer readable storage medium is provided, which stores a computer program, and the computer program, when executed by a processor, implements the above-mentioned visitor identification method based on face recognition.

[0038] According to a third aspect of the present application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor, when executing the computer program, implements the above-mentioned visitor identification method based on face recognition.

[0039] The present application has at least the following beneficial effects:

[0040] In the present application, when a visitor enters a target area, a first video stream generated when the visitor is in the first target area is acquired, and visitor feature processing is performed on the first video stream to generate a first feature value that most accurately represents the face features of the visitor to be entered, and is saved. Similarly, when a visitor leaves the target area, a corresponding second video stream of the visitor is acquired and visitor feature processing is performed, and a second feature value that most accurately represents the face features of the visitor to be left is also generated. Then, similarity calculation is performed between the second feature value and each existing first feature value, and when the similarity is greater than a first similarity threshold, the visitor to be left is determined to be one of the entered visitors. By comparing the similarity between the leaving visitor and the entered visitor, the leaving visitor and the entered visitor can be matched through similarity, and if the matching is successful, the visitor can be released, and if the matching fails, an alarm is given. Therefore, it can be ensured that the visitor who can leave is necessarily one of the entered visitors, and thus the problem of mistaken entry and exit of the visitor during entry and exit can be avoided.

[0041] In addition, the visitor feature processing is performed on each frame of the to-be-detected image in the video stream, and then the face feature value generated from the to-be-detected image in which the same person's face is closest to the frontal face state is taken as the corresponding feature value of the visitor. Therefore, the feature value that most accurately represents the face features of the visitor can be selected, and the accuracy of subsequent similarity calculation is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only show some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0043] Figure 1 The flow chart of the visitor identification method based on face recognition provided by an embodiment of the present application.

[0044] Figure 2 The schematic diagram of the video stream acquisition of the first target area and the second target area provided by another embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the present application.

[0046] As a possible embodiment of the present application, as shown in Figure 1 a visitor identification method based on face recognition is provided, which comprises the following steps:

[0047] S100: acquiring a first video stream generated when an entering visitor is located in a first target area, the first video stream comprising a plurality of first to-be-detected images.

[0048] S200: performing visitor feature processing on each first to-be-detected image in the first video stream to generate a first feature value corresponding to the entering visitor.

[0049] In actual use, in order to reduce the calculation amount of the model in the visitor feature processing, a target selection frame is configured in advance to intercept the region of interest of each to-be-detected image. Since the relative positions of the camera and the shooting area in the present embodiment are fixed, the background part in the image acquired by the camera is fixed and unchanged, and the relative position of the visitor image in the background will change. In this case, there is a part of the image information in the to-be-detected image (the region outside the target selection frame) which is invalid information for subsequent calculation, and thus can be removed to reduce the calculation amount.

[0050] S300: acquiring a second video stream generated when a leaving visitor is located in a second target area, the second video stream comprising a plurality of second to-be-detected images.

[0051] S400: Perform visitor feature processing on each frame of the second image to be detected in the second video stream to generate the second feature value corresponding to the departing visitor.

[0052] like Figure 2 As shown, there are usually corresponding entrances and exits when entering or exiting a certain area, and corresponding surveillance cameras 23 are installed at the entrances and exits, such as in a security access control system. These surveillance cameras 23 can acquire video information of a preset area in the entry / exit channel. Specifically, in this embodiment, the first target area is a preset area in the entry channel that the surveillance camera 23 can capture; similarly, the second target area is a preset area in the exit channel that the surveillance camera 23 can capture. The first and second target areas can be set according to the actual usage scenario.

[0053] Therefore, when a visitor passes through the first or second target area, the corresponding camera can capture the corresponding video stream.

[0054] Then, visitor feature processing is performed on the corresponding video streams. Since the video stream contains multiple frames of images to be detected, the visitor feature processing will use the facial feature value generated from the image of the same face that is closest to a frontal view as the feature value corresponding to that visitor. In this way, the feature value that most accurately represents the visitor's facial features can be selected, thus ensuring the accuracy of subsequent similarity calculations.

[0055] Visitor profile processing includes:

[0056] S201: Input the image to be detected into the SCRFD model to identify whether there is a face image in the image to be detected. The image to be detected includes a first image to be detected and a second image to be detected.

[0057] In order to capture a more comprehensive image of the visitor, the video stream is usually started with a certain amount of advance time. During this advance time, the visitor has not yet entered the shooting range, which will result in the absence of the visitor's image information in the image at the head position in the video stream.

[0058] In this step, the SCRFD model can be used to identify whether there are human faces in the image to be detected, and then the images to be detected without human faces can be removed, further reducing the amount of computation in subsequent processing.

[0059] The SCRFD (Suppression and Context Reasoning for Fast and Accurate Face Detection) model is a deep learning algorithm model for face detection. It can achieve fast and accurate face detection.

[0060] The design goal of SCRFD is to provide faster inference speed while maintaining high detection accuracy. To achieve this goal, SCRFD adopts a series of optimization strategies and network structure designs.

[0061] Firstly, SCRFD uses a multi-scale detection strategy based on a feature pyramid network to detect faces at different scales. This allows the algorithm to have better robustness and accuracy when dealing with faces of different sizes.

[0062] Secondly, SCRFD introduces an operation called "Suppression Pooling" for non-maximum suppression (NMS) on feature maps to reduce the number of overlapping boxes. This operation can significantly improve detection speed and reduce the number of duplicate boxes, thereby improving detection accuracy.

[0063] In addition, SCRFD also introduces a context reasoning mechanism to enhance the robustness of face detection. By utilizing the context information in the feature map, the algorithm can better understand the environment and semantic information around the face, thereby improving the detection ability of faces in various scenes and complex backgrounds.

[0064] In summary, SCRFD is a deep learning algorithm that aims to achieve fast and accurate face detection. It improves the accuracy and efficiency of face detection through optimization strategies such as multi-scale detection, Suppression Pooling, and context reasoning. SCRFD has broad application potential in the fields of face recognition, face analysis, video surveillance, etc. Thus, it can quickly identify whether there is a face image in the image to be detected.

[0065] S202: If there is a face image in the image to be detected, input the image to be detected into the RepVGG model, the IResNet model and the DeepSORT model respectively, to generate the face image corresponding to the positive position parameter, the face feature value and the face tracking identifier.

[0066] The RepVGG model is used to generate the face position parameter of the face image, which is used to represent the closeness between the face image and the target front face image.

[0067] The frame-to-be-detected image is usually an image of a visitor's face obtained by a camera at different shooting angles. Different shooting angles can result in a large difference in the number and shape of the feature parts of the visitor's face in the obtained image. For example, if the image is a front-facing photo of the visitor, it includes all the feature information such as the left and right eyes, the left and right eyebrows, the nose, and the mouth. If the image is a side-facing photo of the visitor, it includes partial feature information such as the left or right eye, the left or right eyebrow, the nose, and the mouth.

[0068] Therefore, the feature values obtained by the same visitor at different shooting angles can have a large difference due to the difference in the feature parts of the face. This can affect the accuracy of the similarity calculation in the later stage.

[0069] The face alignment parameter generated in this step can be used to select a frame closest to a front-facing photo from multiple frame-to-be-detected images as the basis for calculating the feature value of the visitor. In this way, the difference between the face feature values obtained by the same visitor when entering and leaving can be reduced, and the accuracy of the similarity calculation in the later stage can be improved.

[0070] The RepVGG (Re-param VGG) model is a convolutional neural network model used for image classification and object detection tasks.

[0071] The design inspiration of RepVGG comes from VGGNet (Visual Geometry Group), which adopts a similar organization structure of convolutional layers, i.e., multiple consecutive convolutional layers followed by a pooling layer. However, RepVGG introduces an innovative idea of combining convolutional layers and pooling layers together to form a RepVGG Block.

[0072] A RepVGG Block is composed of two consecutive convolutional layers, whose output serves as both the output of the current layer and the input of the next layer. This design allows RepVGG to maintain similar accuracy as conventional convolutional networks while having higher computational efficiency and lower memory consumption. In addition, by using lighter 1x1 convolution in some convolutional layers, RepVGG can also reduce the number of model parameters.

[0073] The design of RepVGG is simple and intuitive, making it easy to understand and implement, and it has good performance and efficiency. It achieves accuracy comparable to more complex networks in image classification tasks while being more efficient in terms of computational resources and memory usage. RepVGG can also be used for object detection tasks by adding additional detection heads to the image classification network to realize the function of object detection.

[0074] In summary, RepVGG is a convolutional neural network architecture with a simple and efficient design, suitable for image classification and object detection tasks, providing a balance between accuracy and computational efficiency. Therefore, in this step, the face image can be classified into multiple face orientation classes by the RepVGG model. Specifically, from the full side face class to the full front face class, the corresponding positive parameter is from 0 to 1.

[0075] The IResNet model is used to generate face feature values for face images. Each face image corresponds to a unique face feature value.

[0076] The IResNet (Improved Residual Networks) model is a convolutional neural network model that improves and optimizes the classic ResNet (Residual Networks) model. ResNet aims to solve the degradation problem of deep neural networks.

[0077] IResNet makes some improvements on the basis of ResNet to further improve the performance and effect of the network.

[0078] Firstly, IResNet introduces a technique called "Identity Mapping". In ResNet, residual connections are added to solve the problem of gradient vanishing and expression capacity decline, but in some deep networks, residual connections may introduce certain errors. In order to reduce this error, IResNet uses identity mapping in the residual block, that is, the residual connection is directly used as an identity transformation, rather than adjusting through learned weights.

[0079] Secondly, IResNet adopts a more efficient convolution operation called "Inception Residual Module". This module combines the Inception structure and the residual connection of ResNet, which can reduce the number of parameters and computational complexity while maintaining high performance.

[0080] In addition, IResNet also introduces techniques such as "Channel Split" and "Residual Scale" to enhance the feature representation of the network and reduce information loss.

[0081] In summary, IResNet is a convolutional neural network structure that improves and optimizes the ResNet model. By introducing Identity Mapping, Inception Residual Module, and other optimization techniques, IResNet reduces the number of parameters and computational complexity while improving the performance and effectiveness of the network. The IResNet model can generate a corresponding facial feature value based on the information of each feature part included in the facial image.

[0082] DeepSORT is used to generate facial tracking labels for facial images. The same visitor's facial image corresponds to the same facial tracking label.

[0083] DeepSORT (Deep Association Network for Real-Time Multiple Object Tracking) is a deep learning algorithm model for real-time multi-object tracking, aiming to solve the problem of multi-object tracking in videos.

[0084] DeepSORT combines target detection and target association techniques, by combining deep learning target detection models (such as YOLO or Faster R-CNN) with classic target association methods (such as Kalman filtering and Hungarian algorithm), it realizes accurate and real-time multi-object tracking.

[0085] The key idea of DeepSORT is to use a convolutional neural network (CNN) for target detection, and then use appearance features and motion information to associate and track the detected targets. Specifically, DeepSORT extracts the appearance features (such as deep features or artificially designed features) and motion information (such as speed and acceleration) of the target to describe the target, and uses these information to associate and track the target.

[0086] DeepSORT also introduces a technique called "Embedding", which represents each target as a unique vector in an embedding space, so that the similarity between targets can be measured by calculating the distance between embedding vectors. This way, embedding vectors can be used to measure the similarity between targets during the association process, improving the accuracy and robustness of tracking.

[0087] In summary, DeepSORT is an algorithm that combines deep learning target detection and target association for real-time multi-object tracking. It uses appearance features, motion information, and embedding techniques to achieve accurate and efficient target association and tracking based on target detection.

[0088] Therefore, at least one visitor existing in a video stream can be continuously tracked. Specifically, the same person face tracking identifier can be assigned to the same visitor in different image frames. Further, it can be ensured that the selection of the target feature value corresponding to each visitor is further avoided in the later screening.

[0089] S203: From all the to-be-detected images corresponding to the same video stream, the target feature value corresponding to each visitor is screened out, and the target feature value is the face feature value corresponding to the maximum positive position parameter in the face feature values corresponding to the same person face tracking identifier.

[0090] Specifically, each feature value obtained by each visitor finally includes the same person face tracking identifier, a corresponding face feature value, and a corresponding positive position parameter. Finally, the face feature value corresponding to the maximum positive position parameter is selected as the target feature value corresponding to the visitor.

[0091] Further, after S203: From all the to-be-detected images corresponding to the same video stream, the target feature value corresponding to each visitor is screened out, the visitor feature processing further includes:

[0092] S204: The target feature value is encrypted using a Base64 encryption algorithm to generate a first encrypted feature value.

[0093] Generally, in the actual use process, the face feature value generated from the face image belongs to information with high security requirements, so it needs to be encrypted to improve its security.

[0094] S500: Calculate the similarity between the second feature value corresponding to the leaving visitor and each first feature value generated.

[0095] Further, before S500: Calculate the similarity between the second feature value corresponding to the leaving visitor and each first feature value generated, the method further includes:

[0096] S410: The first encrypted feature value is decrypted using a Base64 decryption algorithm to generate a first decrypted feature value corresponding to the first encrypted feature value, and the first decrypted feature value is a binary string with a fixed length. The first feature value and the second feature value are both binary strings with a fixed length.

[0097] The encryption and decryption algorithms of Base64 in this embodiment are all existing algorithms, which will not be described here. After this step, each feature value will be converted into a binary string with a fixed length, such as an 8-bit binary string: 01111001.

[0098] S500: Calculate the similarity between the second feature value corresponding to the leaving visitor and each first feature value generated, including:

[0099] S501: Convert the second feature value to be calculated into a corresponding second feature vector.

[0100] S502: Convert the first feature value to be calculated into a corresponding first feature vector. The length of the first feature vector is the same as that of the second feature vector.

[0101] Specifically, each character in the binary string is taken as an element value in the corresponding dimension of the feature vector, i.e., the binary string can be converted into a corresponding feature vector. For example, the binary string 01111001 is converted into the feature vector (0, 1, 1, 1, 1, 0, 0, 1).

[0102] S503: Calculate the cosine value Cosβ between the second feature vector and the first feature vector using the cosine similarity.

[0103] After vector conversion, the cosine similarity can be used to calculate the cosine value between any two vectors, i.e., the degree of similarity between the directions of the vectors. The cosine similarity value ranges from -1 to 1, where 1 indicates complete similarity, -1 indicates complete opposition, and 0 indicates no correlation.

[0104] S504: Perform similarity conversion processing on Cosβ to generate the similarity Pmeβ between the corresponding second feature value and the corresponding first feature value. Pmeβ∈[0, 1].

[0105] Specifically, Pmeβ satisfies the following conditions:

[0106] Pmeβ=(Cosβ+1)*50%.

[0107] This step is a method for converting the cosine value of the face feature value into a similarity percentage. Specifically, the similarity value is mapped to a percentage range of 0%-100% through linear mapping. The specific steps are as follows:

[0108] First, (Cosβ+1) is used to convert the cosine value from -1 to 1 to 0 to 2, i.e., the similarity value is shifted by 1 unit. Then, the mapped cosine value is multiplied by 50 to make its value range between 0 and 100. In this way, the cosine similarity can be converted into a percentage, where 0 indicates complete dissimilarity and 100 indicates complete similarity.

[0109] This conversion method is applicable to the case where the cosine similarity value ranges from -1 to 1. If other similarity measurement methods are used, their value ranges are different, and the values of the conversion parameters need to be adjusted accordingly.

[0110] S600: If the similarity between the second feature value corresponding to the leaving visitor and any of the generated first feature values is greater than the first similarity threshold, it is determined that the leaving visitor is the same visitor as the entering visitor corresponding to any of the first feature values.

[0111] Specifically, the first similarity threshold Y1 can be set according to the actual use scene, such as Y1 can be 80%. In the embodiment, if the similarity between the second feature value corresponding to the leaving visitor and any of the generated first feature values is less than the first similarity threshold, the alarm information corresponding to the leaving visitor is generated.

[0112] In the present application, when the visitor enters the target area, the first video stream generated when the visitor is in the first target area is acquired, and the visitor feature processing is performed on the first video stream to generate the first feature value that can most accurately represent the facial features of the entering visitor, and is saved. Similarly, when the visitor leaves the target area, the second video stream corresponding to the visitor is acquired and the visitor feature processing is performed, and a second feature value that can most accurately represent the facial features of the leaving visitor is also generated. Then the similarity between the second feature value and each of the existing first feature values is calculated, and when the similarity is greater than the first similarity threshold, it is determined that the leaving visitor is one of the entered visitors. By comparing the similarity between the leaving visitor and the entered visitor, the leaving visitor and the entered visitor can be matched through the similarity, and if the matching is successful, the visitor can be released, and if the matching fails, an alarm is given. Therefore, it can be ensured that the leaving visitor must be one of the entered visitors, thereby avoiding the mistake of the visitor in the process of entering and leaving. Therefore, the management accuracy of the visitor can be improved, and the situation of the entered visitor and the internal personnel of the target area being exchanged can be further prevented.

[0113] As another embodiment of the present application, after the first feature value corresponding to the entering visitor is generated, the method further comprises:

[0114] S210: configuring an entering label for the first feature value.

[0115] S220: storing the first feature value and the corresponding entering label in the entering visitor database.

[0116] Similarly, after the second feature value corresponding to the leaving visitor is generated, the method further comprises:

[0117] S230: configuring a leaving label for the second feature value.

[0118] S240: storing the second feature value and the corresponding leaving label in the leaving visitor database.

[0119] Further, after S240: storing the second feature value and the corresponding leaving label in the leaving visitor database, the method further comprises:

[0120] S250: When the preset time is reached, the similarity between each first feature value already existing in the visitor database and each second feature value already existing in the visitor database is calculated, and a similarity sequence corresponding to each first feature value and second feature value is generated. Wherein, A J i is the similarity sequence corresponding to the ith first feature value already existing in the visitor database, i = 1, 2, …, f(J), and f(J) is the total number of first feature values already existing in the visitor database. J i J i1 , A J i2 , …, A J in , …, A J if(L) , i = 1, 2, …, f(J). J in is the similarity between the ith first feature value already existing in the visitor database and the nth second feature value already existing in the visitor database, n = 1, 2, …, f(L), and f(L) is the total number of second feature values already existing in the visitor database.

[0121] The preset time in the embodiment can be set according to the actual use scene, for example, it can be 6 o'clock in the afternoon every day.

[0122] S260: Abnormality detection is performed on the similarity sequence corresponding to each first feature value and second feature value respectively, and if Max(A J i1 , A J i2 , …, A J in , …, A J if(L) ) < Y2, then A J i corresponding first feature value is configured with an abnormality label. Wherein, Y2 is a second similarity threshold. Specifically, Y2 < Y1, and Y2 can be 40%.

[0123] When Max(A J i1 , A J i2 , …, A J in , …, A J if(L) ) < Y2, it can be considered that A J i ​The first feature value corresponding to the entering visitor corresponds to a visitor entering the target area within a specified time and not leaving the target area within the specified time. Similarly, when the maximum similarity value in the similarity sequence corresponding to the second feature value is less than Y2, it can be considered that the second feature value corresponds to a leaving visitor, and the visitor is not a visitor entering the target area within a specified time. The visitors in the above two cases are abnormal visitors, so the corresponding feature values are labeled as abnormal and marked as abnormal.

[0124] In the embodiment, if the preset time is the same time every day, the feature values of all visitors entering the target area and the feature values of all visitors leaving the target area stored in the entering visitor database and the leaving visitor database are within a time period of one day. Therefore, the embodiment can identify the situation that the number of visitors entering and leaving the target area is more or less. Furthermore, the management accuracy of the visitors can be improved, and the situation that the entering visitors are exchanged with the internal personnel of the target area can be further prevented.

[0125] The embodiment of the present application also provides a non-transitory computer readable storage medium, which can be arranged in an electronic device to save at least one instruction or at least one program related to a method in the method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to realize the method provided by the above embodiment.

[0126] The embodiment of the present application also provides an electronic device, which comprises a processor and the aforementioned non-transitory computer readable storage medium.

[0127] The embodiment of the present application also provides a computer program product, which comprises program code, and the program code is used to make the electronic device execute the steps in the method according to various exemplary embodiments of the present application described in the specification when the program product is running on the electronic device.

[0128] In addition, although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0129] Those skilled in the art can clearly understand, through the description of the above embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0130] In the example embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0131] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied in the form of a complete hardware, a complete software (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0132] The electronic device according to this embodiment of the present disclosure. The electronic device is only an example, and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0133] The electronic device is in the form of a general computing device. The components of the electronic device can include but are not limited to the above-mentioned at least one processor, the above-mentioned at least one storage, and a bus connecting different system components (including storage and processor).

[0134] The storage stores program codes which can be executed by the processor, so that the processor executes the steps according to various example embodiments of the present disclosure described in the above "example method" section of the present specification.

[0135] The storage can include a readable medium in the form of a volatile storage, such as a random access memory (RAM) and / or a cache storage, and can further include a read-only memory (ROM).

[0136] The storage can also include programs / utilities with a set of (at least one) program modules, such as operating systems, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0137] The bus can be one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures.

[0138] The electronic device can also communicate with one or more external devices such as a keyboard or a pointing device, through an I / O interface. The I / O interface can also include devices such as a Bluetooth device, a universal serial bus (USB) device, a serial device, a parallel device, or a game port. The electronic device can communicate with one or more devices that enable a user to interact with the electronic device through the I / O interface. The electronic device can also include a communication interface that can enable the electronic device to communicate with one or more other electronic devices. The communication interface can include a modem, a network interface card, a communication port, or a wireless communication device, among other possibilities. The electronic device can communicate with one or more networks, such as a local area network (LAN), a general area network (GAN), a wide area network (WAN), or the Internet, among other possibilities, through the communication interface. The communication interface can include logic encoded in software and / or hardware in a dedicated processing device for enabling communications between the electronic device and one or more networks. In some embodiments, the communication interface might manage a variety of communications protocols or components. In some embodiments, the communication interface can communicate with the input / output interface so as to enable communication between the electronic device and one or more external devices.

[0139] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0140] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which stores a program product capable of implementing the above-mentioned method of the present disclosure. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps according to various example embodiments of the present disclosure described in the above-mentioned "example method" section of the present specification when the program product is run on the terminal device.

[0141] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0142] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0143] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0144] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0145] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0146] It should be noted that, although several modules or units of the devices for action execution are mentioned in the above detailed description, the division into such modules or units is not mandatory. Indeed, according to an embodiment of the present disclosure, the features and functionalities of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functionalities of one of the above-described modules or units can be further divided into several modules or units.

[0147] The above merely shows the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed by the present application can be easily conceived by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A visitor identification method based on face recognition, characterized in that, The method comprises the following steps: Obtaining a first video stream generated when an entering visitor is located in a first target area, the first video stream comprising a plurality of frames of first to-be-detected images; Performing visitor feature processing on each frame of the first to-be-detected images in the first video stream to generate a first feature value corresponding to the entering visitor; Obtaining a second video stream generated when a leaving visitor is located in a second target area, the second video stream comprising a plurality of frames of second to-be-detected images; Performing visitor feature processing on each frame of the second to-be-detected images in the second video stream to generate a second feature value corresponding to the leaving visitor; Calculating the similarity between the second feature value corresponding to the leaving visitor and each first feature value that has been generated; If the similarity between the second feature value corresponding to the leaving visitor and any first feature value that has been generated is greater than a first similarity threshold, determining that the leaving visitor and the entering visitor corresponding to the first feature value are the same visitor; The visitor feature processing comprises: Inputting a to-be-detected image into an SCRFD model to identify whether there is a face image in the to-be-detected image; the to-be-detected image comprises the first to-be-detected image and the second to-be-detected image; If there is a face image in the to-be-detected image, inputting the to-be-detected image into a RepVGG model, an IResNet model and a DeepSORT model respectively to generate a positive position parameter, a face feature value and a face tracking identifier corresponding to the face image; The RepVGG model is used to generate a face position parameter of the face image, and the face position parameter is used to represent the closeness between the face image and a target front face image; the IResNet model is used to generate a face feature value of the face image; each face image corresponds to a unique face feature value; The DeepSORT model is used to generate a face tracking identifier of the face image; face images of the same visitor correspond to the same face tracking identifier; From all to-be-detected images corresponding to the same video stream, a target feature value corresponding to each visitor is screened out, the target feature value being a face feature value corresponding to a maximum face position parameter in a plurality of face feature values corresponding to the same face tracking identifier.

2. The method of claim 1, wherein, After generating the first feature value corresponding to the entering visitor, the method further comprises: Configuring an entering label for the first feature value; Storing the first feature value and the corresponding entering label in an entering visitor database.

3. The method of claim 2, wherein, After generating the second feature value corresponding to the leaving visitor, the method further comprises: Configuring a leaving label for the second feature value; Storing the second feature value and the corresponding leaving label in a leaving visitor database.

4. The method of claim 3, wherein, After storing the second feature value and the corresponding leaving label in the leaving visitor database, the method further comprises: When the preset time is reached, the similarity between each first feature value already existing in the entering visitor database and each second feature value already existing in the leaving visitor database is calculated respectively, and a similarity sequence corresponding to each first feature value and second feature value is generated; wherein A J i is the similarity sequence corresponding to the i th first feature value already existing in the entering visitor database, A J i = (A J i1 , A J i2 , …, A J in , …, A J if(L) , i = 1, 2, …, f (J), f (J) is the total number of first feature values already existing in the entering visitor database; A J in is the similarity between the i th first feature value already existing in the entering visitor database and the n th second feature value already existing in the leaving visitor database, n = 1, 2, …, f (L), f (L) is the total number of second feature values already existing in the leaving visitor database; Anomaly detection is performed separately on the similarity sequences corresponding to each first eigenvalue and second eigenvalue. If Max(A J i1 , A J i2 , …, A J in , …, A J if(L) ) < Y2; then an anomaly label is assigned to the first eigenvalue corresponding to A J i . Here, Y2 is the second similarity threshold.

5. The method of claim 1, wherein, After screening out the target feature value corresponding to each visitor from all to-be-detected images corresponding to the same video stream, the visitor feature processing further comprises: Encrypting the target feature value using a Base64 encryption algorithm to generate a first encrypted feature value.

6. The method of claim 5, wherein, Before calculating the similarity between the second feature value corresponding to the leaving visitor and each first feature value that has been generated, the method further comprises: The first encrypted feature value is decrypted using a Base64 decryption algorithm to generate a first decrypted feature value corresponding to the first encrypted feature value, the first decrypted feature value being a binary string with a fixed length.

7. The method of claim 6, wherein, The first feature value and the second feature value are both binary strings with a fixed length. The similarity between the second feature value corresponding to the leaving visitor and each generated first feature value is calculated, including: The second feature value to be calculated for similarity is converted into a corresponding second feature vector; The first feature value to be calculated for similarity is converted into a corresponding first feature vector; the first feature vector and the second feature vector have the same length; The cosine value Cosβ between the second feature vector and the first feature vector is calculated using the cosine similarity; The similarity Pmeβ between the corresponding second feature value and the corresponding first feature value is generated by performing similarity conversion processing on Cosβ; Pmeβ∈[0, 1].

8. The method of claim 7, wherein, Pmeβ satisfies the following condition: Pmeβ=(Cosβ+1)*50%. 9.A non-transitory computer-readable storage medium storing a computer program, the computer program comprising instructions configured to cause a processor to perform the method according to any one of claims 1 to 8. The computer program, when executed by a processor, implements a visitor identification method based on face recognition according to any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements a visitor identification method based on face recognition according to any one of claims 1 to 8.

Images