Intelligent security monitoring system and method based on face recognition
The intelligent security monitoring system uses liveness detection and advanced face recognition techniques to address face attacks, ensuring accurate identification and detection of illegal intrusions, thereby enhancing security monitoring reliability and safety.
Patent Information
- Application Number
- CN202410620518.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-20
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-05-20
AI Technical Summary
Security monitoring systems based on face recognition face challenges from face attacks using photos, videos, or 3D models, leading to unauthorized access and threats to people and property.
An intelligent security monitoring system and method that combines liveness detection using infrared imaging and RGB face recognition, employing MTCNN network models, channel-enhanced feature pyramids, and attention fusion processing to accurately identify illegal intrusions.
Enhances the reliability and safety of security monitoring by accurately distinguishing between live and non-live objects and identifying illegal intrusions, improving the accuracy and robustness of face recognition.
Smart Images

Figure CN118537791B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent security monitoring, and in particular to an intelligent security monitoring system and method based on face recognition. Background Art
[0002] Security monitoring systems are essential for protecting people and property from security threats. With the development of face recognition technology, security monitoring systems based on face recognition are becoming more and more common. Face recognition technology can accurately identify individuals even if they are wearing masks or hats, which helps to achieve real-time monitoring and ensure the security of sensitive areas, protecting people and property from security threats.
[0003] However, with the development of technology, security monitoring systems based on face recognition are facing more and more security challenges, especially the threat of face attacks. Face attacks refer to illegal intrusions by deceiving face recognition systems by using photos, videos or 3D models, which can lead to unauthorized access to protected areas, posing a serious threat to people and property.
[0004] Therefore, an optimized intelligent security monitoring system based on face recognition is desired, which can combine liveness detection and face recognition technology to more accurately identify illegal intrusions and ensure the security of the area. Summary of the invention
[0005] The present disclosure is made in view of the above problems. One object of the present disclosure is to provide an intelligent security monitoring system and method based on face recognition.
[0006] The embodiment of the present disclosure provides an intelligent security monitoring method based on face recognition, which includes:
[0007] Obtaining an infrared image and an RGB image of the face of the object to be identified, which are captured by an infrared camera and an RGB camera;
[0008] Based on the infrared face image, performing liveness detection to obtain a liveness detection result;
[0009] In response to the liveness detection result being a non-liveness, generating an intrusion warning prompt signal;
[0010] In response to the liveness detection result being liveness, performing face recognition based on the face RGB image to obtain a face recognition result; and
[0011] In response to the face recognition result that the object to be identified is an illegal object, a suspected intrusion warning prompt signal is generated.
[0012] For example, according to an intelligent security monitoring method based on face recognition according to an embodiment of the present disclosure, in response to the liveness detection result being liveness, face recognition is performed based on the face RGB image to obtain a face recognition result, including:
[0013] Performing face detection and positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map;
[0014] The face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector;
[0015] Performing attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhanced features; and
[0016] Based on the facial semantics to guide the shallow facial enhancement features, it is determined whether the object to be identified is an illegal object.
[0017] For example, according to the intelligent security monitoring method based on face recognition of an embodiment of the present disclosure, the face RGB image is subjected to face detection positioning and multi-level semantic capture to obtain a face shallow feature map and a face semantic feature map, including:
[0018] Inputting the face RGB image into the MTCNN network model to obtain a face region of interest image; and
[0019] The face region of interest image is passed through a channel-enhanced feature pyramid network to obtain the face shallow feature map and the face semantic feature map.
[0020] For example, according to the intelligent security monitoring method based on face recognition according to an embodiment of the present disclosure, the MTCNN network model includes P-Net, R-Net and O-Net.
[0021] For example, according to the intelligent security monitoring method based on face recognition of an embodiment of the present disclosure, the face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector, including:
[0022] The face semantic feature map is processed by the face semantic typical feature extraction network using the following face semantic typical feature extraction formula to obtain the typical face semantic feature vector; wherein the face semantic typical feature extraction formula is:
[0023]
[0024]
[0025]
[0026] Among them, v i (x) and v j (x) are respectively the i-th and j-th face semantic feature vectors in the sequence of face semantic feature vectors obtained by quantizing the feature matrices along the channel dimension in the face semantic feature graph, V k is the sequence of the facial semantic feature vectors, log represents the logarithmic function value with base 2, L is the length of each facial semantic feature vector, M is the number of vectors in the sequence of facial semantic feature vectors minus one, and D i is the feature value of each position in the local semantic difference feature vector of the face, mask(·) is the masking operation, is the eigenvalue of each position in the local semantic difference feature vector of the masked face, N s is the length of the local semantic difference feature vector of the masked face, exp(·) is an exponential operation, and V is the typical face semantic feature vector.
[0027] For example, according to the intelligent security monitoring method based on face recognition of an embodiment of the present disclosure, the typical face semantic feature vector and the face shallow feature map are subjected to attention fusion processing based on face semantic guidance to obtain face semantic guidance face shallow enhancement features, including:
[0028] Passing the typical face semantic feature vector through a fully connected layer to perform fully connected encoding on the typical face semantic feature vector to obtain a typical face semantic fully connected feature vector; and
[0029] The typical face semantic fully connected feature vector and the face shallow feature map are weightedly fused based on the channel dimension to obtain the face semantics guided face shallow enhancement feature map as the face semantics guided face shallow enhancement feature.
[0030] For example, according to an intelligent security monitoring method based on face recognition according to an embodiment of the present disclosure, determining whether an object to be identified is an illegal object based on the face semantics guiding the shallow face enhancement features includes:
[0031] The face semantics-guided shallow face enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, and the recognition result is used to indicate whether the object to be recognized is an illegal object.
[0032] The embodiment of the present disclosure also provides an intelligent security monitoring system based on face recognition, which includes:
[0033] An image acquisition module is used to acquire an infrared image and an RGB image of a face of an object to be identified, which are captured by an infrared camera and an RGB camera;
[0034] A liveness detection module, used for performing liveness detection based on the face infrared image to obtain a liveness detection result;
[0035] An intrusion warning prompt signal generating module, used for generating an intrusion warning prompt signal in response to the liveness detection result being a non-liveness;
[0036] a face recognition module, configured to, in response to the liveness detection result being liveness, perform face recognition based on the face RGB image to obtain a face recognition result; and
[0037] The suspected intrusion warning prompt module is used to generate a suspected intrusion warning prompt signal in response to the face recognition result that the object to be identified is an illegal object.
[0038] For example, according to the intelligent security monitoring system based on face recognition according to an embodiment of the present disclosure, the face recognition module is used to:
[0039] Performing face detection and positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map;
[0040] The face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector;
[0041] Performing attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhanced features; and
[0042] Based on the facial semantics to guide the shallow facial enhancement features, it is determined whether the object to be identified is an illegal object.
[0043] According to the intelligent security monitoring system and method based on face recognition of the embodiments of the present disclosure, which performs security monitoring by combining liveness detection and face recognition technology, it can more accurately identify and detect illegal intrusions, thereby improving the reliability and safety of security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings of the embodiments of the present disclosure. Obviously, the drawings described below only relate to some embodiments of the present disclosure, and are not intended to limit the present disclosure.
[0045] Figure 1 A schematic diagram of the application architecture of the intelligent security monitoring method based on face recognition in an embodiment of the present disclosure is shown;
[0046] Figure 2 A flow chart of an intelligent security monitoring method based on face recognition in an embodiment of the present disclosure is shown;
[0047] Figure 3 A flowchart of sub-step S540 of the intelligent security monitoring method based on face recognition in an embodiment of the present disclosure is shown;
[0048] Figure 4 A schematic diagram of the structure of an intelligent security monitoring system based on face recognition in an embodiment of the present disclosure is shown;
[0049] Figure 5 An application scenario diagram of the intelligent security monitoring method based on face recognition in an embodiment of the present disclosure is shown; and
[0050] Figure 6 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0051] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present disclosure.
[0052] The terms used in this specification are those general terms currently widely used in the art in consideration of the functions of the present disclosure, but these terms may vary according to the intention of a person of ordinary skill in the art, precedents, or new technologies in the art. In addition, specific terms may be selected by the applicant, and in this case, their detailed meanings will be described in the detailed description of the present disclosure. Therefore, the terms used in the specification should not be understood as simple names, but rather as an overall description based on the meaning of the terms and the present disclosure.
[0053] Although the present disclosure makes various references to certain modules in the system according to embodiments of the present disclosure, any number of different modules can be used and run on a user terminal and / or server. The modules are illustrative only, and different aspects of the systems and methods can use different modules.
[0054] Flowcharts are used in this disclosure to illustrate the operations performed by the system according to the embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0055] Figure 1 A schematic diagram of the application architecture of an intelligent security monitoring method based on face recognition in an embodiment of the present disclosure is shown, including a server 100 and a terminal device 200.
[0056] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or a virtual private network. In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.
[0057] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.
[0058] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In the embodiment of the present disclosure, the memory 120 may be used to store the program of the intelligent security monitoring method based on face recognition in the embodiment of the present disclosure.
[0059] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the intelligent security monitoring methods based on face recognition in the embodiments of the present disclosure according to the obtained program instructions.
[0060] In addition, the application architecture diagram in the embodiment of the present disclosure is intended to more clearly illustrate the technical solution in the embodiment of the present disclosure, and does not constitute a limitation on the technical solution provided in the embodiment of the present disclosure. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present disclosure is also applicable to similar problems.
[0061] The following is a non-restrictive description of the intelligent security monitoring method based on face recognition provided according to at least one embodiment of the present disclosure through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present disclosure.
[0062] In view of the above technical problems, in the technical solution of the present application, an intelligent security monitoring system based on face recognition is proposed, which includes the following steps: first, an infrared image of the face and an RGB image of the face of the object to be identified collected by an infrared camera and an RGB camera are obtained. It should be understood that since the intruder may use the face image and the three-dimensional model to perform face recognition, the liveness detection technology can be used to distinguish between live and non-live objects. For example, the infrared camera can capture infrared thermal imaging data, which can reveal the blood flow and heat distribution pattern of the face, which cannot be imitated by photos or three-dimensional models, thereby helping to distinguish between live and non-live objects. Based on this, liveness detection is further performed based on the infrared image of the face to obtain a liveness detection result. In particular, in response to the liveness detection result being non-live, an intrusion warning prompt signal is generated. Then, in response to the liveness detection result being live, face recognition is performed based on the RGB image of the face to obtain a face recognition result. In particular, in response to the face recognition result being that the object to be identified is an illegal object (an illegal object is defined as an object that is not in the authorized database), a suspected intrusion warning prompt signal is generated. Security monitoring through the combination of liveness detection and face recognition technology can more accurately identify and detect illegal intrusions, improving the reliability and safety of security monitoring.
[0063] Figure 2 FIG. 1 is a flow chart of an intelligent security monitoring method based on face recognition in an embodiment of the present disclosure. For example, the intelligent security monitoring method based on face recognition can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 2 As shown, according to the embodiment of the present disclosure, the intelligent security monitoring method based on face recognition includes the following steps: S510, acquiring a face infrared image and a face RGB image of an object to be identified collected by an infrared camera and an RGB camera; S520, performing liveness detection based on the face infrared image to obtain a liveness detection result; S530, in response to the liveness detection result being non-liveness, generating an intrusion warning prompt signal; S540, in response to the liveness detection result being liveness, performing face recognition based on the face RGB image to obtain a face recognition result; and, S550, in response to the face recognition result being that the object to be identified is an illegal object, generating a suspected intrusion warning prompt signal.
[0064] Accordingly, in the above-mentioned intelligent security monitoring system based on face recognition, the key step is to identify and detect illegal objects by analyzing the features of the face RGB image. Specifically, in the process of performing face recognition based on the face RGB image to obtain the face recognition result, in order to more fully capture the face feature information in the face RGB image, thereby improving the accuracy of identifying illegal objects, the technical concept of the present application is to perform feature mining on the face RGB image using an image processing and analysis algorithm based on artificial intelligence and deep learning, so as to capture the rich face feature information in the image, thereby performing identity recognition of the face object to determine whether the object is a legal object, so that illegal intrusion detection can be performed more accurately, the reliability of security monitoring can be improved, and the safety of the area can be ensured.
[0065] Accordingly, if Figure 3 As shown, in response to the liveness detection result being liveness, face recognition is performed based on the face RGB image to obtain a face recognition result, including: S541, performing face detection positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map; S542, passing the face semantic feature map through a face semantic typical feature extraction network to obtain a typical face semantic feature vector; S543, performing attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhancement features; and, S544, determining whether the object to be identified is an illegal object based on the face semantics guided face shallow enhancement features.
[0066] Specifically, in response to the liveness detection result being liveness, in the process of performing face recognition based on the face RGB image to obtain the face recognition result, firstly, considering that there is a large amount of information in the face RGB image, which not only contains face semantic feature information related to the identity recognition of the face object, but also contains some background interference noise. Therefore, the face RGB image needs to be input into the MTCNN network model to obtain the face region of interest image. It should be understood that the MTCNN network model is a multi-task convolutional neural network for face detection and key point positioning, wherein the MTCNN network model includes P-Net, R-Net and O-Net, the P-Net is used to generate face candidate areas, it is a lightweight convolutional neural network, using multi-scale feature fusion to quickly detect face candidate areas in a large range; the R-Net can screen and accurately locate the face candidate areas output by P-Net, it is a full convolutional neural network, can adjust the size and position of the bounding box according to the location information of the candidate area; the O-Net can perform final face verification and fine segmentation on the adjusted face candidate area output by R-Net, it is also a full convolutional neural network, can finely segment the face, and verify whether the candidate area is a face. Through the processing of the MTCNN network model, face detection can be performed at different scales, and the accuracy of detection can be gradually improved by cascading. In addition, the MTCNN network model can detect the key positions and sizes of the face in the face RGB image. That is to say, the MTCNN network model can be used to locate the key points of the face, including the positions of the eyes, nose and mouth, laying the foundation for subsequent face recognition.
[0067] Then, considering that in the actual process of face recognition, it is necessary not only to focus on the shallow feature information such as the edge and texture of the key face area in the face RGB image, but also to capture the semantic features therein, such as the shape of the face, the semantic category of the face organs and other deep semantic feature information. Therefore, in the technical solution of the present application, the face region of interest image is further processed through a channel-enhanced feature pyramid network to obtain a shallow face feature map and a face semantic feature map. The channel-enhanced feature pyramid network can extract feature information of different scales and semantic levels in the face region of interest image to enhance the feature expression ability of the face image, thereby improving the accuracy of face detection and recognition. In particular, the shallow face feature map contains low-level and local feature information about the face in the face image, such as shallow features such as the edge, texture and color of the face, which are very important for face detection and alignment; the face semantic feature map contains more advanced and global semantic feature information in the face image, such as face shape, identity and expression, which can help detect the position and size of the face, and are used to help identify the faces of different individuals, which is very important for face recognition.
[0068] Accordingly, in step S541, the face RGB image is subjected to face detection and positioning and multi-level semantic capture to obtain a face shallow feature map and a face semantic feature map, including: inputting the face RGB image into the MTCNN network model to obtain a face region of interest image; and passing the face region of interest image through a channel-enhanced feature pyramid network to obtain the face shallow feature map and the face semantic feature map. The MTCNN network model includes P-Net, R-Net and O-Net.
[0069] It should be understood that since the face semantic feature map contains a large amount of face semantic feature information, there are both useful information for face recognition and some redundant or irrelevant information. Therefore, in order to compress and extract the deep semantic feature information of these faces, so as to more accurately capture the key semantic features of the face, in the technical solution of the present application, the face semantic feature map is further passed through the face semantic typical feature extraction network to obtain a typical face semantic feature vector. Through the analysis of the face semantic typical feature extraction network, the correlation relationship between each face semantic can be learned from the face semantic feature map, and the key semantic features related to the face recognition task can be captured, helping to convert the face semantic feature map into a more compact and more representative typical face semantic feature vector. In this way, it helps to reduce the complexity of the data, and at the same time learn a more representative feature face semantic feature representation, which is more conducive to face recognition tasks, speeds up recognition speed and accuracy, and improves the accuracy and robustness of the system.
[0070] Accordingly, in step S542, the face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector, including: the face semantic feature map is passed through the face semantic typical feature extraction network to process the face semantic typical feature extraction formula to obtain the typical face semantic feature vector; wherein the face semantic typical feature extraction formula is:
[0071]
[0072]
[0073]
[0074] Among them, v i (x) and v j (x) are respectively the i-th and j-th face semantic feature vectors in the sequence of face semantic feature vectors obtained by quantizing the feature matrices along the channel dimension in the face semantic feature graph, V k is the sequence of the facial semantic feature vectors, log represents the logarithmic function value with base 2, L is the length of each facial semantic feature vector, M is the number of vectors in the sequence of facial semantic feature vectors minus one, and D i is the feature value of each position in the local semantic difference feature vector of the face, mask(·) is the masking operation, is the eigenvalue of each position in the local semantic difference feature vector of the masked face, N s is the length of the local semantic difference feature vector of the masked face, exp(·) is an exponential operation, and V is the typical face semantic feature vector.
[0075] Furthermore, since the typical face semantic feature vector captures the high-level semantic feature information of the face image, and the face shallow feature map contains shallow features such as low-level textures and edges in the face image. Therefore, in order to simultaneously consider the different levels of features of the face as well as the global semantic information and local detail information of the face in the final face image feature representation, so as to more comprehensively describe the face features for face recognition, in the technical solution of the present application, the typical face semantic feature vector and the face shallow feature map are further fused through a face semantic guided attention network to obtain a face semantic guided face shallow enhanced feature map. Through the processing of the face semantic guided attention fusion network, it is possible to learn how to effectively fuse the face shallow feature map and the face high-level semantic feature information according to the guidance of the typical face semantic feature vector. In other words, the face semantic guided attention fusion network can use face semantic features for guided feature fusion, and highlight the deep face semantic features related to face recognition in the face shallow feature map during the fusion process, thereby enhancing the expression of face shallow features. In this way, the facial features in the face semantics-guided shallow face enhancement feature map are more prominent and represented, which helps to improve the performance of the face recognition system, making it more robust and accurate.
[0076] Correspondingly, in step S543, the typical face semantic feature vector and the face shallow feature map are subjected to attention fusion processing based on face semantics guidance to obtain face shallow enhancement features guided by face semantics, including: passing the typical face semantic feature vector through a fully connected layer to perform fully connected encoding on the typical face semantic feature vector to obtain a typical face semantic fully connected feature vector; and, performing weighted fusion based on the channel dimension on the typical face semantic fully connected feature vector and the face shallow feature map to obtain the face semantics guided face shallow enhancement feature map as the face semantics guided face shallow enhancement feature.
[0077] Then, the face semantics-guided shallow face enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, which is used to indicate whether the object to be identified is an illegal object. In other words, the facial feature enhancement expression information guided by facial semantics is used for classification processing, so as to use more sufficient facial feature representation to improve the accuracy of identity recognition of facial objects, thereby determining whether the object is a legal object. In this way, illegal intrusion detection can be performed more accurately, the reliability of security monitoring can be improved, and the safety of the area can be ensured.
[0078] Accordingly, in step S544, based on the face semantics guiding the shallow face enhancement features, it is determined whether the object to be identified is an illegal object, including: passing the face semantics guiding the shallow face enhancement feature map through a classifier-based face recognizer to obtain a recognition result, and the recognition result is used to indicate whether the object to be identified is an illegal object.
[0079] Accordingly, in a possible implementation method, the face semantics-guided shallow face enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, and the recognition result is used to indicate whether the object to be identified is an illegal object, including: flattening the face semantics-guided shallow face enhancement feature map to obtain a face semantics-guided shallow face enhancement feature vector; using the fully connected layer of the classifier-based face recognizer to fully connect encode the face semantics-guided shallow face enhancement feature vector to obtain an encoded classification feature vector; and inputting the encoded classification feature vector into the Softmax classification function of the classifier-based face recognizer to obtain the recognition result.
[0080] It should be understood that the role of the classifier is to use the given categories and known training data to learn classification rules and classifiers, and then classify (or predict) unknown data. Logistic regression and SVM are often used to solve binary classification problems. For multi-class classification problems, logistic regression or SVM can also be used, but multiple binary classifications are required to form a multi-classification. However, this is prone to errors and is not efficient. Commonly used multi-classification methods include the Softmax classification function.
[0081] In the technical solution described above, the superficial face feature map and the face semantic feature map respectively express the image semantic features of different scales and depths of the face region of interest image based on the enhanced feature pyramid network, and passing the face semantic feature map through the face semantic typical feature extraction network is to use the face semantic typical feature extraction network to perform facial feature saliency on the face semantic feature map to highlight the important semantic information in the face semantic feature map. Therefore, after the typical face semantic feature vector and the face shallow feature map are passed through the face semantics guided attention fusion network, the face semantics guided face shallow enhanced feature map further expresses the scale depth image semantics guided fusion feature, that is, the face feature semantic information expressed by the typical face semantic feature vector is used to perform feature guidance and enhancement based on the attention mechanism-like mechanism on the face shallow feature map so that the face semantics guided face shallow enhanced feature map not only has the face high-dimensional semantic information, but also can use the alignment structure between the face high-dimensional semantic information and the face shallow information to perform feature saliency based on the attention mechanism, thereby improving the semantic expression distinguishability of the feature matrix image under the channel dimension of the face semantics guided face shallow enhanced feature map. However, this will also make the feature distribution of the face semantics guided face shallow enhanced feature map less constrained than the overall distribution regression of the classifier-based face recognizer, affecting the accuracy of the recognition result.
[0082] In a preferred example, the face semantics-guided shallow face enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, including the following steps: feature flattening the face semantics-guided shallow face enhancement feature map to obtain a face semantics-guided shallow face enhancement feature vector; calculating the point sum of the face semantics-guided shallow face enhancement feature vector and the square root of the length of the face semantics-guided shallow face enhancement feature vector and the reciprocal of the square root of the bi-norm of the face semantics-guided shallow face enhancement feature vector to obtain a face semantics-guided shallow face enhancement offset feature vector; calculating the face semantics-guided shallow face enhancement offset feature vector An exponential function with natural constants as base is used to obtain a face semantics-guided face shallow enhancement offset prediction feature vector; the face semantics-guided face shallow enhancement feature vector is calculated as the dot product of a norm and a weight hyperparameter of the face semantics-guided face shallow enhancement feature vector to obtain a face semantics-guided face shallow enhancement constraint feature vector; the face semantics-guided face shallow enhancement offset prediction feature vector and the face semantics-guided face shallow enhancement constraint feature vector are calculated as the dot sum to obtain an optimized face semantics-guided face shallow enhancement feature vector; the optimized face semantics-guided face shallow enhancement feature vector is passed through a classifier-based face recognizer to obtain a recognition result.
[0083] Specifically, in the above preferred example, the structured norm representation of the face semantics guided face shallow enhancement feature vector obtained by expanding the face semantics guided face shallow enhancement feature map is used as the local canonical coordinates for each eigenvalue of the face semantics guided face shallow enhancement feature vector to determine the class rotation offset of the vector overall distribution of the face semantics guided face shallow enhancement feature vector relative to the eigenvalue for the class offset prediction direction of each eigenvalue of the face semantics guided face shallow enhancement feature vector as the center, and the bounding box of the vector eigenvalue distribution of the face semantics guided face shallow enhancement feature vector is used for eigenvalue constraint to improve the constraint of the face semantics guided face shallow enhancement feature vector under the overall regression distribution, thereby improving the training speed of the model and the accuracy of the recognition result obtained by the face semantics guided face shallow enhancement feature map through the classifier-based face recognizer. In this way, illegal objects and illegal intrusions can be identified more accurately, thereby improving the reliability of security monitoring and ensuring the safety of the area.
[0084] Based on the above embodiments, see Figure 4 As shown, it is a structural schematic diagram of an intelligent security monitoring system 800 based on face recognition in an embodiment of the present disclosure. The intelligent security monitoring system 800 based on face recognition includes: an image acquisition module 810, which is used to acquire a face infrared image and a face RGB image of an object to be identified collected by an infrared camera and an RGB camera; a liveness detection module 820, which is used to perform liveness detection based on the face infrared image to obtain a liveness detection result; an intrusion warning prompt signal generation module 830, which is used to generate an intrusion warning prompt signal in response to the liveness detection result being a non-liveness; a face recognition module 840, which is used to perform face recognition based on the face RGB image to obtain a face recognition result in response to the liveness detection result being a liveness; and a suspected intrusion warning prompt module 850, which is used to generate a suspected intrusion warning prompt signal in response to the face recognition result being that the object to be identified is an illegal object.
[0085] In one example, in the above-mentioned intelligent security monitoring system 800 based on face recognition, the face recognition module 840 is used to: perform face detection and positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map; pass the face semantic feature map through a face semantic typical feature extraction network to obtain a typical face semantic feature vector; perform attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhancement features; and, based on the face semantics guided face shallow enhancement features, determine whether the object to be identified is an illegal object.
[0086] Here, those skilled in the art can understand that the specific functions and operations of each module in the above-mentioned intelligent security monitoring system 800 based on face recognition have been referred to above. Figure 2 to Figure 3 The invention has been introduced in detail in the description of the intelligent security monitoring method based on face recognition, and therefore, its repeated description will be omitted.
[0087] Figure 5 FIG. 1 is an application scenario diagram of the intelligent security monitoring method based on face recognition according to an embodiment of the present disclosure. Figure 5 As shown, in this application scenario, first, obtain the infrared camera (for example, Figure 5 C1 as shown in the figure) and an RGB camera (e.g. Figure 5 The infrared image of the face of the object to be identified (for example, Figure 5 D1 as shown in the figure) and face RGB images (e.g., Figure 5 Then, the face infrared image and the face RGB image are input to a server (for example, Figure 5 In S) shown in , the server can use the intelligent security monitoring algorithm based on face recognition to process the face infrared image and the face RGB image to generate a suspected intrusion warning prompt signal.
[0088] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present disclosure. In some possible implementations, the electronic device in the embodiments of the present disclosure may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the intelligent security monitoring method based on face recognition in the above embodiments when executing the program.
[0089] For example, the electronic device is disclosed in the present invention Figure 1 Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.
[0090] An embodiment of the present disclosure also provides a computer-readable storage medium. Figure 6 FIG. 1 is a schematic diagram of a computer-readable storage medium 1000 according to an embodiment of the present disclosure. Figure 6As shown, the computer-readable storage medium 1000 stores computer-executable instructions 1001. When the computer-executable instructions 1001 are executed by the processor, the intelligent security monitoring method based on face recognition according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.
[0091] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the intelligent security monitoring method based on face recognition according to the embodiments of the present disclosure.
[0092] Those skilled in the art will appreciate that the contents disclosed in this disclosure may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.
[0093] In addition, although the present disclosure makes various references to certain units in the system according to embodiments of the present disclosure, any number of different units may be used and run on the client and / or server. The units are illustrative only, and different aspects of the systems and methods may use different units.
[0094] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of a software functional module. The present disclosure is not limited to any particular form of combination of hardware and software.
[0095] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.
[0096] The above is an explanation of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It should be understood that the above is an explanation of the present disclosure and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.
Claims
1. An intelligent security monitoring method based on face recognition, characterized in that: include: Obtaining an infrared image and an RGB image of the face of the object to be identified, which are captured by an infrared camera and an RGB camera; Based on the infrared face image, performing liveness detection to obtain a liveness detection result; In response to the liveness detection result being a non-liveness, generating an intrusion warning prompt signal; In response to the liveness detection result being liveness, performing face recognition based on the face RGB image to obtain a face recognition result; as well as In response to the face recognition result that the object to be identified is an illegal object, generating a suspected intrusion warning prompt signal; Wherein, in response to the liveness detection result being liveness, performing face recognition based on the face RGB image to obtain a face recognition result includes: Performing face detection and positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map; The face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector; Performing attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhanced features; and Determining whether the object to be identified is an illegal object based on the face semantics-guided shallow face enhancement features; The typical face semantic feature vector and the face shallow feature map are subjected to attention fusion processing based on face semantic guidance to obtain face semantic guidance face shallow enhancement features, including: Passing the typical face semantic feature vector through a fully connected layer to perform fully connected encoding on the typical face semantic feature vector to obtain a typical face semantic fully connected feature vector; and Performing a weighted fusion based on the channel dimension on the typical face semantic fully connected feature vector and the face shallow feature map to obtain the face semantics guided face shallow enhancement feature map as the face semantics guided face shallow enhancement feature; Wherein, determining whether the object to be identified is an illegal object based on the face semantics guiding the shallow face enhancement features includes: Passing the face semantics-guided shallow face enhancement feature map through a classifier-based face recognizer to obtain a recognition result, wherein the recognition result is used to indicate whether the object to be recognized is an illegal object; Among them, the face semantics guided face shallow enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, including the following steps: feature flattening the face semantics guided face shallow enhancement feature map to obtain a face semantics guided face shallow enhancement feature vector; calculating the point sum of the face semantics guided face shallow enhancement feature vector and the square root of the length of the face semantics guided face shallow enhancement feature vector and the reciprocal of the square root of the bi-norm of the face semantics guided face shallow enhancement feature vector to obtain a face semantics guided face shallow enhancement offset feature vector; calculating the natural square root of the face semantics guided face shallow enhancement offset feature vector An exponential function with a constant as the base is used to obtain a face semantics-guided face shallow enhancement offset prediction feature vector; the face semantics-guided face shallow enhancement feature vector is calculated as the product of a norm and a weight hyperparameter of the face semantics-guided face shallow enhancement feature vector to obtain a face semantics-guided face shallow enhancement constraint feature vector; the face semantics-guided face shallow enhancement offset prediction feature vector and the face semantics-guided face shallow enhancement constraint feature vector are calculated as the dot sum to obtain an optimized face semantics-guided face shallow enhancement feature vector; the optimized face semantics-guided face shallow enhancement feature vector is passed through a classifier-based face recognizer to obtain a recognition result.
2. The intelligent security monitoring method based on face recognition according to claim 1 is characterized in that: The face RGB image is subjected to face detection and positioning and multi-level semantic capture to obtain a face shallow feature map and a face semantic feature map, including: Inputting the face RGB image into the MTCNN network model to obtain a face region of interest image; and The face region of interest image is passed through a channel-enhanced feature pyramid network to obtain the face shallow feature map and the face semantic feature map.
3. The intelligent security monitoring method based on face recognition according to claim 2 is characterized in that: The MTCNN network model includes P-Net, R-Net and O-Net.
4. The intelligent security monitoring method based on face recognition according to claim 3 is characterized in that: The facial semantic feature map is passed through a facial semantic typical feature extraction network to obtain a typical facial semantic feature vector, including: The face semantic feature map is processed by the face semantic typical feature extraction network using the following face semantic typical feature extraction formula to obtain the typical face semantic feature vector; wherein the face semantic typical feature extraction formula is: Among them, v i (x) and v j (x) are respectively the i-th and j-th face semantic feature vectors in the sequence of face semantic feature vectors obtained by quantizing the feature matrices along the channel dimension in the face semantic feature graph, V k is the sequence of the facial semantic feature vectors, log represents the logarithmic function value with base 2, L is the length of each facial semantic feature vector, M is the number of vectors in the sequence of facial semantic feature vectors minus one, and D i is the feature value of each position in the local semantic difference feature vector of the face, mask(·) is the masking operation, is the eigenvalue of each position in the local semantic difference feature vector of the masked face, N s is the length of the local semantic difference feature vector of the masked face, exp(·) is an exponential operation, and V is the typical face semantic feature vector.
5. An intelligent security monitoring system based on face recognition, characterized in that: include: An image acquisition module is used to acquire an infrared image and an RGB image of a face of an object to be identified, which are captured by an infrared camera and an RGB camera; A liveness detection module, used for performing liveness detection based on the face infrared image to obtain a liveness detection result; An intrusion warning prompt signal generating module, used for generating an intrusion warning prompt signal in response to the liveness detection result being a non-liveness; A face recognition module, configured to, in response to the liveness detection result being liveness, perform face recognition based on the face RGB image to obtain a face recognition result; as well as A suspected intrusion warning prompt module is used to generate a suspected intrusion warning prompt signal in response to the face recognition result that the object to be identified is an illegal object; Wherein, the face recognition module is used to: Performing face detection and positioning and multi-level semantic capture on the face RGB image to obtain a face shallow feature map and a face semantic feature map; The face semantic feature map is passed through a face semantic typical feature extraction network to obtain a typical face semantic feature vector; Performing attention fusion processing based on face semantics guidance on the typical face semantic feature vector and the face shallow feature map to obtain face semantics guided face shallow enhanced features; and Determining whether the object to be identified is an illegal object based on the face semantics-guided shallow face enhancement features; The typical face semantic feature vector and the face shallow feature map are subjected to attention fusion processing based on face semantic guidance to obtain face semantic guidance face shallow enhancement features, including: Passing the typical face semantic feature vector through a fully connected layer to perform fully connected encoding on the typical face semantic feature vector to obtain a typical face semantic fully connected feature vector; and Performing a weighted fusion based on the channel dimension on the typical face semantic fully connected feature vector and the face shallow feature map to obtain the face semantics guided face shallow enhancement feature map as the face semantics guided face shallow enhancement feature; Wherein, determining whether the object to be identified is an illegal object based on the face semantics guiding the shallow face enhancement features includes: Passing the face semantics-guided shallow face enhancement feature map through a classifier-based face recognizer to obtain a recognition result, wherein the recognition result is used to indicate whether the object to be recognized is an illegal object; Among them, the face semantics guided face shallow enhancement feature map is passed through a classifier-based face recognizer to obtain a recognition result, including the following steps: feature flattening the face semantics guided face shallow enhancement feature map to obtain a face semantics guided face shallow enhancement feature vector; calculating the point sum of the face semantics guided face shallow enhancement feature vector and the square root of the length of the face semantics guided face shallow enhancement feature vector and the reciprocal of the square root of the bi-norm of the face semantics guided face shallow enhancement feature vector to obtain a face semantics guided face shallow enhancement offset feature vector; calculating the natural square root of the face semantics guided face shallow enhancement offset feature vector An exponential function with a constant as the base is used to obtain a face semantics-guided face shallow enhancement offset prediction feature vector; the face semantics-guided face shallow enhancement feature vector is calculated as the product of a norm and a weight hyperparameter of the face semantics-guided face shallow enhancement feature vector to obtain a face semantics-guided face shallow enhancement constraint feature vector; the face semantics-guided face shallow enhancement offset prediction feature vector and the face semantics-guided face shallow enhancement constraint feature vector are calculated as the dot sum to obtain an optimized face semantics-guided face shallow enhancement feature vector; the optimized face semantics-guided face shallow enhancement feature vector is passed through a classifier-based face recognizer to obtain a recognition result.
