Subway turnstile intelligent release method and system based on face recognition

Abstract

The invention relates to the technical field of subway turnstiles, and discloses a subway turnstile intelligent release method and system based on face recognition, and the method comprises the steps: firstly obtaining an image frame sequence containing a face through a camera, screening and enhancing a qualified face image through an advanced image processing technology, and guaranteeing the accuracy of a subsequent recognition process; and then, a convolutional neural network model is adopted to carry out occlusion detection, an unoccluded key face region is accurately positioned, and recognition failure caused by missing of partial face features is avoided. Then, the key areas are input into a specially optimized feature extraction model to generate face feature vectors, and the face feature vectors are compared with face template features pre-stored in a database to complete identity verification. And finally, according to the identity authentication result and the validity of the state of the associated account, determining whether to send a gate opening instruction, thereby realizing rapid and safe automatic passage control. Therefore, the identification efficiency and accuracy are improved, and the stability of the system and the user experience are enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of subway gates, and more specifically, to a method and system for intelligently releasing subway gates based on facial recognition. Background Art

[0002] In the current context of intelligent transportation systems, intelligent subway gate release methods based on facial recognition have gradually become one of the key technologies for improving traffic efficiency and optimizing user experience. With the increasing expansion of urban rail transit networks, efficient and accurate passenger authentication has become particularly important. However, traditional facial recognition technology based on complete facial features faces many challenges in practical applications, especially the robustness of recognition when partially obscured by the face (such as wearing a mask or glasses).

[0003] In public places like subways, it's common for passengers to wear masks. This isn't just due to hygiene requirements, but also to public health policy requirements during specific periods. The presence of a mask significantly reduces the effective facial information available for recognition, particularly covering the nose and mouth, two areas crucial for distinguishing individuals. Traditional facial recognition algorithms often rely on feature points across the entire face to construct a facial template. When these critical areas are obscured, the system's recognition performance significantly degrades. Glasses also pose similar challenges: frames can hinder the extraction of important features around the eyes, while lens reflections can reduce image quality, further impacting recognition.

[0004] Therefore, we look forward to an optimized intelligent release solution for subway gates based on facial recognition. Summary of the Invention

[0005] In order to solve the problem of recognition robustness in the existing technology under the condition of partial facial occlusion (such as wearing a mask or glasses), this application is proposed. The embodiments of this application provide a method and system for intelligent release of subway gates based on face recognition.

[0006] According to one aspect of the present application, a method for intelligent release of subway gates based on face recognition is provided, including: obtaining a sequence of image frames containing faces captured by a camera; extracting a face image of qualified quality from the sequence of image frames containing faces; inputting the face image into an occlusion detection model to obtain an occlusion detection result; based on the occlusion detection result, locating an unobstructed key area in the face image; inputting the unobstructed key area into an occlusion optimization feature extraction model to obtain a face feature vector; comparing the face feature vector with the face template feature vector of the corresponding user in a database to obtain an identity authentication result, and querying whether the account status of the associated account is valid based on the identity authentication result; in response to the identity authentication result being successful and the account status being valid, sending an opening instruction signal to the gate control module.

[0007] In the above-mentioned intelligent release method for subway gates based on face recognition, extracting a face image of qualified quality from the image frame sequence containing the face includes: performing face detection on the image frame sequence containing the face to obtain a face ROI image frame sequence; performing image quality assessment on each face ROI image frame in the face ROI image frame sequence to obtain a sequence of face image quality assessment results; selecting the face ROI image corresponding to the best face image quality assessment result in the sequence of face image quality assessment results as the face target image; and performing image enhancement on the face target image to obtain the face image of qualified quality.

[0008] In the above-mentioned intelligent release method for subway gates based on face recognition, the face image is input into an occlusion detection model to obtain an occlusion detection result, including: inputting the face image into a face feature extractor based on a convolutional neural network model to obtain a local visual feature coding map of the face; performing adaptive spatial saliency on the local visual feature coding map of the face to obtain a spatially significant coding map of the local visual feature of the face; inputting the spatially significant coding map of the local visual feature of the face into a segmentation model based on a classification layer to obtain an occlusion detection result, and the occlusion detection result includes an occlusion type label and a face segmentation mask map.

[0009] In the above-mentioned intelligent release method for subway gates based on face recognition, the local visual feature coding map of the face is adaptively spatially saliency to obtain a local visual feature spatially salient coding map of the face, including: performing spatial position saliency prediction based on void convolution coding on the local visual feature coding map of the face to obtain a local visual feature position saliency prediction map of the face; inputting the local visual feature position saliency prediction map of the face into the activation layer to obtain a local visual feature position saliency prediction probability map of the face; calculating the position point multiplication between the local visual feature position saliency prediction probability map of the face and the local visual feature coding map to obtain the local visual feature spatial saliency coding map of the face.

[0010] In the above-mentioned intelligent release method for subway gates based on face recognition, the local visual feature coding map of the face is adaptively spatially saliency to obtain a local visual feature spatially salient coding map of the face, including: performing spatial position saliency prediction based on void convolution coding on the local visual feature coding map of the face to obtain a local visual feature position saliency prediction map of the face; performing direction-adaptive response sensitivity correction on the local visual feature position saliency prediction map of the face to obtain a corrected local visual feature position saliency prediction map of the face; inputting the corrected local visual feature position saliency prediction map of the face into the activation layer to obtain a local visual feature position saliency prediction probability map of the face; and calculating the positional point multiplication between the local visual feature position saliency prediction probability map of the face and the local visual feature coding map to obtain the local visual feature spatially salient coding map of the face.

[0011] In the above-mentioned intelligent release method for subway gates based on face recognition, the direction-adaptive response sensitivity correction is performed on the significant prediction map of the local visual feature position of the face to obtain a corrected significant prediction map of the local visual feature position of the face, including: performing mean dimension compression on the significant prediction map of the local visual feature position of the face along the width dimension, height dimension and channel dimension respectively to obtain a width-direction feature vector, a height-direction feature vector and a channel-direction feature vector; performing matrix multiplication on the width-direction feature vector, the height-direction feature vector and the channel-direction feature vector with the significant prediction map of the local visual feature position of the face in the corresponding dimensions to obtain a width-direction response feature matrix, a height-direction response feature matrix and a channel-direction response feature matrix; based on the width-direction feature vector, the height-direction feature vector and the channel-direction feature vector, calculating the spatial significance response weight in each direction to obtain a width-direction feature vector. The method comprises the following steps: performing matrix dot multiplication on the width direction response feature matrix, the height direction response feature matrix and the channel direction response feature matrix with the facial local visual feature position saliency prediction map in corresponding dimensions to obtain a width direction response feature map, a height direction response feature map and a channel direction response feature map; weighting the width direction response feature map, the height direction response feature map and the channel direction response feature map with the spatial significance response weight in the width direction, the height direction spatial significance response weight and the channel direction spatial significance response weight to obtain a corrected feature map; performing dot multiplication correction on the facial local visual feature position saliency prediction map with the corrected feature map to obtain a corrected facial local visual feature position saliency prediction map.

[0012] In the above-mentioned subway gate intelligent release method based on face recognition, the classification layer includes a point convolution layer and a Softmax classification function.

[0013] In the above-mentioned intelligent release method for subway gates based on face recognition, based on the occlusion detection result, the unobstructed key area in the face image is located, including: extracting the face segmentation mask map from the occlusion detection result; extracting several key areas from the face image; for each key area, querying its corresponding pixel in the face segmentation mask map; calculating the proportion of pixels with a type label of non-obstruction in the key area, and determining whether the key area belongs to an unobstructed key area based on a comparison between the proportion and a preset threshold.

[0014] In the above-mentioned intelligent release method for subway gates based on face recognition, based on the occlusion detection result, the unobstructed key area in the face image is located, including: extracting the face segmentation mask map from the occlusion detection result; calculating the position point multiplication between the face segmentation mask map and the face image to obtain the unobstructed key area.

[0015] According to another aspect of the present application, a subway gate intelligent release system based on face recognition is also provided, including: an image frame sequence acquisition module, used to acquire an image frame sequence containing a face captured by a camera; a qualified face image extraction module, used to extract a face image of qualified quality from the image frame sequence containing the face; an occlusion detection result determination module, used to input the face image into an occlusion detection model to obtain an occlusion detection result; a key area positioning module, used to locate the unobstructed key area in the face image based on the occlusion detection result; a face feature extraction module, used to input the unobstructed key area into an occlusion optimization feature extraction model to obtain a face feature vector; an identity authentication module, used to compare the face feature vector with the face template feature vector of the corresponding user in the database to obtain an identity authentication result, and query whether the account status of the associated account is valid based on the identity authentication result; a gate control instruction sending module, used to send an opening instruction signal to the gate control module in response to the identity authentication result being successful and the account status being valid.

[0016] Compared with the existing technology, the face recognition-based intelligent release method and system for subway gates provided in this application first obtains a sequence of image frames containing faces through a camera, and uses advanced image processing technology to screen and enhance facial images of qualified quality to ensure the accuracy of the subsequent recognition process. To address the occlusion problem, a convolutional neural network model is used for occlusion detection to accurately locate key facial areas that are not obscured, avoiding recognition failures caused by the lack of some facial features. Then, these key areas are input into a specially optimized feature extraction model to generate facial feature vectors, which are compared with the pre-stored facial template features in the database to complete identity authentication. Finally, based on the identity authentication results and the validity of the associated account status, it is decided whether to send a gate opening command to achieve fast and safe automatic passage control. In this way, not only the recognition efficiency and accuracy are improved, but also the stability of the system and the user experience are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 The figure illustrates a schematic flow chart of a method for intelligent release of subway gates based on face recognition according to an embodiment of the present application.

[0019] Figure 2 The figure shows a schematic flow chart of step S2 in the intelligent release method of subway gates based on face recognition according to an embodiment of the present application.

[0020] Figure 3 The figure shows a schematic flow chart of step S3 in the intelligent release method of subway gates based on face recognition according to an embodiment of the present application.

[0021] Figure 4 The figure shows a schematic flowchart of step S32 in the intelligent release method of subway gates based on face recognition according to an embodiment of the present application.

[0022] Figure 5 The figure illustrates a schematic flowchart of a preferred embodiment of step S32 in the intelligent release method for subway gates based on face recognition according to an embodiment of the present application.

[0023] Figure 6 The figure illustrates a schematic flow chart of step S4 in the intelligent release method of subway gates based on face recognition according to an embodiment of the present application.

[0024] Figure 7The figure shows a schematic block diagram of a subway gate intelligent release system based on face recognition according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0026] Figure 1 The figure shows a schematic flow chart of the intelligent release method of subway gate based on face recognition according to an embodiment of the present application. Figure 1 As shown, the present application provides a method for intelligent release of subway gates based on face recognition, including: S1, obtaining a sequence of image frames containing faces captured by a camera; S2, extracting a face image of qualified quality from the image frame sequence containing faces; S3, inputting the face image into an occlusion detection model to obtain an occlusion detection result; S4, based on the occlusion detection result, locating the unobstructed key area in the face image; S5, inputting the unobstructed key area into an occlusion optimization feature extraction model to obtain a face feature vector; S6, comparing the face feature vector with the face template feature vector of the corresponding user in the database to obtain an identity authentication result, and querying whether the account status of the associated account is valid based on the identity authentication result; S7, in response to the identity authentication result being successful and the account status being valid, sending an opening command signal to the gate control module.

[0027] For example, in step S1, a sequence of image frames containing human faces, captured by a camera, is acquired. It should be understood that subway stations are highly dynamic environments, with passengers passing through gates at varying speeds and angles. By continuously acquiring a sequence of image frames, facial information in various poses can be captured, thereby improving recognition accuracy and robustness.

[0028] In one embodiment, a camera installed at a subway gate captures a sequence of image frames containing human faces. Specifically, the camera must have high resolution and low latency to capture clear and timely facial images. Given the variable lighting conditions within a subway station, the selected camera should also support automatic exposure and automatic white balance to ensure high-quality images under varying lighting conditions. Furthermore, due to the high speed at which passengers pass through the gate, the camera must also have high-speed continuous shooting capabilities to capture key facial moments and provide sufficient image data for subsequent processing. Furthermore, given the high instantaneous passenger flow within a subway station, a large number of continuous image frame sequences are generated in a short period of time, requiring effective management measures to process this data. To this end, an efficient caching mechanism can be designed to temporarily store the most recent few seconds of video frames and dynamically adjust the number of retained frames using a sliding window algorithm. This approach not only reduces system burden but also minimizes redundant data without compromising user experience, improving overall processing efficiency.

[0029] Exemplarily, in step S2, a facial image of qualified quality is extracted from the sequence of image frames containing faces. It should be understood that the original video stream obtained from the camera usually contains a large number of image frame sequences, not all of which are suitable for identity verification. Due to factors such as the complex and changeable environment in the subway station, the different speeds of passengers moving, and unstable lighting conditions, directly using unprocessed image frames for recognition will result in a high false recognition rate and missed recognition rate. For example, in the case of rapid movement or uneven lighting, some frames may appear blurred, overexposed, or underexposed, seriously affecting the accurate extraction of facial features. Therefore, these image frames must be further quality evaluated and screened to ensure that the images used have sufficient clarity and contrast, thereby improving the accuracy of recognition. Therefore, facial images of qualified quality are extracted from the sequence of image frames containing faces.

[0030] In one embodiment, Figure 2 As shown, in step S2, a face image of qualified quality is extracted from the image frame sequence containing the face, including: S21, performing face detection on the image frame sequence containing the face to obtain a face ROI image frame sequence; S22, performing image quality assessment on each face ROI image frame in the face ROI image frame sequence to obtain a sequence of face image quality assessment results; S23, selecting the face ROI image corresponding to the best face image quality assessment result in the sequence of face image quality assessment results as the face target image; S24, performing image enhancement on the face target image to obtain the face image of qualified quality.

[0031] In step S21, a convolutional neural network (CNN) model in deep learning is used to locate the face position in the image frame. In a specific embodiment of the present application, MTCNN (multi-task convolutional neural network) can be used. This is a model specifically used for multi-task cascade face detection and alignment, which gradually refines the position and bounding box of the face area through three networks. The first stage uses P-Net (Proposal Network), which quickly scans the entire image and proposes candidate windows; the second stage R-Net (Refine Network) further screens these candidate windows and performs preliminary corrections; finally, O-Net (Output Network) is responsible for more accurately locating facial key points and adjusting the final bounding box. In this way, a high-quality face ROI image frame sequence can be effectively extracted from the original video stream. The advantage of this method is that it can adapt to different lighting conditions, angle changes, and partial occlusion, thereby ensuring the accuracy of the basic data in subsequent steps.

[0032] In step S22, the best image most suitable for recognition is screened out from a large number of possible face ROI image frames through image quality assessment. In this process, a quality assessment model based on machine learning is adopted, such as the SVM combined with feature engineering method or the self-supervised learning framework based on deep learning. In a specific embodiment of the present application, VGG16 is used as the basic network, and the feature extractor is obtained after removing its fully connected layer, and then the extracted features are input into the SVM classifier for training. In this process, a large number of sample data sets marked as good / poor quality need to be collected. By learning from these data, the model learns how to automatically judge the quality of facial images. In addition, in other embodiments of the present application, features in multiple dimensions such as clarity, contrast, noise level, etc. can also be introduced as auxiliary information to improve the accuracy of the evaluation. The purpose of this is to ensure that the selected image has sufficient visual information to facilitate the subsequent identity verification process.

[0033] In steps S23 and S24, after the optimal facial ROI image is determined as the target image, the image enhancement process begins. Image enhancement aims to improve the overall image quality and detail representation, making facial features more prominent. In a specific embodiment of the present application, the facial target image is enhanced through histogram equalization. Histogram equalization can effectively improve the overall brightness distribution of the image, making facial features more distinct. Specifically, histogram equalization enhances the dynamic range of the image by redistributing the frequency distribution of pixel values, making it particularly suitable for conditions with uneven lighting. In addition, local contrast adjustment is also an important method, focusing on highlighting important details such as the contours around the eyes, nose, and mouth. For example, the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm can independently adjust the contrast of each small block based on the characteristics of different image regions, avoiding the problem of over-enhancement caused by global adjustments. At the same time, appropriate denoising is also essential. Techniques such as Gaussian filters or bilateral filters can also be used to effectively reduce interference caused by shooting equipment or environmental factors, ensuring image purity.

[0034] Exemplarily, in step S3, the facial image is input into the occlusion detection model to obtain an occlusion detection result. It should be understood that the presence of obstructions will directly affect the effectiveness of traditional face recognition algorithms. For example, a mask obstructs the nose and mouth areas, and glasses may affect the extraction of eye features. These obstructions will cause the loss or blurring of key feature information, thereby reducing recognition accuracy. By using the occlusion detection model, it is possible to accurately locate which areas are obstructed and determine the key unobstructed areas. In this way, in the subsequent feature extraction process, the system can focus on those areas that are not obstructed and have important recognition value, thereby improving the overall recognition accuracy.

[0035] In one embodiment, Figure 3 As shown, in step S3, the face image is input into the occlusion detection model to obtain an occlusion detection result, including: S31, inputting the face image into a face feature extractor based on a convolutional neural network model to obtain a face local visual feature coding map; S32, performing adaptive spatial saliency on the face local visual feature coding map to obtain a face local visual feature spatial salient coding map; S33, inputting the face local visual feature spatial salient coding map into a segmentation model based on a classification layer to obtain an occlusion detection result, the occlusion detection result including an occlusion type label and a face segmentation mask map.

[0036] In step S31, the preprocessed face image needs to be input into a face feature extractor based on a convolutional neural network (CNN). In a specific embodiment of the present application, ResNet50 is selected as the basic model, which constructs a deep network structure by stacking multiple residual blocks, and can effectively capture complex patterns and detail features in face images while maintaining a low parameter amount. ResNet50 can extract rich local visual features from the input image and convert it into a high-dimensional feature vector representation, that is, a local visual feature encoding map of the face. This encoding map not only contains the key information of the face, but also retains a certain degree of spatial structure information, which is very critical for subsequent occlusion detection.

[0037] In step S32, adaptive spatial saliency processing is performed on the local visual feature coding map of the face. This process aims to highlight the areas that are most important for occlusion detection while suppressing irrelevant background noise. In one embodiment, Figure 4 As shown, in step S32, adaptive spatial saliency is performed on the facial local visual feature encoding map to obtain a facial local visual feature spatial saliency encoding map, including: S321, performing spatial position saliency prediction based on dilated convolution coding on the facial local visual feature encoding map to obtain a facial local visual feature position saliency prediction map; S322, inputting the facial local visual feature position saliency prediction map into an activation layer to obtain a facial local visual feature position saliency prediction probability map; and S323, calculating the position-wise multiplication between the facial local visual feature position saliency prediction probability map and the facial local visual feature encoding map to obtain the facial local visual feature spatial saliency encoding map. It should be understood that the dilated convolution technique is used to expand the receptive field without increasing computational cost. By adjusting the dilation rate at different levels, the spatial relationship between various parts of the face can be more comprehensively captured without losing resolution. For example, a smaller dilation rate is set in the first layer to focus on small facial features, and the dilation rate is gradually increased in deeper layers to include more contextual information. After a series of these operations, a saliency prediction map of the facial local visual feature locations is obtained. This prediction map is then fed into an activation layer, such as a ReLU or Leaky ReLU activation function, to enhance nonlinear representation capabilities and output a saliency prediction probability map of the facial local visual feature locations. The final step is to calculate the positional dot product between the predicted probability map and the original encoding map to generate the final spatial saliency encoding map of the facial local visual features.

[0038] Here, when the spatial position saliency prediction of the local visual feature coding map of the face is performed based on the void convolution coding, the spatial distribution characteristics of the local visual features of the face image are actually encoded based on the convolution receptive field of the void convolution coding. Therefore, the local visual feature saliency of the local visual feature position saliency prediction map of the face will have a high local spatial correlation with the convolution receptive field of the void convolution coding, which will lead to a weakening of the global adaptive response adjustment ability of the joint spatial correlation, and reduce the global spatial saliency prediction coding expression effect of the spatial position saliency prediction based on the void convolution coding.

[0039] Based on this, in a preferred embodiment, Figure 5 As shown, in step S32, the facial local visual feature coding map is adaptively spatially salient to obtain a facial local visual feature spatially salient coding map, including: S321, performing spatial position saliency prediction based on dilated convolutional coding on the facial local visual feature coding map to obtain a facial local visual feature position saliency prediction map; S322, performing direction-adaptive response sensitivity correction on the facial local visual feature position saliency prediction map to obtain a corrected facial local visual feature position saliency prediction map; S323, inputting the corrected facial local visual feature position saliency prediction map into an activation layer to obtain a facial local visual feature position saliency prediction probability map; S324, calculating the position-wise multiplication between the facial local visual feature position saliency prediction probability map and the facial local visual feature coding map to obtain the facial local visual feature spatially salient coding map. That is, before inputting into the activation layer, the facial local visual feature position saliency prediction map is subjected to direction-adaptive response sensitivity correction to improve the global spatial saliency prediction coding expression effect of the facial local visual feature position saliency prediction map.

[0040] In this preferred embodiment, the direction-adaptive response sensitivity correction is performed on the local visual feature position saliency prediction map of the human face to obtain a corrected local visual feature position saliency prediction map of the human face, including: first, the local visual feature position saliency prediction map F∈R W×H×C , where R represents a set of real numbers, W, H, and C represent the width, height, and number of channels of the significant prediction map of the local visual feature position of the face, respectively. Mean dimension compression is performed along the width dimension, height dimension, and channel dimension to obtain the width-wise feature vector V1∈R 1×W , height direction feature vector V2∈R 1×H and channel direction feature vector V3∈R 1×C , thereby decomposing the spatial distribution with directional feature vectors with direction sensitivity to better reflect the directional feature distribution pattern on the global scale.

[0041] Then, the width direction feature vector V1, the height direction feature vector V2 and the channel direction feature vector V3 are matrix multiplied with the local visual feature position saliency prediction map of the face in the corresponding dimensions to obtain the width direction response feature matrix M1∈R H×C , height direction response characteristic matrix M2∈R W×C And the channel direction response feature matrix M3∈R W×H , which is used to reflect the global spatial directional saliency response in the overall specific direction, that is, to decouple the global spatial saliency for each distribution direction. Here, those skilled in the art should know that when the feature vector is multiplied by the feature map, the feature map is actually weighted summed in a specific direction to generate a new feature matrix. Taking the width direction feature vector V1 as an example, the width direction feature vector V1∈R 1×W and the salient prediction map F∈R of the local visual feature position of the face W×H×C Performing matrix multiplication in the width direction is equivalent to weighted summing of all pixel values in the width direction with V1 for each height H and channel C of the significant prediction map of the local visual feature position of the face. The result is an H×C matrix that represents the global response in the width direction.

[0042] Based on the width direction feature vector, the height direction feature vector and the channel direction feature vector, the spatial significance response weight in each direction is calculated using the following formula to obtain the spatial significance response weight in the width direction, the spatial significance response weight in the height direction and the spatial significance response weight in the channel direction. The formula is:

[0043]

[0044] Among them, ||·||2 represents the bi-norm of the matrix, ω1 represents the spatial significance response weight in the width direction, ω2 represents the spatial significance response weight in the height direction, and ω3 represents the spatial significance response weight in the channel direction.

[0045] Therefore, the spatial significance response weight can comprehensively consider the continuous change law of spatial distribution in various directions, so that the direction with stronger continuity response has a higher weight, thereby enhancing the global directional importance of the feature map in multiple dimensions.

[0046] The width-direction response feature matrix M1, the height-direction response feature matrix M2 and the channel-direction response feature matrix M3 are matrix-multiplied with the local visual feature position significant prediction map F of the human face in corresponding dimensions to obtain the width-direction response feature map F1, the height-direction response feature map F2 and the channel-direction response feature map F3. Here, those skilled in the art should know that matrix-multiplying the feature moment with the feature map in corresponding dimensions is actually multiplying each eigenvalue of the feature matrix with all eigenvalues of the feature map in the corresponding dimension. Taking the matrix-dot multiplication of the width-direction response feature matrix M1 with the local visual feature position significant prediction map F of the human face in corresponding dimensions as an example, it is equivalent to multiplying each eigenvalue of the width-direction response feature matrix M1 with all eigenvalues of the local visual feature position significant prediction map F in the width direction (note that only multiplication is performed here, no summation is performed, so there will be no change in dimension), and the result will still be a feature map of [W, H, C].

[0047] The width direction response feature map, the height direction response feature map, and the channel direction response feature map are weighted by the spatial significance response weight ω1 in the width direction, the spatial significance response weight ω2 in the height direction, and the spatial significance response weight ω3 in the channel direction to obtain a correction feature map F c =ω1F1+ω2F2+ω3F3.

[0048] Then use the correction characteristic map F c The point multiplication correction is performed on the local visual feature position saliency prediction map F of the human face to obtain a corrected local visual feature position saliency prediction map of the human face, thereby improving the directional adaptive response sensitivity in the joint association space and improving the global spatial saliency prediction coding expression effect of the local visual feature position saliency prediction map of the human face.

[0049] In step S33, the spatial saliency coding map of the local visual feature of the face is input into a segmentation model based on a classification layer to obtain an occlusion detection result. In one embodiment, the classification layer includes a point convolution layer and a Softmax classification function. The point convolution layer allows the model to act directly on each pixel, rather than the traditional convolution kernel sliding method, which enables it to process the information of each pixel more finely. The Softmax classification function is responsible for converting the probability distribution of each pixel into a specific category label, such as unoccluded, mask occluded, glasses occluded, etc. In this way, not only can the occluded areas be accurately marked, but also a detailed segmentation mask map can be provided to indicate which category each pixel belongs to. For example, when faced with the situation of wearing a mask, the model can accurately distinguish that the part below the nose belongs to the occluded area, while the part above the eyes may not be affected.

[0050] Illustratively, in step S4, based on the occlusion detection result, the unobstructed key areas in the face image are located. It should be understood that by locating the unobstructed key areas, the problem of feature loss due to occlusion can be effectively overcome. For example, under current public health measures, many passengers wear masks, which directly cover the part below the nose, making it difficult for traditional face recognition algorithms that rely on the entire facial features to work properly. If the system can accurately identify and utilize unobstructed key areas (such as the eyes and the surrounding areas), it can compensate for the information lost due to occlusion to a certain extent, thereby maintaining a higher recognition accuracy. This method not only improves the system's adaptability to different occlusion situations, but also ensures reliable operation even in complex environments.

[0051] In one embodiment, Figure 6 As shown, in step S4, based on the occlusion detection result, the unobstructed key area in the face image is located, including: S41, extracting the face segmentation mask image from the occlusion detection result; S42, extracting several key areas from the face image; S43, for each key area, querying its corresponding pixel in the face segmentation mask image; S44, calculating the proportion of pixels with a type label of non-occlusion in the key area, and determining whether the key area belongs to an unobstructed key area based on a comparison between the proportion and a preset threshold.

[0052] In this embodiment, the first step is to obtain a face segmentation mask after processing it with an occlusion detection model. This step relies on the deep learning model architecture used in the previous step, such as ResNet50 combined with a point convolutional layer and a Softmax classification function, to generate a mask that details the category (occluded or unoccluded) of each pixel. The next task is to extract multiple key regions from the face image. These regions typically correspond to important facial features, such as the eyes, nose, and mouth. For each selected key region, the corresponding pixel in the face segmentation mask is queried. This means mapping the coordinates of each pixel within the key region back to the mask and checking whether the label at that location is occluded or unoccluded. The proportion of pixels within the key region that are labeled as unoccluded is then calculated. If this proportion exceeds a pre-set threshold, the key region is considered to be not significantly occluded and can be used in the subsequent face recognition process. In a specific scenario of this embodiment, when a passenger is wearing a mask, only the eyes and the surrounding area may be considered as unoccluded key regions, while the nose and mouth are excluded. The advantage of this approach is that it can precisely locate suitable areas for authentication, maintaining high accuracy even in the presence of partial occlusion. Furthermore, by adjusting the threshold parameters, the sensitivity of the system can be flexibly controlled based on the needs of the actual application scenario, thereby maximizing the user experience while ensuring security.

[0053] In another embodiment, based on the occlusion detection result, locating the unobstructed key area in the facial image includes: extracting the face segmentation mask map from the occlusion detection result; calculating the position point multiplication between the face segmentation mask map and the facial image to obtain the unobstructed key area.

[0054] In this embodiment, a more direct approach is used to locate unobstructed key regions. The core idea here is to utilize a position-by-position dot product operation between the face segmentation mask and the original face image. Specifically, once the face segmentation mask generated by the occlusion detection model is obtained, it can be directly multiplied pixel by pixel with the original face image. The result of this operation is a new image that retains only those parts that were unobstructed in the original image, while all occluded areas are suppressed or converted to the background color. The advantages of this method lie in its simplicity and efficiency. Since each key region does not need to be defined and evaluated separately, the task can be completed through a one-time global operation, which greatly simplifies the algorithm flow and improves execution speed.

[0055] Exemplarily, in step S5, the unobstructed key area is input into the occlusion optimization feature extraction model to obtain a facial feature vector. It should be understood that a high-quality facial feature vector is the basis for achieving accurate identity matching. However, if features are extracted directly from an image containing occlusions, a large amount of invalid or low-quality information may be introduced, affecting the accuracy and stability of the feature representation. Through the occlusion optimization feature extraction model, the most discriminative features can be extracted from clearly visible unobstructed areas. For example, when a passenger wears a mask, the model can focus on the eyes and the area around them to ensure that the extracted features are sufficiently representative. This not only improves the quality of the feature vector, but also reduces the risk of misjudgment due to occlusion.

[0056] In one embodiment, an improved version of ResNet (such as ResNet50 or a deeper version) is used as the basic network structure. The network constructs a deep network by stacking multiple residual blocks, which can effectively capture complex patterns and detailed features in facial images while maintaining a low number of parameters. However, considering the negative impact that partial occlusion may have on feature extraction, the traditional ResNet model needs to be adjusted and optimized. Specifically, an attention mechanism (Attention Mechanism) can be introduced on the basis of the original network, such as the SENet (Squeeze-and-Excitation Networks) module or the CBAM (Convolutional Block Attention Module). These attention mechanisms can dynamically emphasize feature channels or spatial positions that are critical to the recognition task, while suppressing irrelevant interference information. In this way, even if some key parts are occluded, the model can still focus on other available effective information, thereby improving the quality of overall feature extraction.

[0057] Specifically, first, high-quality facial image fragments that have undergone pre-processing (including face detection, quality assessment, occlusion detection, and positioning of unobstructed key areas) are input into the above-mentioned optimized deep learning model. In this process, the input data is usually preprocessed into a standard format of fixed size (for example, 224x224 pixels) to meet the requirements of the model. After entering the model, the image data first undergoes preliminary feature extraction through a series of convolutional layers and pooling layers. These layers are responsible for capturing low-level visual features such as edges and textures. As the network layer deepens, more abstract and advanced semantic representations are gradually formed. These high-level features contain information about facial shape, expression, etc., which are crucial for distinguishing different individuals.

[0058] In particular, the effect of incorporating an attention mechanism is particularly pronounced during the occlusion-optimized feature extraction stage. Taking SENet as an example, it evaluates the importance of each feature channel and adaptively adjusts its weight, allowing the model to focus more on features that are helpful for the classification task while ignoring those that are significantly affected by occlusion or are insignificant. This mechanism not only enhances the model's robustness to occlusion but also improves the compactness and discriminability of the feature representation. Similarly, CBAM further expands the application scope of the attention mechanism by applying additional attention to the spatial dimension in addition to the channel dimension, thereby achieving more refined feature selection. Combining these two approaches can effectively alleviate the problem of feature loss caused by partial occlusion, ensuring the high reliability and consistency of the resulting facial feature vector.

[0059] After the feature extraction is completed, the next step is to reduce the dimensionality and standardize these high-dimensional features to facilitate subsequent identity comparison operations. In the present application, the feature map output by the last convolutional layer is converted into a one-dimensional vector form using a global average pooling layer, and then sent to a fully connected layer for linear transformation to obtain a face feature vector. In other embodiments, normalization techniques (such as L2 regularization) can also be applied to further standardize the scale of the feature vector so that it meets specific distance measurement requirements. The purpose of doing this is to ensure that the feature vectors between different samples can be compared under a unified standard to avoid misjudgment caused by excessive differences in numerical ranges.

[0060] Exemplarily, in step S6, the facial feature vector is compared with the facial template feature vector of the corresponding user in the database to obtain an identity authentication result, and based on the identity authentication result, the account status of the associated account is queried to see whether it is valid. It should be understood that by comparing the facial feature vector obtained in real time with the user's facial template feature vector pre-stored in the database, the accuracy of recognition can be ensured. Each facial template feature vector is carefully extracted and stored, representing the facial features of a specific user. When a passenger attempts to pass through the gate, the system captures his or her facial image in real time, extracts the corresponding feature vector, and matches it with the template in the database. This method can effectively identify the identity of the passenger and reduce the false recognition rate and missed recognition rate.

[0061] Specifically, before comparing facial feature vectors with template feature vectors in the database, an efficient database must be built to store and manage the template feature vectors of all registered users. This database must not only provide fast retrieval capabilities but also support high concurrent access, as large numbers of passengers may pass through the gates simultaneously during peak hours. To this end, a distributed database system such as Apache Cassandra or MongoDB can be used. These systems offer excellent scalability and fault tolerance, making them ideally suited to handling large-scale data storage needs.

[0062] In a specific embodiment, a metric such as cosine similarity or Euclidean distance is used to calculate the similarity between the input facial feature vector and the feature vector of each user template in the database. Taking cosine similarity as an example, it determines whether the directions of two vectors are consistent by measuring the cosine value of the angle between them. The closer the value is to 1, the higher the similarity. The advantage of this method is that it is not affected by the length of the feature vector and can improve the accuracy of the comparison to a certain extent.

[0063] In another specific embodiment, a Siamese network structure is used to perform facial feature comparison. The Siamese network consists of two sub-networks that share weights, and each sub-network is responsible for extracting a feature vector of a facial image. In the last layer of the network, the two feature vectors are input into a contrast loss function, which adjusts the network parameters based on whether the two inputs belong to the same person. In this way, the Siamese network is able to learn a more robust feature representation, so that a high recognition rate can be maintained even in the presence of partial occlusion. When a new user tries to pass through the gate, the system converts the captured facial image into a feature vector and uses the trained Siamese network to compare it one by one with the template feature vectors of all registered users in the database, and selects the match with the highest similarity as the candidate identity.

[0064] After determining the most likely identity, the system then queries the associated account's status based on the authentication result to determine if it is valid. This step involves interaction with the backend management system. Typically, a user's identity information and account status are recorded in a centralized management system, which can communicate with the front-end gate equipment through an API interface. When the gate receives the authentication result, it initiates a request to the backend management system to query the corresponding user's account details. If the account status is shown as valid, the gate will send an open command to allow the user to pass; otherwise, if there is a problem with the account (such as insufficient balance, account freeze, etc.), the user will be prompted to resolve the problem and try again.

[0065] For example, in step S7, in response to a successful identity authentication result and a valid account status, an opening command signal is sent to the gate control module. It should be understood that once the user's identity is confirmed and their account status is valid, the backend system generates a command signal in a specific format containing information allowing passage. To ensure communication security, this command signal is typically encrypted to prevent tampering or theft during transmission.

[0066] In a specific embodiment, a message queue (such as RabbitMQ or Kafka) can be used as middleware to implement asynchronous communication between the front-end and back-end. When the back-end system receives the notification of successful identity authentication and verifies that the account status is valid, it will create a message with the necessary information and publish it to a predefined message queue. The front-end device or the controller directly connected to the gate subscribes to this message queue. Once a new message arrives, it will immediately read the message content and parse the instructions therein. The advantage of this approach is that it can effectively decouple the front-end and back-end systems and improve the scalability and fault tolerance of the system. Even in high concurrency situations, it can ensure that each instruction can be delivered to the target device in a timely and accurate manner.

[0067] In addition, considering the needs of actual application scenarios, redundancy mechanisms can be introduced to further enhance system stability. For example, a backup power supply and fault recovery program can be set up within the gate control module to prevent single point failures from causing the gate to malfunction. At the same time, to improve the user experience, indicator lights or display screens can be installed near the gate to display the current operation status in real time (such as "Verifying", "Please Wait", "Pass Permit", etc.), allowing passengers to intuitively understand the progress of the entire process.

[0068] In summary, the face recognition-based intelligent release method for subway gates provided in this application first obtains a sequence of image frames containing faces through a camera, and uses advanced image processing technology to screen and enhance facial images of qualified quality to ensure the accuracy of the subsequent recognition process. In response to the occlusion problem, a convolutional neural network model is used for occlusion detection to accurately locate key facial areas that are not obscured, thereby avoiding recognition failures caused by the lack of some facial features. Next, these key areas are input into a specially optimized feature extraction model to generate facial feature vectors, which are compared with the pre-stored facial template features in the database to complete identity authentication. Finally, based on the identity authentication results and the validity of the associated account status, it is decided whether to send a gate opening command to achieve fast and safe automatic passage control. This method not only improves recognition efficiency and accuracy, but also enhances the stability of the system and user experience.

[0069] This application also provides a subway gate intelligent release system based on face recognition, such as Figure 7 As shown, the face recognition-based intelligent subway gate release system 600 includes: an image frame sequence acquisition module 610 for acquiring a sequence of image frames containing faces captured by a camera; a qualified face image extraction module 620 for extracting qualified face images from the sequence of image frames containing faces; an occlusion detection result determination module 630 for inputting the face images into an occlusion detection model to obtain an occlusion detection result; a key area positioning module 640 for locating unobstructed key areas in the face images based on the occlusion detection result; a face feature extraction module 650 for inputting the unobstructed key areas into an occlusion optimization feature extraction model to obtain a face feature vector; an identity authentication module 660 for comparing the face feature vector with the face template feature vector of the corresponding user in a database to obtain an identity authentication result, and querying whether the account status of the associated account is valid based on the identity authentication result; and a gate control instruction sending module 670 for sending an opening instruction signal to the gate control module in response to a successful identity authentication result and a valid account status.

[0070] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0071] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0072] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0073] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0074] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0075] Finally, it should be noted that the personal information and other data involved in this application have been obtained with full consent and authorization, and the collection, use and processing of relevant information comply with the relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A subway gate intelligent release method based on face recognition, characterized in that: include: Obtain a sequence of image frames containing human faces captured by a camera; Extracting a human face image of qualified quality from the image frame sequence containing the human face; Inputting the face image into an occlusion detection model to obtain an occlusion detection result; Based on the occlusion detection result, locating unoccluded key areas in the facial image; Inputting the unobstructed key area into an occlusion optimization feature extraction model to obtain a facial feature vector; Comparing the facial feature vector with a facial template feature vector of a corresponding user in a database to obtain an identity authentication result, and querying whether the account status of the associated account is valid based on the identity authentication result; In response to the identity authentication result being successful and the account status being valid, an opening instruction signal is sent to the gate control module.

2. The intelligent release method for subway gates based on face recognition according to claim 1 is characterized in that: Extracting a face image of qualified quality from the image frame sequence containing the face comprises: Performing face detection on the image frame sequence containing the face to obtain a face ROI image frame sequence; Performing image quality assessment on each face ROI image frame in the face ROI image frame sequence to obtain a sequence of face image quality assessment results; Selecting a face ROI image corresponding to the best face image quality assessment result in the sequence of face image quality assessment results as a face target image; Image enhancement is performed on the target face image to obtain the face image of qualified quality.

3. The intelligent release method for subway gates based on face recognition according to claim 2 is characterized in that: Inputting the face image into an occlusion detection model to obtain an occlusion detection result, including: Inputting the facial image into a facial feature extractor based on a convolutional neural network model to obtain a facial local visual feature encoding map; Adaptively performing spatial saliency on the facial local visual feature coding map to obtain a facial local visual feature spatial saliency coding map; The facial local visual feature spatial saliency coding map is input into a segmentation model based on a classification layer to obtain an occlusion detection result, wherein the occlusion detection result includes an occlusion type label and a facial segmentation mask map.

4. The intelligent release method for subway gates based on face recognition according to claim 3 is characterized in that: Adaptively performing spatial saliency on the facial local visual feature coding map to obtain a facial local visual feature spatial saliency coding map, comprising: Performing spatial position saliency prediction based on dilated convolutional coding on the local visual feature coding map of the human face to obtain a local visual feature position saliency prediction map of the human face; Inputting the facial local visual feature position saliency prediction map into an activation layer to obtain a facial local visual feature position saliency prediction probability map; The position-wise multiplication between the local visual feature position saliency prediction probability map of the human face and the local visual feature coding map of the human face is calculated to obtain the spatial saliency coding map of the local visual feature of the human face.

5. The method and system for intelligent release of subway gates based on face recognition according to claim 3 is characterized in that: Adaptively performing spatial saliency on the facial local visual feature coding map to obtain a facial local visual feature spatial saliency coding map, comprising: Performing spatial position saliency prediction based on dilated convolutional coding on the local visual feature coding map of the human face to obtain a local visual feature position saliency prediction map of the human face; Performing direction-adaptive response sensitivity correction on the facial local visual feature position saliency prediction map to obtain a corrected facial local visual feature position saliency prediction map; Inputting the corrected facial local visual feature position saliency prediction map into an activation layer to obtain a facial local visual feature position saliency prediction probability map; The position-wise multiplication between the local visual feature position saliency prediction probability map of the human face and the local visual feature coding map of the human face is calculated to obtain the spatial saliency coding map of the local visual feature of the human face.

6. The method and system for intelligent release of subway gates based on face recognition according to claim 5 is characterized in that: Performing direction-adaptive response sensitivity correction on the facial local visual feature position saliency prediction map to obtain a corrected facial local visual feature position saliency prediction map, comprising: Performing mean dimension compression on the facial local visual feature position saliency prediction map along the width dimension, the height dimension, and the channel dimension to obtain a width-wise feature vector, a height-wise feature vector, and a channel-wise feature vector; Performing matrix multiplication on the width direction feature vector, the height direction feature vector, and the channel direction feature vector with the facial local visual feature position saliency prediction map in corresponding dimensions to obtain a width direction response feature matrix, a height direction response feature matrix, and a channel direction response feature matrix; Based on the width direction feature vector, the height direction feature vector and the channel direction feature vector, calculating the spatial significance response weight in each direction to obtain the spatial significance response weight in the width direction, the spatial significance response weight in the height direction and the spatial significance response weight in the channel direction; Performing matrix point multiplication on the width direction response feature matrix, the height direction response feature matrix, and the channel direction response feature matrix with the face local visual feature position saliency prediction map in corresponding dimensions to obtain a width direction response feature map, a height direction response feature map, and a channel direction response feature map; weighting the width direction response feature map, the height direction response feature map, and the channel direction response feature map by the spatial significance response weight in the width direction, the spatial significance response weight in the height direction, and the spatial significance response weight in the channel direction to obtain a correction feature map; The correction feature map is used to perform point multiplication correction on the local visual feature position saliency prediction map of the human face to obtain a corrected local visual feature position saliency prediction map of the human face.

7. The intelligent release method for subway gates based on face recognition according to claim 4 or 5, characterized in that: The classification layer includes a point convolution layer and a Softmax classification function.

8. The intelligent release method for subway gates based on face recognition according to claim 3 is characterized in that: Locating an unobstructed key area in the face image based on the occlusion detection result includes: Extracting the face segmentation mask image from the occlusion detection result; extracting a plurality of key areas from the face image; For each key area, query the corresponding pixel in the face segmentation mask image; The proportion of pixels with a type label of non-occluders in the key area is calculated, and based on a comparison between the proportion and a preset threshold, it is determined whether the key area belongs to an unoccluded key area.

9. The intelligent release method for subway gates based on face recognition according to claim 3 is characterized in that: Locating an unobstructed key area in the face image based on the occlusion detection result includes: Extracting the face segmentation mask image from the occlusion detection result; The unobstructed key area is obtained by calculating the point-by-point multiplication between the face segmentation mask image and the face image.

10. A subway gate intelligent release system based on face recognition, characterized in that: include: An image frame sequence acquisition module is used to acquire an image frame sequence containing a human face captured by a camera; A qualified face image extraction module, configured to extract a face image of qualified quality from the sequence of image frames containing faces; an occlusion detection result determination module, configured to input the face image into an occlusion detection model to obtain an occlusion detection result; A key area positioning module, configured to locate unobstructed key areas in the face image based on the occlusion detection result; A facial feature extraction module, configured to input the unobstructed key area into an occlusion optimization feature extraction model to obtain a facial feature vector; an identity authentication module, configured to compare the facial feature vector with a facial template feature vector of a corresponding user in a database to obtain an identity authentication result, and query whether the account status of the associated account is valid based on the identity authentication result; The gate control instruction sending module is used to send an opening instruction signal to the gate control module in response to the identity authentication result being successful and the account status being valid.

Images