Face recognition area network intelligent monitoring system and method

Abstract

The invention relates to the technical field of face recognition, in particular to a face recognition area network intelligent monitoring system and method. According to the technical scheme, the system comprises an image acquisition module, a data preprocessing module, a feature extraction module, a face recognition module, a behavior analysis module and a data storage and transmission module. The face recognition accuracy is improved from multiple aspects through multiple algorithms, the false recognition rate is reduced, by means of multi-module collaborative preprocessing and multi-scale feature fusion, the adaptability of the system to a complex environment and the robustness of the system in different scenes are enhanced, meanwhile, the image quality is improved by optimizing image acquisition equipment, and the face recognition accuracy is improved. And the behavior analysis module is used for carrying out target detection and trajectory tracking to analyze personnel behaviors in real time and early warning abnormity, so that the overall performance of the system is improved, and the intelligent level and practicability are improved.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face recognition regional network intelligent monitoring system and method. Background Art

[0002] With the rapid development of science and technology, the demand for intelligent, efficient and accurate security monitoring is growing. As an important research direction in the field of biometrics, face recognition technology has unique advantages in identity recognition. Compared with other recognition methods, such as fingerprint recognition and iris recognition, face recognition does not require the active cooperation of the object to be recognized, and can perform identity authentication in a natural state at a long distance and contactlessly. However, existing face recognition monitoring systems still face many challenges.

[0003] On the one hand, lighting conditions vary widely in actual monitoring environments. Lighting changes will change the pixel value distribution of facial images, making it difficult for traditional face recognition algorithms to accurately extract stable facial features, resulting in a significant drop in recognition accuracy. On the other hand, when the monitored person is walking, turning his head, or lowering his head, his face will show different pitch, head tilt and rotation angles. Posture changes will cause the relative positions of key feature points of the face to change, and some facial areas will be blocked or deformed, making traditional recognition algorithms based on frontal face model training invalid and the recognition effect poor. Therefore, we propose a face recognition regional network intelligent monitoring system and method. Summary of the invention

[0004] The purpose of the present invention is to address the problem in the background technology that changes in lighting conditions and diversity of facial postures in face recognition can lead to poor recognition results, and to propose a face recognition area network intelligent monitoring system and method.

[0005] First aspect: This application provides a face recognition area network intelligent monitoring system, including:

[0006] Image acquisition module, data preprocessing module, feature extraction module, face recognition module, behavior analysis module and data storage and transmission module;

[0007] The image acquisition module is used to acquire image data;

[0008] The data preprocessing module is used to perform illumination compensation, posture correction and occlusion processing on the collected image data, and to crop and scale the processed image to extract the face area image and transmit it to the feature extraction module;

[0009] The feature extraction module is used to extract facial features and generate feature vectors;

[0010] The face recognition module performs face recognition matching by calculating feature vectors, and transmits the recognition results to the behavior analysis module and the data storage and transmission module;

[0011] The behavior analysis module analyzes the behavior of personnel in the monitoring area and transmits the behavior analysis results to the data storage and transmission module;

[0012] The data storage and transmission module is used to receive and store data from the face recognition module and the behavior analysis module.

[0013] Optionally, the image acquisition module includes multiple groups of cameras distributed in the monitoring area for collecting data, and the cameras use a resolution of not less than 1920×1080, a wide dynamic range of 120dB, and automatic focus and image stabilization functions.

[0014] Optionally, the data preprocessing module includes an illumination processing unit, a posture correction unit and an occlusion processing unit;

[0015] The illumination processing unit uses an adaptive histogram equalization algorithm to perform illumination compensation on the image;

[0016] The posture correction unit determines the facial posture parameters through feature point matching and performs projection correction by using a posture correction algorithm based on a 3D facial model;

[0017] The occlusion processing unit processes the occlusion situation through an occlusion detection and feature reconstruction algorithm based on deep learning, and then crops and scales the processed image to extract the face area image and transmits it to the feature extraction module.

[0018] Optionally, the adaptive histogram equalization algorithm includes calculating the cumulative distribution function of the image data collected by the image acquisition module, and the function is as follows:

[0019]

[0020] And based on

[0021] J(x,y)=CDF I [I(x,y)]×L

[0022] Adjusting the brightness of the image;

[0023] Where [I(x,y)] is the original image, [m,n] is its grayscale range, P I (i) is the probability distribution of gray level i, L is the number of gray levels of the output image;

[0024] The posture correction algorithm determines the face posture parameter θ by matching feature points. x ,θ y,θ z and the translation parameter t x ,t y ,t z , according to the formula:

[0025]

[0026] The 3D model is projected onto the 2D image plane for rectification, where R is the rotation matrix constructed from the pose angles and T is the translation vector [t x ,t y ,t z ] T , (x, y, z) are the coordinates of the point on the 3D face model, and (x', y', z') are the coordinates of the corresponding point after correction.

[0027] Optionally, the deep learning-based occlusion detection and feature reconstruction algorithm includes:

[0028] Occlusion detection: Occlusion detection is performed using a convolutional neural network, which has multiple convolutional layers, pooling layers, and fully connected layers. The convolution kernel size includes 3x3 and 5x5 specifications, with a step size of 1 or 2. The pooling layer uses a maximum pooling operation with a kernel size of 2x2 and a step size of 2. The convolution layer calculation follows the formula:

[0029]

[0030] Among them, C l (x,y) is the output feature map of the lth convolutional layer at coordinate (x,y), W l (i, j) is the weight of the l-th layer convolution kernel at coordinate (i, j), C l-1 (x, y) is the output feature map of the l-1th layer. For the first convolutional layer, C l-1 (x, y) is the input image I(x, y), b l is the bias of the lth layer, k is the convolution kernel size, and classification is performed through the fully connected layer to determine whether there is occlusion in the face of the image and the location information of the occluded area;

[0031] Feature reconstruction: The detected occluded part is reconstructed using a generative adversarial network. The generative adversarial network includes a generative network G and a discriminative network D. The generative network G takes a random noise vector z as input and generates a feature map G(z) similar to the real unoccluded face features through a deconvolution layer. The calculation formula of the deconvolution layer is as follows:

[0032]

[0033] Among them, U l(x,y) is the output feature map of the deconvolution layer at the coordinate (x,y) of the generative network. is the weight of the l-th layer deconvolution kernel at coordinate (i, j), U l-1 (x, y) is the output feature map of the l+1th layer. For the last deconvolution layer, the output is the generated feature map G(z). is the bias of the lth layer, s is the step size of the deconvolution;

[0034] The discriminant network D includes a convolutional layer, a pooling layer and a fully connected layer. The convolutional layer distinguishes the real feature x from the generated feature G(z) by extracting and classifying the input features. During the training process, the generated network G and the discriminant network D are trained adversarially, and the adversarial loss function is expressed as:

[0035]

[0036] in, represents the mathematical expectation, P data (x) is the distribution of real data, which is the unobstructed face features, P z (z) is the distribution of the random noise vector z. The parameters of the generation network G and the discriminant network D are optimized through iterative training, so that the generation network G generates features for replacing the occluded part.

[0037] Optionally, the feature extraction module includes a multi-scale feature extraction unit, the multi-scale feature extraction unit includes a deep residual network and a local binary pattern feature, and the deep residual network output feature map is fused with the local binary pattern feature to obtain a final feature vector;

[0038] The deep residual network uses the Res Net architecture to extract deep semantic features. Res Net contains multiple residual blocks, each of which is composed of multiple convolutional layers, including at least one shortcut connection to solve the gradient vanishing problem of the deep network. Suppose the feature map input to a residual block is X in , the output feature map after the convolution operation is F(X in ), then the output of the residual block is X out =F(X in )+X in , where the output feature map of a layer of the Res Net network is F res (x, y), whose dimension is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively;

[0039] The local binary pattern feature part calculates the feature F for the face image lbp(x, y), for each pixel point P(x, y) in the image, select P pixels in a circular neighborhood with a radius of R as the center, compare the grayscale value of the pixels in the neighborhood with the grayscale value of the central pixel, if it is greater than or equal to the grayscale value of the central pixel, mark it as 1, otherwise mark it as 0, and arrange it in clockwise or counterclockwise order to get a P-bit binary number, which is converted into a decimal number as the LBP value of the pixel point. The formula is expressed as:

[0040]

[0041] Among them, g p is the gray value of the pth pixel in the neighborhood, g c is the gray value of the center pixel;

[0042]

[0043] By calculating the LBP value of the entire face image, F lbp (x,y);

[0044] Feature fusion part: The feature map F extracted by the deep residual network res (x,y) and local binary pattern feature F lbp (x, y) is fused to obtain the final feature vector F = [F res ,F lbp ].

[0045] Optionally, the face recognition module includes a recognition unit based on metric learning and a face feature library unit;

[0046] The recognition unit based on metric learning uses the Triplet Loss function for model training. The Triplet Loss function is defined as:

[0047] L = max(d(x i ,x j )-d(x i ,x k )+α,0)

[0048] Among them, x i , x j , x k is a face sample, d(x i ,x j and d(x i ,x k ) represent the samples x i With x j 、x i With x k The feature distance, α is the margin parameter;

[0049] The facial feature library unit is used to store facial features of registered persons and adopts a hash-based index structure for matching and retrieval.

[0050] Optionally, the behavior analysis module includes a target detection and tracking unit and a crowd behavior analysis unit, wherein the crowd behavior analysis unit calculates the density distribution of personnel, the speed mean and variance, and the distance relationship between personnel based on the trajectory data, and determines abnormal behavior and issues an early warning by setting a threshold;

[0051] The target detection and tracking unit uses the YOLO algorithm to detect the target and obtain the personnel position information P (x, y, w, h), and uses the Kalman filter algorithm to track the trajectory, and updates the equation through state prediction and measurement:

[0052]

[0053] Among them, F k is the state transfer matrix, B k is the control input matrix, u k is the control input vector, P k|k-1 and P kk are the prediction covariance matrix and the update covariance matrix, K k is the Kalman gain, H k is the measurement matrix, Q k is the process noise covariance matrix, R k is the measurement noise covariance matrix, z k It is the measurement vector, i.e. the position information obtained by target detection.

[0054] Optionally, the data storage and transmission module includes an encrypted storage unit, a secure transmission unit and a data backup unit. The encrypted storage unit uses an AES encryption algorithm to encrypt the facial data for storage, the secure transmission unit uses an SSL / TLS protocol to encrypt and transmit the data, and the data backup unit is used to periodically back up the data to an off-site storage device.

[0055] Second aspect: This application provides a face recognition area network intelligent monitoring method, comprising the following steps:

[0056] Image acquisition, collecting the image data of the monitoring area through the image acquisition module and transmitting it to the data preprocessing module;

[0057] Data preprocessing, the data preprocessing module performs illumination compensation, posture correction and occlusion processing on the image, crops and scales to extract the face area image, and transmits the processed image cropped and scaled to extract the face area image to the feature extraction module;

[0058] Feature extraction, the feature extraction module extracts feature vectors from the face area image using the multi-scale feature extraction unit;

[0059] Face recognition: Match the extracted feature vector with the features in the face feature library through the face recognition module, calculate the feature distance based on the model trained by the Triplet Loss function and determine whether the recognition is successful. If successful, the person's identity information is obtained; if failed, it is marked as an unknown identity and further analyzed;

[0060] Behavior analysis: Target detection and trajectory tracking of people in the monitoring area are performed through the behavior analysis module, crowd behavior analysis is performed based on trajectory data, and abnormal behavior is judged;

[0061] Result output and storage: face recognition results and behavior analysis results are displayed and stored through the data storage and transmission module, and an alarm is issued in case of abnormal behavior. The stored data includes original image data, processing process data, recognition result data and behavior analysis data.

[0062] In summary, the present application includes at least one of the following beneficial technical effects:

[0063] The present invention uses an adaptive histogram equalization algorithm to perform illumination compensation on images under different illumination conditions, thereby enhancing image contrast and clarity and reducing recognition errors caused by illumination changes. At the same time, a posture correction algorithm based on a 3D face model can determine and correct facial posture parameters to make the face as close to the frontal posture as possible, effectively solving the problem of feature extraction being affected by changes in facial pitch, rotation and other postures, thereby improving recognition accuracy. In addition, a deep learning occlusion detection and feature reconstruction algorithm is used for feature reconstruction, so that the system can still accurately extract features for recognition when part of the face is occluded. The deep residual network and local binary pattern features are combined to enhance the robustness of the features, improve the recognition of facial features under different conditions, and thus improve the accuracy of face recognition. The face recognition model is trained through a TripletLoss function, and the distance between facial features of the same identity and different identities is reasonably adjusted to further improve the accuracy of face recognition by the system and reduce the misrecognition rate.

[0064] The multi-module collaborative preprocessing in the present invention enables the subsequent feature extraction and recognition process to be carried out on the basis of relatively stable image data, thereby enhancing the system's adaptability to complex environments. By fusing the deep residual network and local binary pattern features through the multi-scale feature extraction unit, the system can extract effective features when facing face images of different resolutions and qualities, thereby improving the robustness of the system in different scenarios.

[0065] The present invention improves the image quality from the source by optimizing the image acquisition equipment and adopting a high-resolution, wide dynamic range camera, providing clear and stable image data for subsequent processing, which helps to improve the performance of the entire system. At the same time, the behavior analysis module performs target detection through the YOLO algorithm and trajectory tracking through the Kalman filter algorithm, and can analyze the behavior of people in the monitoring area in real time, including personnel density, speed, distance relationship, etc., and can detect abnormal behavior in time and issue warnings, thereby improving the intelligence level and practicality of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A principle block diagram of a face recognition area network intelligent monitoring system of the present invention is given. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example

[0069] On the one hand, the present application provides a face recognition area network intelligent monitoring method, comprising the following steps:

[0070] Image acquisition: The image acquisition module collects the image data of the monitored area and transmits it to the data preprocessing module;

[0071] Data preprocessing: The data preprocessing module performs illumination compensation, posture correction and occlusion processing on the image, crops and scales to extract the face area image, and transmits the processed image cropped and scaled to extract the face area image to the feature extraction module;

[0072] Feature extraction: The feature extraction module uses a multi-scale feature extraction unit to extract feature vectors from the face area image;

[0073] Face recognition: The extracted feature vector is matched with the features in the face feature library through the face recognition module. The feature distance is calculated based on the model trained by the Triplet Loss function to determine whether the recognition is successful. If successful, the person's identity information is obtained. If failed, it is marked as an unknown identity and further analyzed.

[0074] Behavior analysis: The behavior analysis module detects targets and tracks trajectories of people in the monitoring area, analyzes crowd behavior based on trajectory data, and determines abnormal behavior;

[0075] Result output and storage: The face recognition results and behavior analysis results are displayed and stored through the data storage and transmission module, and an alarm is issued in case of abnormal behavior. The stored data includes original image data, processing process data, recognition result data and behavior analysis data.

[0076] On the other hand, the present application provides a face recognition area network intelligent monitoring system, such as Figure 1 As shown, it includes an image acquisition module, a data preprocessing module, a feature extraction module, a face recognition module, a behavior analysis module, and a data storage and transmission module;

[0077] The image acquisition module includes multiple groups of cameras distributed in the monitoring area for data collection. The cameras use a resolution of no less than 2560×1920 pixels, a wide dynamic range of 120dB, and autofocus and image stabilization functions. In a 1,000-square-meter shopping mall monitoring scene, a total of 20 cameras are installed at key locations such as the mall entrance, passages, and elevator entrances to ensure that the monitoring area is covered without blind spots.

[0078] The data preprocessing module includes an illumination processing unit, a posture correction unit and an occlusion processing unit;

[0079] The adaptive histogram equalization algorithm is implemented in the light processing unit through software programming. A face image under strong light at the entrance of a shopping mall is collected. The grayscale range of the original image is [0,255]. First, the cumulative distribution function CDF of the image is calculated. I (k), the formula is as follows:

[0080]

[0081] For gray level k, by counting the number of pixels at each gray level and calculating the probability distribution P I (i) Then according to J(x,y) = CDF I [I(x,y)]×L adjusts the brightness of the image. After the adjustment, the clarity of the details of the face area that was originally too bright due to strong light is improved, and the shadow part is also compensated, which enhances the contrast of the image and reduces the recognition errors caused by lighting changes;

[0082] The posture correction unit determines the facial posture parameters and projects the correction through the posture correction algorithm based on the 3D face model through feature point matching. When a face image of a person walking in a shopping mall with his head tilted about 30 degrees to the left is detected, the posture parameter θ of the face in the image is determined through feature point matching, which includes eyes, mouth corners, and nose. x =0,θ y =30°,θ z = 0 and translation parameter t x,t y ,t x , the posture parameters are calculated according to the relationship between the image coordinates and the actual space coordinates, and then according to the formula:

[0083]

[0084] The 3D model is projected onto the 2D image plane for correction. The corrected face image is close to the frontal posture, which is convenient for subsequent feature extraction.

[0085] The occlusion processing unit processes the occlusion situation through the occlusion detection and feature reconstruction algorithm based on deep learning, and then crops and scales the processed image to extract the face area image and transmits it to the feature extraction module, wherein the occlusion detection and feature reconstruction algorithm based on deep learning includes occlusion test and feature reconstruction;

[0086] Occlusion test: Convolutional neural network is used for occlusion detection. The convolutional neural network structure contains 5 convolutional layers. The convolution kernel sizes are 3x3 and 5x5, respectively. The step size is set to 1 or 2. The pooling layer uses the maximum pooling operation, the kernel size is 2x2, and the step size is 2. The calculation formula of the convolution layer is as follows:

[0087]

[0088] Among them, C l (x,y) is the output feature map of the lth convolutional layer at coordinate (x,y), W l (i, j) is the weight of the l-th layer convolution kernel at coordinate (i, j), C l-1 (x, y) is the output feature map of the l-1th layer. For the first convolutional layer, C l-1 (x, y) is the input image I(x, y), b l is the bias of the lth layer, k is the convolution kernel size, and classification is performed through the fully connected layer to determine whether the face in the image is blocked and the location information of the blocked area. For an image of a face wearing a mask in a shopping mall, after being input into the convolutional neural network, it undergoes feature extraction through multiple convolutional layers and pooling layers, and finally is classified through the fully connected layer to detect the area where the face is blocked by the mask;

[0089] Feature reconstruction: For the detected occluded part, the generative adversarial network is used to reconstruct the features. The generative adversarial network includes a generative network G and a discriminative network D. The generative network G takes the random noise vector z as input and generates a feature map G(z) similar to the real unoccluded face features through the deconvolution layer. The calculation formula of the deconvolution layer is as follows:

[0090]

[0091] Among them, U l(x,y) is the output feature map of the deconvolution layer at the coordinate (x,y) of the generative network. is the weight of the l-th layer deconvolution kernel at coordinate (i, j), U l-1 (x, y) is the output feature map of the l+1th layer. For the last deconvolution layer, the output is the generated feature map G(z). is the bias of the lth layer, s is the step size of the deconvolution;

[0092] The discriminant network D contains convolutional layers, pooling layers, and fully connected layers. Its convolutional layer extracts and classifies the input features to distinguish between the real features x and the generated features G(z). During the training process, the generated network G and the discriminant network D are trained adversarially, and the adversarial loss function is expressed as:

[0093]

[0094] in, represents the mathematical expectation, P data (x) is the distribution of real data, which is the unobstructed face features, P z (z) is the distribution of the random noise vector z. The parameters of the generative network G and the discriminative network D are optimized through iterative training, so that the generative network G generates features for replacing the mask-occluded parts;

[0095] The feature extraction module includes a multi-scale feature extraction unit, which includes a deep residual network and a local binary pattern feature. The output feature map of the deep residual network is fused with the local binary pattern feature to obtain a final feature vector.

[0096] The deep residual network adopts the ResNet-50 architecture. When a face image of a person in a shopping mall is input after being preprocessed by the data preprocessing module, the image first enters the first convolutional layer of ResNet-50 and then is processed by multiple residual blocks. Suppose the feature map input to a residual block is X in , the output feature map after the convolution layer and other operations is F(X in ), the output of the residual block is X out =F(X in )+X in Through this residual structure, the multi-scale features of face images can be effectively extracted at different network depths, and valuable feature information can be adaptively extracted when face images with different resolutions are input, and finally the output feature map F of a certain layer of the Res Net network is obtained. res (x, y), whose dimension is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively;

[0097] The local binary pattern feature calculates the LBP feature for the above face image. Taking each pixel point P(x, y) in the image as the center, select P pixels in the circular neighborhood with a radius of R, and compare the grayscale value of the pixels in the neighborhood with the grayscale value of the central pixel. If it is greater than or equal to the grayscale value of the central pixel, it is marked as 1, otherwise it is marked as 0. Arrange it in clockwise order to get an 8-bit binary number, which is converted into a decimal number as the LBP value of the pixel. The formula is expressed as:

[0098]

[0099] By calculating the LBP value of the entire face image, F lbp (x,y);

[0100] The feature map F extracted by the deep residual network res (x,y) and local binary pattern feature F lbp (x, y) is fused to obtain the final feature vector F = [F res ,F lbp ];

[0101] The face recognition module includes a recognition unit based on metric learning and a face feature library unit;

[0102] The recognition unit based on metric learning uses the Triplet Loss function to train the model, collects face images of employees and frequent customers in the mall as training samples, and constructs triplet sample pairs, namely x i , x j , x k , where x i and x j For faces of the same identity, x i and x k For faces of different identities, the TripletLoss function is defined as:

[0103] L = max(d(x i ,x j )-d(x i ,x k )+α,0)

[0104] Assume the margin parameter α = 0.5. Through training with a large number of sample pairs, the distance between facial features of the same identity is made as small as possible, and the distance between facial features of different identities is greater than the margin parameter α.

[0105] Establish a facial feature database to store the facial features of registered people in the mall and use an efficient hash-based index structure for fast matching and retrieval. When a customer enters the mall, extract the facial feature vector and match it with the features in the feature database, calculate the feature distance, and determine whether the recognition is successful based on the set threshold. If the recognition is successful, obtain the corresponding person's identity information, which includes membership level and consumption preferences, so that the mall can provide personalized services; if the recognition fails, mark the facial feature as an unknown identity and compare it with the suspect database for further analysis;

[0106] The behavior analysis module includes a target detection and tracking unit and a crowd behavior analysis unit;

[0107] The target detection and tracking unit uses the YOLOv5 algorithm for target detection. In the mall surveillance video, the YOLOv5 algorithm performs target detection to obtain the personnel position information P(x, y, w, h). In the crowd gathering area in the mall atrium, the position frame of each person is detected. Then the Kalman filter algorithm is used for trajectory tracking. Suppose the system state vector X = [x, y, v x ,v y ], the target person is at x=100, y=200, and the speed is v x =2, v y =1), update equation by state prediction and measurement:

[0108]

[0109] Among them, F k is the state transfer matrix, B k is the control input matrix, u k is the control input vector, P k|k-1 and P k|k are the prediction covariance matrix and the update covariance matrix, K k is the Kalman gain, H k is the measurement matrix, Q k is the process noise covariance matrix, R k is the measurement noise covariance matrix, z k It is the measurement vector, i.e. the position information obtained by target detection, which can track the trajectory of the target person;

[0110] The crowd behavior analysis unit calculates the population density distribution, speed mean and variance, and the distance relationship between people based on trajectory data. During the mall promotion, it is found that the population density in a certain area exceeds the set threshold of 5 people / square meter, the average speed of people is lower than the set threshold of 0.5 meters / second, and the average distance between people is less than the set threshold of 0.5 meters. It is judged that there is a risk of crowding and trampling, and an early warning is issued in time to notify the mall security personnel to check and deal with it;

[0111] The data storage and transmission module includes an encryption storage unit, a security transmission unit and a data backup unit;

[0112] The encrypted storage unit uses the AES-256 encryption algorithm to encrypt the facial data and then stores it in the storage server of the mall monitoring data center. Each facial feature data after recognition is encrypted to ensure the security of the data during storage.

[0113] The secure transmission unit uses the SSL / TLS protocol to encrypt data transmission. When the data collected by cameras in different areas of the mall is transmitted to the data center, or when the data center transmits the face recognition results and behavior analysis results to the mall management terminal, they are all encrypted through the SSL / TLS protocol to prevent the data from being stolen or tampered with during network transmission;

[0114] Through the data backup unit, the previous day's data is backed up to an off-site storage device at 2 a.m. every day. The backup content includes original image data, processing process data, recognition result data and behavior analysis data to prevent local data loss.

[0115] The present invention improves the accuracy of face recognition and reduces the false recognition rate from multiple aspects through multiple algorithms, including adaptive histogram equalization, posture correction, occlusion detection and feature reconstruction, and training models based on Triplet Loss functions. With the help of multi-module collaborative preprocessing and multi-scale feature fusion, the present invention enhances the system's adaptability to complex environments and robustness in different scenarios.

[0116] At the same time, the present invention relies on optimizing image acquisition equipment to improve image quality, and uses a behavior analysis module to perform target detection and trajectory tracking to analyze personnel behavior in real time and warn of abnormalities, thereby improving the overall performance of the system and enhancing the level of intelligence and practicality.

[0117] The above specific embodiments are only several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A face recognition area network intelligent monitoring system, characterized in that: It includes image acquisition module, data preprocessing module, feature extraction module, face recognition module, behavior analysis module and data storage and transmission module; The image acquisition module is used to acquire image data; The data preprocessing module is used to perform illumination compensation, posture correction and occlusion processing on the collected image data, and to crop and scale the processed image to extract the face area image and transmit it to the feature extraction module; The feature extraction module is used to extract facial features and generate feature vectors; The face recognition module performs face recognition matching by calculating feature vectors, and transmits the recognition results to the behavior analysis module and the data storage and transmission module; The behavior analysis module analyzes the behavior of personnel in the monitoring area and transmits the behavior analysis results to the data storage and transmission module; The data storage and transmission module is used to receive and store data from the face recognition module and the behavior analysis module.

2. The face recognition area network intelligent monitoring system according to claim 1 is characterized in that: The image acquisition module includes a plurality of cameras distributed in the monitoring area for acquiring data. The cameras have a resolution of not less than 1920×1080, a wide dynamic range of 120 dB, and automatic focus and image stabilization functions.

3. The face recognition area network intelligent monitoring system according to claim 1 is characterized in that: The data preprocessing module includes an illumination processing unit, a posture correction unit and an occlusion processing unit; The illumination processing unit uses an adaptive histogram equalization algorithm to perform illumination compensation on the image; The posture correction unit determines the facial posture parameters through feature point matching and performs projection correction through a posture correction algorithm based on a 3D facial model; The occlusion processing unit processes the occlusion situation through an occlusion detection and feature reconstruction algorithm based on deep learning, and then crops and scales the processed image to extract the face area image and transmits it to the feature extraction module.

4. The face recognition area network intelligent monitoring system according to claim 3 is characterized in that: The adaptive histogram equalization algorithm includes calculating the cumulative distribution function of the image data collected by the image acquisition module, and the function is as follows: And based on J(x,y)=CDF I [I(x,y)]×L Adjusting the brightness of the image; Where [I(x,y)] is the original image, [m,n] is its grayscale range, P I (i) is the probability distribution of gray level i, L is the number of gray levels of the output image; The posture correction algorithm determines the face posture parameter θ by matching feature points. x ,θ y ,θ z and the translation parameter t x ,t y ,t z , according to the formula: The 3D model is projected onto the 2D image plane for rectification, where R is the rotation matrix constructed from the pose angles and T is the translation vector [t x ,t y ,t z ] T , (x, y, z) are the coordinates of the point on the 3D face model, and (x', y', z') are the coordinates of the corresponding point after correction.

5. The face recognition area network intelligent monitoring system according to claim 3 is characterized in that: The deep learning-based occlusion detection and feature reconstruction algorithm includes: Occlusion detection: A convolutional neural network is used for occlusion detection. The convolutional neural network has multiple convolutional layers, pooling layers, and fully connected layers. The convolution kernel size includes 3x3 and 5x5 specifications, with a step size of 1 or 2. The pooling layer uses the maximum pooling operation and the kernel size is 2x2 with a step size of 2. The convolution layer calculation formula is as follows: Among them, C l (x,y) is the output feature map of the lth convolutional layer at coordinate (x,y), W l (i, j) is the weight of the l-th layer convolution kernel at coordinate (i, j), C l-1 (x, y) is the output feature map of the l-1th layer. For the first convolutional layer, C l-1 (x, y) is the input image I(x, y), b l is the bias of the lth layer, k is the convolution kernel size, and classification is performed through the fully connected layer to determine whether there is occlusion in the face of the image and the location information of the occluded area; Feature reconstruction: The detected occluded part is reconstructed using a generative adversarial network. The generative adversarial network includes a generative network G and a discriminative network D. The generative network G takes a random noise vector z as input and generates a feature map G(z) similar to the real unoccluded face features through a deconvolution layer. The calculation formula of the deconvolution layer is as follows: Among them, U l (x,y) is the output feature map of the deconvolution layer at the coordinate (x,y) of the generative network. is the weight of the l-th layer deconvolution kernel at coordinate (i, j), U l-1 (x, y) is the output feature map of the l+1th layer. For the last deconvolution layer, the output is the generated feature map G(z). is the bias of the lth layer, s is the step size of the deconvolution; The discriminant network D includes a convolutional layer, a pooling layer and a fully connected layer. The convolutional layer distinguishes the real feature x from the generated feature G(z) by extracting and classifying the input features. During the training process, the generated network G and the discriminant network D are trained adversarially, and the adversarial loss function is expressed as: in, represents the mathematical expectation, P data (x) is the distribution of real data, which is the unobstructed face features, P z (z) is the distribution of the random noise vector z. The parameters of the generation network G and the discriminant network D are optimized through iterative training, so that the generation network G generates features for replacing the occluded part.

6. The face recognition area network intelligent monitoring system according to claim 1, characterized in that: The feature extraction module includes a multi-scale feature extraction unit, which includes a deep residual network and a local binary pattern feature, and the output feature map of the deep residual network is fused with the local binary pattern feature to obtain a final feature vector; The deep residual network uses the Res Net architecture to extract deep semantic features. Res Net contains multiple residual blocks, each of which is composed of multiple convolutional layers, including at least one shortcut connection to solve the gradient vanishing problem of the deep network. Suppose the feature map input to a residual block is X in , the output feature map after the convolution operation is F(X in ), then the output of the residual block is X out =F(X in )+X in , where the output feature map of a layer of the Res Net network is F res (x, y), whose dimension is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively; The local binary pattern feature part calculates the feature F for the face image lbp (x, y), for each pixel point P(x, y) in the image, select P pixels in a circular neighborhood with a radius of R as the center, compare the grayscale value of the pixels in the neighborhood with the grayscale value of the central pixel, if it is greater than or equal to the grayscale value of the central pixel, mark it as 1, otherwise mark it as 0, and arrange it in clockwise or counterclockwise order to get a P-bit binary number, which is converted into a decimal number as the LBP value of the pixel point. The formula is expressed as: Among them, g p is the gray value of the pth pixel in the neighborhood, g c is the gray value of the center pixel; By calculating the LBP value of the entire face image, F lbp (x,y); Feature fusion part: The feature map F extracted by the deep residual network res (x,y) and local binary pattern feature F lbp (x, y) is fused to obtain the final feature vector F = [F res ,F lbp ].

7. The face recognition area network intelligent monitoring system according to claim 1, characterized in that: The face recognition module includes a recognition unit based on metric learning and a face feature library unit; The recognition unit based on metric learning uses the Triplet Loss function for model training. The Triplet Loss function is defined as: L=max(d(x i ,x j )-d(x i ,x k )+α,0) Among them, x i , x j , x k is a face sample, d(x i ,x j and d(x i ,x k ) represent the samples x i With x j 、x i With x k The feature distance, α is the margin parameter; The facial feature library unit is used to store facial features of registered persons and adopts a hash-based index structure for matching and retrieval.

8. The face recognition area network intelligent monitoring system and method according to claim 1, characterized in that: The behavior analysis module includes a target detection and tracking unit and a crowd behavior analysis unit. The crowd behavior analysis unit calculates the density distribution of people, the mean and variance of speed, and the distance relationship between people based on trajectory data, and determines abnormal behavior and issues warnings by setting thresholds. The target detection and tracking unit uses the YOLO algorithm to detect the target and obtain the personnel position information P (x, y, w, h), and uses the Kalman filter algorithm to track the trajectory, and updates the equation through state prediction and measurement: Among them, F k is the state transfer matrix, B k is the control input matrix, u k is the control input vector, P k|k-1 and P k|k are the prediction covariance matrix and the update covariance matrix, K k is the Kalman gain, H k is the measurement matrix, Q k is the process noise covariance matrix, R k is the measurement noise covariance matrix, z k It is the measurement vector, i.e. the position information obtained by target detection.

9. The face recognition area network intelligent monitoring system according to claim 1, characterized in that: The data storage and transmission module includes an encrypted storage unit, a secure transmission unit and a data backup unit. The encrypted storage unit uses the AES encryption algorithm to encrypt the facial data for storage. The secure transmission unit uses the SSL / TLS protocol to encrypt and transmit the data. The data backup unit is used to regularly back up the data to an off-site storage device.

10. A face recognition area network intelligent monitoring method, applied to the face recognition area network intelligent monitoring system according to any one of claims 1 to 9, characterized in that: The following steps are involved: Image acquisition, collecting the image data of the monitoring area through the image acquisition module and transmitting it to the data preprocessing module; Data preprocessing, the data preprocessing module performs illumination compensation, posture correction and occlusion processing on the image, crops and scales to extract the face area image, and transmits the processed image cropped and scaled to extract the face area image to the feature extraction module; Feature extraction, the feature extraction module extracts feature vectors from the face area image using the multi-scale feature extraction unit; Face recognition: Match the extracted feature vector with the features in the face feature library through the face recognition module, calculate the feature distance based on the model trained by the Triplet Loss function and determine whether the recognition is successful. If successful, the person's identity information is obtained; if failed, it is marked as an unknown identity and further analyzed; Behavior analysis: Target detection and trajectory tracking of people in the monitoring area are performed through the behavior analysis module, crowd behavior analysis is performed based on trajectory data, and abnormal behavior is judged; Result output and storage: face recognition results and behavior analysis results are displayed and stored through the data storage and transmission module, and an alarm is issued in case of abnormal behavior. The stored data includes original image data, processing process data, recognition result data and behavior analysis data.

Images