Law enforcement site management and supervision method and system based on deep learning

Abstract

The present invention relates to a deep learning-based law enforcement site management and supervision method and system, comprising: collecting law enforcement process data within the site; performing face recognition based on the data using a face recognition model, wherein the face recognition model is constructed based on an improved Dense-MobileFaceNet network; and performing target recognition on the face-recognized data using a target recognition model, thereby identifying abnormal behavior during the law enforcement process; wherein the target recognition model is constructed based on an improved YOLOv7-tiny network. This invention enables real-time monitoring of law enforcement sites, identification of violations, and early warning of violations, thereby improving the transparency and legitimacy of law enforcement management.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for managing and supervising law enforcement sites based on deep learning. Background Art

[0002] Existing supervision and management of law enforcement sites typically rely on manual inspections and video surveillance. However, due to aging monitoring equipment and inefficient manual oversight, the law enforcement process suffers from a lack of transparency and irregular management. Traditional oversight methods struggle to accurately analyze behavior across the entire enforcement site in real time, making it difficult to detect violations promptly. To address this issue, the introduction of deep learning technology can significantly improve the efficiency and accuracy of oversight and reduce errors in manual oversight. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for the management and supervision of law enforcement sites based on deep learning to solve the problems existing in the above-mentioned existing technologies. With the help of computer vision technology and deep learning models, real-time monitoring, violation identification and early warning in law enforcement sites can be achieved, thereby improving the transparency and legality of law enforcement management.

[0004] In the first aspect, to achieve the above-mentioned objectives, the present invention provides the following solutions:

[0005] The deep learning-based law enforcement site management and supervision method includes:

[0006] Collect law enforcement process data within law enforcement venues;

[0007] Based on the law enforcement process data, face recognition is performed using a face recognition model; wherein the face recognition model is constructed based on an improved Dense-MobileFaceNet network;

[0008] A target recognition model is used to perform target recognition on the data after face recognition to identify abnormal behavior during law enforcement; wherein the target recognition model is built based on an improved YOLOv7-tiny network.

[0009] Optionally, collecting law enforcement process data within the law enforcement venue includes:

[0010] Collect video data of law enforcement processes in law enforcement locations;

[0011] Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data;

[0012] The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

[0013] Optionally, improving the Dense-MobileFaceNet network includes:

[0014] The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset.

[0015] The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

[0016] Optionally, improve the YOLOv7-tiny network by:

[0017] An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network;

[0018] The enhanced pyramid segmentation attention mechanism is added to the Neck part of the YOLOv7-tiny network to enhance the feature network;

[0019] The processing process of the enhanced pyramid segmentation attention mechanism is:

[0020] Z i =SEWeight(F i ),i=0,1,2,…,S-1,

[0021]

[0022] Out=Cat([Y0,Y1,…,Y S-1 ]),

[0023] Among them, Z i The weight vector representing the feature map channel attention, att i It represents the normalized weighting process of Softmax on the channel attention feature weight, Out is the feature map obtained by the final output, SEWeight(F i ) indicates that the Squeeze-and-Excitation (SE) attention module is used to perform channel-level weighted processing on the feature Fi, i indicates the index of the feature map Fi, i.e., the i-th feature map, Softmax(Z i ) represents the application of the Softmax function to the vector Zi. The Softmax function is usually used to convert a vector into a probability distribution so that each element in the vector is between 0 and 1, and the sum of all elements is 1.i represents the channel weight vector of the i-th feature map Fi calculated by the SE attention module, S represents the number of groups into which the input feature map X is split by the SPC (Split Pyramid Convolution) module, and Y S-1 Represents the last feature map after the attention mechanism is processed. Cat() represents the concatenation operation of the feature map, which merges multiple feature maps into a new feature map along a certain dimension (usually the channel dimension).

[0024] During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network.

[0025] Optionally, identifying abnormal behavior during law enforcement includes:

[0026] Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model;

[0027] According to the violation index, different levels of warning push are carried out;

[0028] The violation index calculation model is:

[0029]

[0030] Among them, S i represents the severity score of the behavior, Fi represents the behavior frequency factor, and W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

[0031] In a second aspect, to achieve the above-mentioned purpose, the present invention further provides the following solutions:

[0032] A law enforcement site management and supervision system based on deep learning, comprising: a data acquisition module, a data processing module, a face recognition module, a target recognition module, a report generation module, a confidentiality transmission module, and an edge computing module;

[0033] The data collection module is used to collect law enforcement process data in the law enforcement site;

[0034] The data processing module is used to perform data enhancement processing on the law enforcement process data;

[0035] The face recognition module is used to perform face recognition on the enhanced law enforcement process data using a face recognition model; wherein the face recognition model is constructed based on an improved Dense-MobileFaceNet network;

[0036] The target recognition module is used to use a target recognition model to perform target recognition on the data after face recognition and identify abnormal behavior during law enforcement; wherein the target recognition model is built based on an improved YOLOv7-tiny network;

[0037] The report generation module is used to generate a report on the identified abnormal behavior;

[0038] The transmission module is used to encrypt data transmission between modules using a single sign-on and a transmission mechanism based on hash-salted encryption;

[0039] The edge computing module is used to adopt edge computing methods for the data acquisition module, data processing module, face recognition module, and target recognition module.

[0040] Optionally, the data processing module:

[0041] Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data;

[0042] The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

[0043] Optionally, improving the Dense-MobileFaceNet network includes:

[0044] The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset.

[0045] The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

[0046] Optionally, improve the YOLOv7-tiny network by:

[0047] An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network;

[0048] The enhanced pyramid segmentation attention mechanism is added to the Neck part of the YOLOv7-tiny network to enhance the feature network;

[0049] The processing process of the enhanced pyramid segmentation attention mechanism is:

[0050] Z i =SEWeight(F i ),i=0,1,2,…,S-1,

[0051]

[0052] Out=Cat([Y0,Y1,…,Y S-1 ]),

[0053] Among them, Z i The weight vector representing the feature map channel attention, att i It represents the normalized weighting process of Softmax on the channel attention feature weight, Out is the feature map obtained by the final output, SEWeight(F i ) indicates that the Squeeze-and-Excitation (SE) attention module is used to perform channel-level weighted processing on the feature Fi, i indicates the index of the feature map Fi, i.e., the i-th feature map, Softmax(Z i ) represents the application of the Softmax function to the vector Zi. The Softmax function is usually used to convert a vector into a probability distribution so that each element in the vector is between 0 and 1, and the sum of all elements is 1. i represents the channel weight vector of the i-th feature map Fi calculated by the SE attention module, S represents the number of groups into which the input feature map X is split by the SPC (Split Pyramid Convolution) module, and Y S-1 Represents the last feature map after the attention mechanism is processed. Cat() represents the concatenation operation of the feature map, which merges multiple feature maps into a new feature map along a certain dimension (usually the channel dimension).

[0054] During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network.

[0055] Optionally, identifying abnormal behavior during law enforcement includes:

[0056] Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model;

[0057] The violation index calculation model is:

[0058]

[0059] Among them, S i Indicates the severity score of the behavior, F i represents the behavior frequency factor, W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

[0060] In a third aspect, the present invention further provides: an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0061] In a fourth aspect, the present invention further provides: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0062] The beneficial effects of the present invention are:

[0063] The present invention first collects data on the law enforcement process within law enforcement sites. Secondly, based on this data, a facial recognition model is used to perform facial recognition. Finally, a target recognition model is used to perform target recognition on this data, identifying abnormal behavior during the law enforcement process. By leveraging image preprocessing technology and deep learning models, the present invention enables intelligent monitoring and real-time early warning of personnel behavior within law enforcement sites. The system can identify and monitor behaviors such as not wearing a police uniform, illegal mobile phone use, and illegal smoking, ensuring the standardization and legality of law enforcement processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 This is a flowchart of a method for managing and supervising law enforcement sites based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] like Figure 1 As shown, this embodiment proposes a method for managing and supervising law enforcement sites based on deep learning, including:

[0069] Collect law enforcement process data within law enforcement venues;

[0070] Based on law enforcement process data, face recognition is performed using a face recognition model; the face recognition model is built based on the improved Dense-MobileFaceNet network;

[0071] The target recognition model is used to perform target recognition on the data after face recognition to identify abnormal behavior during law enforcement. The target recognition model is built based on the improved YOLOv7-tiny network.

[0072] Furthermore, the collection of law enforcement process data within the law enforcement venue includes:

[0073] Collect video data of law enforcement processes in law enforcement locations;

[0074] Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data;

[0075] The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

[0076] Specifically, in this embodiment, the data acquisition business generally covers two core links: human-involved collection and automated collection technology. The human collection link involves directly capturing raw visual data through manual photography, and then manually applying precise annotations to these data to ensure the quality of the data and the accuracy of the labels. On the other hand, automated collection relies on the initial weights of the pre-trained model to perform intelligent data screening and preprocessing operations, significantly improving the efficiency and scale of data processing. The data collection process is as follows: Figure 1 shown.

[0077] The image processing in this embodiment is performed as follows:

[0078] The captured images are preprocessed using an image enhancement algorithm for non-uniform lighting conditions. A deep learning-based image enhancement model is employed, using semantically guided image enhancement techniques to improve image quality in low-light conditions and enhance image details, thereby increasing the accuracy of subsequent recognition modules.

[0079] Sample generation and frame extraction: Separate key frames from the video stream, automatically generate samples, use deep convolutional neural networks to perform preliminary processing on the video frames, and extract high-quality images for model training and real-time monitoring.

[0080] Image enhancement technology: A low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through a semantic segmentation model and performs local contrast adjustment to ensure image clarity in low-light environments.

[0081] Further improvements to the Dense-MobileFaceNet network include:

[0082] The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset.

[0083] The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

[0084] Specifically, in this embodiment, an improved Dense-MobileFaceNet network is used for face recognition, which particularly optimizes the recognition ability of small targets and blurred faces, and supports face detection under infrared imaging.

[0085] Fuzzy Small Target Face Detection: By introducing the lightweight Dense-MobileFaceNet network, computing power consumption is reduced while improving face recognition accuracy. Combined with the ArcFace loss function, the system can accurately distinguish different people and reduce false positives.

[0086] Infrared imaging recognition: CycleGAN is used to generate adversarial networks to convert infrared images into visible light images, and then an improved MobileFaceNet model is used for face recognition, solving the problem of difficult recognition in low-light environments at night.

[0087] Specifically, in this embodiment, recognizing a face using a face recognition model includes:

[0088] Police information entry: Clearly record the key information fields of police officers, including the police officer's name, gender, ID number, contact information and other important information, and then establish a complete and reliable information database.

[0089] Using a camera combined with a Python script, facial images of police officers were captured. The camera's resolution and shooting conditions met the requirements for face detection and recognition, ensuring image quality and accuracy. During implementation, an efficient and stable facial image acquisition system was developed by calling the camera's API or leveraging the OpenCV library in Python. This ensured that the captured facial images could be reliably matched and recognized during the subsequent face recognition process.

[0090] Shooting Requirements: (a) The image must be taken against a pure white background with good lighting. (b) No hat and proper attire required. (c) If not wearing glasses, only eight images are required. (d) If wearing glasses, two sets of images are required (16 images total, with and without glasses). (e) The subject's facial features must be clearly visible. File Upload Instructions: Each person must have their own folder, named according to their police officer's name.

[0091] Data Correction: We use a variety of face detection technologies, such as the classic Haar cascade detector and deep learning-based face detection algorithms, to cope with different scenarios and situations, ensuring accurate positioning of faces in images captured by the camera. These technologies can accurately locate the position of faces in the image, providing accurate basic information for subsequent processing steps.

[0092] After completing face detection, this embodiment will align the detected faces to ensure that the facial features are in a consistent position and scale in the image. By locating key points such as the eyes and nose, the face is fine-tuned to make its position and size in the image consistent, thereby providing more accurate data for subsequent face recognition and analysis.

[0093] By integrating different detection and alignment technologies, a complete face information processing process is established to provide reliable face recognition and feature extraction functions for the police information management system.

[0094] Facial feature extraction: Using an efficient facial feature extraction algorithm, the aligned facial images are converted into a set of unique numerical feature vectors. These feature vectors will reflect the key features of the face in different dimensions, such as contour, texture, structure, etc., thereby converting the complex information of the face into machine-understandable data representation.

[0095] By converting facial images into feature vectors, efficient extraction and comparison of facial features can be achieved. This numerical representation can reduce the dimension of the data, improve computing efficiency, and reduce storage and transmission costs, becoming the core data for subsequent comparison and recognition tasks.

[0096] Police Officer Information Annotation: Detailed data annotation is performed on the extracted facial data, adding specific label information to each facial image to accurately identify the police officer to whom it belongs. During the annotation process, a name, work number, or other unique identifier is added to each image to ensure that each face is associated with a specific police officer's information.

[0097] Establish a sample library: The extracted facial feature vectors are tightly combined with the corresponding police officer information and stored in a dedicated sample library. This storage can be done using the CSV (comma-separated values) file format. In this sample library, each row will contain a facial feature vector and the corresponding police officer information, such as name and employee number. This structure provides an organized and manageable data storage method for subsequent face recognition and query tasks.

[0098] Sample library division: In order to effectively train, adjust and evaluate the model, the sample library is reasonably divided into three parts according to appropriate proportions: training set, validation set and test set.

[0099] The training set, which makes up the majority of the sample library, is used to train the model. By iteratively optimizing the training set, the model learns the association between facial feature vectors and corresponding police officer information, thereby improving recognition accuracy and reliability.

[0100] The validation set will be used to adjust the model's hyperparameters, such as learning rate, batch size, etc. By testing and comparing on the validation set, the best hyperparameter combination can be selected to improve the performance and generalization ability of the model.

[0101] The test set will be used to comprehensively evaluate the model's performance. At this stage, the model is tested using samples from the test set to assess its accuracy and effectiveness in real-world applications. By evaluating performance on the test set, this embodiment can determine whether the model achieves the expected performance level and demonstrates robustness in different situations.

[0102] This division of the sample library will provide orderly data support for model training and evaluation, ensuring that model training, adjustment and evaluation can be carried out in a targeted manner.

[0103] The face recognition process is shown below.

[0104] (1) Loading the dataset

[0105] Use PyTorch's Dataset and DataLoader for data loading and preprocessing, and split it into training and validation sets.

[0106] (2) Normalization

[0107] Normalize the image pixel values ​​to the range [0,1] or [1,1].

[0108] (3) Model construction

[0109] (a) Load the Dense-MobileFaceNet model:

[0110] We used the Dense-MobileFaceNet model introduced by the team and added a custom fully connected layer to adapt to specific classification tasks. We also initialized the model parameters to ensure that the model was in a suitable state before starting training.

[0111] (b) Define the ArcFace loss function:

[0112] Customize the ArcFace loss function, including constraints on angle boundaries. Select an appropriate optimization algorithm and learning rate, and configure the optimizer to adjust model parameters to minimize the loss function.

[0113] (4) Training model

[0114] The model training phase is the most important part of the entire process, which involves multiple iterations of optimizing the model parameters to better fit the training data. Operations in this phase include:

[0115] Iterative training: The model is trained multiple times using the data in the training set, and each iteration is called an epoch.

[0116] Calculate loss: In each epoch, calculate the loss between the current model's prediction of the training data and the true label.

[0117] Backpropagation and optimization: Based on the gradient of the loss function, the model parameters are updated using the backpropagation algorithm to optimize the model to reduce the loss.

[0118] Monitor the training process: Track loss changes and model performance during training so that you can adjust hyperparameters and model structure in a timely manner.

[0119] (5) Save the best model:

[0120] Save the best model weights during training.

[0121] Through the above steps, this example uses PyTorch and Dense-MobileFaceNet to train a face recognition model, and uses the ArcFace loss function to improve the model's discriminative ability. This process ensures that the model can efficiently and accurately identify the faces of public security bureau officers and is suitable for real-world application scenarios.

[0122] Further improvements to the YOLOv7-tiny network include:

[0123] An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network;

[0124] The enhanced pyramid segmentation attention mechanism is added to the Neck part of the YOLOv7-tiny network to enhance the feature network;

[0125] During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network.

[0126] Specifically, in this embodiment, target recognition is performed based on an improved version of the YOLOv7-tiny algorithm, which can quickly identify specific targets such as police uniforms and perform behavior detection.

[0127] Police uniform recognition: The system improves its ability to recognize details by introducing an enhanced pyramid segmentation attention mechanism. It can monitor in real time whether police officers are dressed properly, such as whether they are wearing police badges and armbands.

[0128] Violation detection: Through posture estimation and target recognition, the system can monitor changes in a person's posture, such as whether law enforcement officers use mobile phones, smoke, or engage in other inappropriate behaviors during the investigation process.

[0129] Specifically, in this embodiment, the target recognition process includes:

[0130] 1. Sample library sample project, the process is as follows:

[0131] (1) Data cleaning

[0132] Ensure the accuracy and completeness of the data, adopt a series of effective data processing strategies, and deal with missing values, outliers, and erroneous values ​​in the data.

[0133] For missing values, appropriate methods are used to fill or interpolate to ensure data integrity. By analyzing the characteristics of the data, appropriate filling methods are selected, or interpolation is performed through model prediction to maintain data accuracy and continuity.

[0134] Outliers are detected and addressed. Statistical analysis or machine learning methods are used to identify potential outliers, which are then verified and corrected based on actual circumstances. Outliers may be caused by data entry errors or other unusual circumstances. Appropriate strategies, such as correcting, deleting, or marking outliers, are implemented to ensure data reliability and authenticity.

[0135] Perform data validation and logic checks on erroneous values ​​to ensure the internal consistency and correctness of the data. By establishing data validation rules, illogical data is excluded and corrected or checked in a timely manner to reduce the impact of data errors and ensure the credibility and accuracy of the data.

[0136] (2) Feature Engineering

[0137] To meet the needs of face recognition, joint interrogation, and clothing judgment, a series of targeted feature engineering steps are performed to optimize model performance and generalization capabilities.

[0138] First, feature selection screens out features that have a significant impact on face recognition, joint interrogation, and clothing judgment. By analyzing the requirements of different tasks, features closely related to the tasks are selected, and redundant or irrelevant features are eliminated to streamline the data and improve the efficiency of the model.

[0139] Secondly, feature transformation is performed to map the original features into a more meaningful representation space, which involves dimensionality reduction techniques such as principal component analysis (PCA) or linear discriminant analysis (LDA) to reduce feature dimensions and extract key information. Through feature transformation, data noise can be reduced, model stability can be enhanced, and performance can be improved.

[0140] Finally, feature extraction is performed according to the needs of different tasks to extract higher-level abstract features from the data in order to capture more discriminative features, thereby improving the generalization ability and performance of the model.

[0141] (3) Data Integration

[0142] Different data from various police stations will be collected, including relevant data such as facial recognition, joint interrogation and clothing judgment. These data are scattered in different databases, folders or systems. Using data integration tools and technologies, they will be integrated in an orderly manner to facilitate subsequent analysis and modeling work.

[0143] 2. Joint interrogation warning

[0144] The interrogation process is a crucial step in law enforcement and case handling. Its standardization and fairness are directly related to the outcome of the case and the credibility of the judiciary. This system uses artificial intelligence technology to comprehensively monitor the interrogation process, ensuring its legality and standardization.

[0145] 1. Single police officer inquiries about violations

[0146] (a) Face detection

[0147] Face detection is performed on frame images captured from the video stream to accurately identify and locate all faces in the image. In the frame image, the face detection algorithm is applied to find the position of the face and obtain the coordinate information of the face in the image, including the coordinates of the upper left and lower right corners of the face. The coordinate information can determine the position and size of the face in the image.

[0148] (b) Face matching

[0149] The system uses the constructed sample database data to perform facial comparisons to determine the identity of the recognized face. By comparing similarity scores, it finds the facial features in the sample database that are most similar to the recognized face, thereby determining the possible identity. If the similarity score reaches the set threshold, it can be determined that the recognized face matches the facial information of a police officer, thus confirming the identity.

[0150] (c) Face counting

[0151] After completing the face detection, the number of faces detected is counted, and based on the number, it is determined whether there are at least two faces. If there are less than two faces for more than 10 minutes, it is determined to be a violation of the single police interrogation.

[0152] 2. Timeout violation

[0153] The process is relatively simple. The system automatically monitors the duration of each interrogation by setting a counter based on a preset time limit. If an interrogation exceeds the preset time limit, the system automatically records it and issues an alert, reminding officers to end the interrogation promptly and avoid excessive questioning that could cause physical or psychological harm to the suspect.

[0154] 3. Warning of non-compliance by law enforcement entities

[0155] 1. Judgment of police uniforms

[0156] (1) Introduction to the YOLOv7 algorithm

[0157] The YOLOv7 object detection algorithm, released in July 2022 by the authors of YOLOv4, is a novel and superior detector. Compared to the previous YOLO series, it utilizes an efficient aggregation network, specifically the EAN module, which effectively enhances the network's learning capabilities. Furthermore, YOLOv7 employs re-parameterized convolutions to accelerate the network while ensuring model performance. Furthermore, adjustments have been made to auxiliary head detection, extracting shallow features from the head network as auxiliary heads, while the deeper network serves as the guiding head. Finally, YOLOv7 leverages the strengths of YOLOv5 and YOLOX to adopt a dynamic label allocation strategy.

[0158] The YOLOv7-tiny network model consists of four parts: Input, Backbone, Neck, and Head. The input part preprocesses the input image using methods such as mosaic data augmentation and adaptive anchor box calculation. The Backbone is composed of several CBS blocks, ELAN layers, and MP layers. The CBS block is composed of Conv layers, BN layers, and SiLU functions, while the ELAN layer is composed of multiple CBS modules. The MP layer consists of CBS blocks and Maxpool. The Neck uses the SPPCSPC and PAN structures to fuse features from various layers to detect objects of different scales. The structural connection consists of the CBS module, MP layer, SPPCSPC module, and ELAN+. The SPPCSPC module connects the Backbone and Neck. It is composed of multiple CBS modules and three Maxpools. The difference between ELAN+ and ELAN lies only in the number of outputs selected during the CBS module connection process. The head layer uses REP layers and CBM blocks. The grid structure of the REP module differs during training and inference, while the CBM module consists of a Conv layer, a BN layer, and a Sigmoid function. In the head layer stage, the output image passes through three REP layers and a CBM layer to output three prediction results at different scales.

[0159] (2) Attention Mechanism - EPSANet Block Module

[0160] The attention mechanism focuses on the important parts of the target, thereby obtaining its key information. This method plays a huge role in the field of deep learning. PSA pyramid split attention is a more efficient dual attention mechanism. For input features, the PSA attention mechanism first splits them through the SPC module. SPC divides the input tensor X into S groups, each of which is named X0-X s-1 Then all the features extracted from these S groups are spliced ​​together to extract the target information in each channel feature map. This extraction process is shown in formulas (1) to (3):

[0161] F i =Conv(K i ×K i , G i )(X) i=0,1,2,...S-1, (1)

[0162]

[0163] F=Cat([F0,F1,...,F s-1 ]), (3)

[0164] Where: k is the size of the multi-scale convolution kernel, G is the size of the group, and F is the feature map obtained after splicing.

[0165] After SPC splitting, the feature map F is obtained and passed through the SE attention module, which uses it to identify the key areas of each channel feature and further obtain the feature weights of these channels. Finally, the Softmax normalization is used to weight each group of feature weights to further obtain the final output. The entire principle process is shown in Equations (4) to (6):

[0166] Z i =SEWeight(F i ), i=0, 1, 2,…, S-1, (4)

[0167]

[0168] Out=Cat([Y0,Y1,…,Y S-1 ]), (6)

[0169] Where: Z i Represents the weight vector of the feature map channel attention, att i The Softmax normalization weighting process for channel attention features is shown in Figure 2, where Out is the final output feature map. Based on the PSA attention mechanism and combined with the classic residual network model ResNet, the 3x3 convolution in the BottleNeck portion of the module is replaced with a PSA module, forming a new EPSABlock. The EPSA module builds on the concept of channel attention and simultaneously enables flexible and efficient feature extraction at different scales.

[0170] To accurately detect police uniforms of various sizes and improve uniform recognition accuracy in the complex background environment of a public security bureau, this embodiment employs an improved strategy by adding an EPSA block after the ELAN layer of each scale in the YOLOv7-tiny network backbone. This optimizes the network structure and enables accurate recognition of objects of three sizes: near, medium, and far. The EPSA block is then added to the enhanced feature network in the neck region to enhance the network's ability to perceive feature maps, thereby improving YOLOv7-tiny's performance in detecting helmet wear.

[0171] (3) Loss function

[0172] The YOLOv7-tiny loss function consists of three parts: confidence prediction loss, category loss, and positioning loss. Our team did not change the first two loss functions of YOLOv7-tiny and still used the meta-cross entropy loss function in the original network. For positioning loss, the original YOLOv7-tiny used CIoU Loss, which is:

[0173]

[0174] Where: is the penalty term, b and b gt Represents B and B respectively gt The center point, ρ 2 is the Euclidean distance, c is the distance between B and B gt The diagonal distance of the minimum outer rectangle of the prediction box, α is a positive balance parameter, and v represents the consistency with the aspect ratio of the prediction box. For parameters α and v, there are:

[0175]

[0176] Where: w and w gt Indicates the width of B and G, h and h gt Indicates the height of B and G.

[0177] Compared to CIoU Loss, SIoU Loss, while also maintaining scale invariance, not only takes into account geometric parameters such as overlapping area, center point distance, and aspect ratio, but more importantly, based on the above, SIoU Loss also considers angle parameters and the vector angle between the required regressions, redefining the penalty index. Therefore, this embodiment uses SIoULoss as the positioning loss function of YOLOv7-tiny:

[0178]

[0179] Since this loss also takes the direction box angle into consideration, it makes the prediction box regression stable, while also speeding up the convergence speed and optimizing the regression effect.

[0180] Specifically, in this embodiment, identifying abnormal behavior during law enforcement includes:

[0181] Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model;

[0182] According to the violation index, different levels of warnings are pushed.

[0183] In this embodiment, a violation index calculation model is preset as follows:

[0184]

[0185] Among them, S i represents the severity score of the behavior, Fi represents the behavior frequency factor, and W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

[0186] Severity score Si of the violation:

[0187] Define the behavior type and corresponding score:

[0188] Set multiple types of illegal and irregular behaviors i = {1, 2, ..., n}, and give each behavior a severity score based on its impact on the fairness and security of law enforcement.

[0189] For example: Interrogation timeout: S = 10, Unauthorized entry into key areas: S = 7, Off duty: S = 5, Smoking violation: S = 3 Dynamic adjustment score: Consider the situation when the behavior occurs and add dynamic adjustment factors. For example:

[0190] If the behavior occurs in a high-risk area (such as an interrogation room): the score is increased by 20%.

[0191] formula:

[0192] S i =Base score, (1+adjustment factor).

[0193] Frequency factor Fi

[0194] The behavior frequency factor measures the frequency of a certain type of behavior during a specific monitoring period. The specific calculation method is as follows:

[0195]

[0196] Specific rules:

[0197] Low Frequency Behavior: Set the lowest frequency factor to 1.

[0198] High-frequency behavior: The frequency factor increases linearly with the number of times, but an upper limit Fmax is set to avoid excessive weight.

[0199] Example:

[0200] If a behavior occurs 5 times within 8 hours:

[0201]

[0202] Personnel identity weight Wi

[0203] In law enforcement settings, the impact of different people's behavior may be different, so person identity weights are introduced. The weight settings are as follows:

[0204] Law enforcement officers: Weight W = 2 (their behavior directly affects the fairness of law enforcement).

[0205] Subject to law enforcement: Weight W = 1.5 (behavior may lead to law enforcement risks).

[0206] Third-party personnel (such as guardians, etc.): weight W = 1.

[0207] If the personnel's identity is sensitive (e.g., law enforcement officers on duty or on special missions), an additional adjustment factor may be added:

[0208] W i = basic weight × (1 + identity adjustment coefficient)

[0209] Example:

[0210] If a law enforcement officer violates discipline during a sensitive period, the weight will be adjusted to:

[0211] W = 2 × (1 + 0.3) = 2.6

[0212] 5. Site risk correction value R

[0213] According to the specific functions and risk levels of the law enforcement sites, set the correction value RRR and modify the violation index. The risk level is divided into three levels: low, medium and high:

[0214] Low-risk places (such as ordinary office areas): R=0.

[0215] Medium-risk places (such as interrogation rooms): R=5.

[0216] High-risk places (such as gun depots): R=10.

[0217] Specific corrections:

[0218] R = risk factor × number of regional events

[0219] Time weight factor Ti

[0220] The time period during which enforcement occurs has a significant impact on the risk of violation. For example, enforcement at night or during shift changes is more likely to result in management loopholes. Therefore, a time weight Ti is added to reflect time risk.

[0221] Weight rules:

[0222] High-risk time period (such as 0:00-6:00 in the morning): T = 2.0.

[0223] Medium-risk time period (such as after-work hours 18:00-23:59): T=1.5.

[0224] Low-risk time period (such as daytime working hours 6:00-18:00): T = 1.0.

[0225] Dynamic adjustment mechanism:

[0226] For some special periods, risk weights can be adjusted dynamically. For example:

[0227] Approaching shift change time (e.g. within 15 minutes): weight increased by 10%.

[0228] Major holidays or special law enforcement missions: weight increased by 20%.

[0229] formula:

[0230] T i = Basic weight × (1 + adjustment coefficient)

[0231] Threshold determination of violation index

[0232] Set thresholds based on the violation index results to determine whether there is a risk of violation at the law enforcement site:

[0233] Violation index < T1: Risk is controllable and no warning is required.

[0234] T1≤Violation Index<T2: The risk is high and warning is required.

[0235] Violation index ≥ T2: Serious violation, requiring immediate action.

[0236] Example thresholds:

[0237] T1=20 (low risk).

[0238] T2=50 (high risk).

[0239] Early warning mechanism

[0240] If the violation index exceeds T2, the system generates a red warning signal and pushes it to the management personnel.

[0241] If the index is between T1 and T2, a yellow warning signal is generated, prompting a directional inspection.

[0242] This implementation also proposes a law enforcement site management and supervision system based on deep learning, including: data collection module, data processing module, face recognition module, target recognition module, report generation module, confidentiality transmission module and edge computing module;

[0243] Data collection module, used to collect law enforcement process data in law enforcement sites;

[0244] Data processing module, used for data enhancement processing of law enforcement process data;

[0245] A face recognition module is used to perform face recognition on the enhanced law enforcement process data using a face recognition model; the face recognition model is built based on an improved Dense-MobileFaceNet network;

[0246] The target recognition module is used to perform target recognition on the data after face recognition using the target recognition model, and identify abnormal behavior during law enforcement. The target recognition model is built based on the improved YOLOv7-tiny network.

[0247] Report generation module, used to generate reports on identified abnormal behaviors;

[0248] The confidential transmission module is used to encrypt data transmission between modules using single sign-on and a transmission mechanism based on hash and salt encryption;

[0249] The edge computing module is used to adopt edge computing methods for the data acquisition module, data processing module, face recognition module, and target recognition module.

[0250] Furthermore, the data processing module:

[0251] Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data;

[0252] The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

[0253] Further improvements to the Dense-MobileFaceNet network include:

[0254] The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset.

[0255] The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

[0256] Further improvements to the YOLOv7-tiny network include:

[0257] An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network;

[0258] The enhanced pyramid segmentation attention mechanism is added to the Neck part of the YOLOv7-tiny network to enhance the feature network;

[0259] The processing process of the enhanced pyramid segmentation attention mechanism is:

[0260] Z i =SEWeight(F i ),i=0,1,2,…,S-1,

[0261]

[0262] Out=Cat([Y0,Y1,…,Y S-1 ]),

[0263] Among them, Z i The weight vector representing the feature map channel attention, att i It represents the normalized weighting process of Softmax on the channel attention feature weight, Out is the feature map obtained by the final output, SEWeight(F i ) indicates that the Squeeze-and-Excitation (SE) attention module is used to perform channel-level weighted processing on the feature Fi, i indicates the index of the feature map Fi, i.e., the i-th feature map, Softmax(Z i ) represents the application of the Softmax function to the vector Zi. The Softmax function is usually used to convert a vector into a probability distribution so that each element in the vector is between 0 and 1, and the sum of all elements is 1. i represents the channel weight vector of the i-th feature map Fi calculated by the SE attention module, S represents the number of groups into which the input feature map X is split by the SPC (Split Pyramid Convolution) module, and Y S-1 Represents the last feature map after the attention mechanism is processed. Cat() represents the concatenation operation of the feature map, which merges multiple feature maps into a new feature map along a certain dimension (usually the channel dimension).

[0264] During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network.

[0265] Specifically, in this embodiment, identifying abnormal behavior during law enforcement includes:

[0266] Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model.

[0267] In this embodiment, a violation index calculation model is preset as follows:

[0268]

[0269] Among them, S i Indicates the severity score of the behavior, F i represents the behavior frequency factor, W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

[0270] Specifically, in this embodiment, the secure transmission module includes:

[0271] Single Sign-On:

[0272] Considering the actual use cases of the public security inspection system, this embodiment introduces the CAS system in SSO. Through the centralized identity authentication process, the defense barrier of the system entrance is strengthened, effectively limiting the potential channels for unauthorized access.

[0273] CAS is an open source single sign-on solution that provides a central server and a set of client libraries for authentication. When a user first accesses a system, CAS redirects the user to the central server for authentication and generates a ticket granting ticket (TGT) granted by the identity service, which is stored in the user's browser cookie. When the user accesses another system, CAS checks the TGT in the user's cookie. If a valid TGT exists, CAS generates a service ticket (ST) and redirects the user back to that system. The system then sends the ST to CAS for verification. After confirmation, the user can access the system.

[0274] Hash-based salt encryption algorithm:

[0275] Solution Implementation

[0276] Taking into account the actual performance of the public security inspection system database, this embodiment proposes to use the hash algorithm PBKDF2 combined with the salting method to encrypt the key information of the database. PBKDF2 (Password-based Key Derivation Function 2) can be regarded as an advanced version of the hash salt encryption algorithm, especially in scenarios involving password storage and verification. PBKDF2 increases the complexity of calculating the hash value by using a random salt value and a configurable number of iterations, thereby improving security. The following is an introduction to the method of combining the hash algorithm PBKDF2 with the salt information summary model in the public security inspection system database.

[0277] Transmission mechanisms using single sign-on and hash-based salt encryption include:

[0278] 1. User authentication stage

[0279] User request access: A user attempts to access a protected service or application.

[0280] Redirect to authentication server: If the user is not logged in, the service redirects the user to the authentication server for authentication.

[0281] Submit credentials: The user enters information such as username and password on the authentication page.

[0282] Voucher processing:

[0283] After receiving the user's credentials, the authentication server first hashes the password using a salting algorithm, which adds a randomly generated string (salt) to the original password value, and then encrypts the result using a secure hash algorithm (such as SHA-256).

[0284] Doing so can increase the difficulty of password cracking. Even if the database is leaked, it will be difficult for attackers to reverse the original password from the hash value.

[0285] 2. Authentication and session establishment phase

[0286] Verify the credentials: The authentication server compares the salted hash value with the user password hash value stored in the database to verify the validity of the credentials.

[0287] Create a session: Once the authentication is successful, the authentication server will create a unique session identifier (Session ID) for the user and may generate a security token (Token).

[0288] Return to user: The authentication server returns the Session ID or Token to the user, usually using a secure transport layer protocol (such as HTTPS) to ensure the security of the transmission process.

[0289] User access to resources: The user uses the obtained Session ID or Token to request access to the original target service or application again. At this time, the service will check the validity of this identifier or token to determine whether to allow access.

[0290] 3. Subsequent request processing

[0291] Session persistence: For subsequent requests, users no longer need to re-enter their passwords, but instead directly use SessionID or Token for authentication.

[0292] Regular updates: To improve security, the Session ID or Token may be updated regularly to reduce the risks of long-term use of the same credentials.

[0293] Experimental analysis

[0294] During model operation, the salt value serves as key information for the model, initializing the random algorithm and serving as padding data. Because the salt value is relatively confidential, attempting to obtain the final digest information by brute-forcing the digest algorithm identifier ciphertext is unrealistic. On the one hand, the cost of enumerating each digest algorithm and performing calculations is not worth the value of the information to be obtained. On the other hand, even if an attacker happens to encounter the algorithm identifier plaintext during enumeration, the length of the algorithm identifier differs from the specified length of the salt value, preventing the attacker from encountering the corresponding salt value and failing to complete subsequent steps correctly.

[0295] The functions of the salted information summary model mainly include two parts: ① calculating the information summary; ② verifying the information summary.

[0296] ① Calculate information summary

[0297] The following experiment is conducted using a password in the public security inspection system database as an example.

[0298] Perform a circular left shift algorithm on the input string content to identify the index position. For example, if the input string strop is: 0x1032547698badcfe efcdab89 67452301, then the corresponding target string destop after strop<<<8 is: 0x32547698badcfeef cdab896745230110;

[0299] Two strings of the same length are added byte-by-byte using modulo 28. For example, if string strop1 is 0x1032547698badcfe efcdab89 67452301 and strop2 is 0x 32547698badcfeef cdab896745230110, then the result of strop1 + strop2 is the target string destop 0x 4286ca0e 5296daedbc7834f0 ac682411.

[0300] Concatenate two strings of arbitrary lengths, for example, strop||strop2, which means adding strop2 to the end of strop1, that is, merging strop1 and strop2 to obtain the target string;

[0301] The XOR operation is used to XOR the i-th byte of the string strop (an even number of length) with the length-1-i-th byte, and store the result in the i-th byte, where i is 0, 1, 2, ..., (length / 2)-1. This optimizes the padding data by leveraging the uniformity and diffusion effects of the basic digest algorithm.

[0302] ②Verification information summary

[0303] Padding number ciphertext xor1: padding the ciphertext ciphertype output by the encryption algorithm. The padding data comes from the preprocessing of Salt Value. The length of xor1 after padding is the same as the digest length of the currently randomly selected basic digest algorithm. For example, if the currently selected digest algorithm is MD5, the length of xor1 after padding is 16Bytes, that is, 128bits; Padding number ciphertext xor2: padding the ciphertext ciphertype output by the encryption algorithm. The padding data comes from the preprocessing of SaltValue. The length of xor2 after padding is: Lvsalt, that is, Lend-Lbasic.

[0304] Preprocessing: Use Salt as the data source to generate padding data. If the length of the padding data is less than or equal to the length of the data source, the padding data comes directly from the data source. If the length of the padding data is greater than the length of the data source, when reusing the data source, the source data needs to be subjected to necessary incremental processing such as XOR and circular left shift to enhance the randomness of the padding data.

[0305] The above experiments combined with practical experience can prove that the hash algorithm combined with the salt encryption strategy is feasible and superior for encrypting data in the public security inspection system database.

[0306] ③PBKDF2 algorithm application

[0307] Initialization parameters, expressed as a byte string based on the original password provided by the user, are denoted as P. A random byte string is the salt value, used to increase password strength and prevent rainbow table attacks. c is the number of iterations, which controls the algorithm's computational overhead and is typically set high to increase cracking difficulty. dkLen is the expected length of the key, in bytes.

[0308] Calculate the subkey: For each i from 1 to dkLen / HMAC output length + 1, calculate the subkey F(i), where F(i) is defined as follows:

[0309] F(i)=HMAC_hash(P,S||INT(i))

[0310] Where HMAC_hash is HMAC using the selected hash function (such as SHA-256), S||INT(i) represents concatenating the salt value S with the binary representation of the integer i, and INT(i) is the integer i converted to a fixed-length byte string.

[0311] Iterative hashing, when calculating F(i), an iterative hashing process is used:

[0312]

[0313] Where U_1 = HMAC_hash(P, S||INT(i)), and for j from 2 to c, U_j = HMAC_hash(P, U_(j-1))

[0314] Combine the subkeys and concatenate all the calculated subkeys F(i) to form a large byte string. If dkLen exceeds the HMAC output length * n, additional subkeys are required, that is, increase the value of i and repeat steps 2 and 3 until the dkLen requirement is met.

[0315] Adjust the key length. If the final byte string length exceeds dkLen, truncate the extra bytes. If it is less than dkLen, continue to generate more subkeys until the required length is reached. The final output key is the required byte string.

[0316] Based on this, this embodiment can perform additional processing before the final output. For example, a circular left shift operation can be performed on the final key to increase the randomness of the key; a byte-by-byte addition modulo operation can be used to add the final key to another random sequence byte by byte, taking the modulus to stay within the byte range; a folded XOR operation can be used to XOR the first half of the final key with the second half to increase the key's entropy. Finally, string concatenation can be performed. If multiple related keys are required, the PBKDF2 results can be concatenated with different suffixes and then applied to PBKDF2 or other hash functions.

[0317] Specifically, the edge computing module in this embodiment includes:

[0318] Application Practice

[0319] Edge-Enhanced Cloud Computing (EECC) refers to a computing model that combines the capabilities of cloud computing with the advantages of edge computing. EECC pushes computing and data processing capabilities to the "edge" of the network—to devices or small data centers closer to data sources and users. The cloud platform handles non-real-time, large-scale data analysis, long-term storage, and complex computing tasks.

[0320] In the law enforcement venue management and supervision system, thanks to the system model deployed on the edge, the devices can analyze video streams in real time, detect abnormal behavior, and immediately trigger alerts. Simultaneously, more complex video analysis or historical data mining is performed on the cloud platform, facilitating deeper data processing and long-term storage. This architecture not only improves efficiency but also optimizes resource allocation, achieving a higher level of intelligence and automation.

[0321] (1) Edge computing

[0322] In order to solve the problems of high data response latency and unstable data transmission caused by the large span of law enforcement sites, the law enforcement site management and supervision system performs edge computing on edge devices in law enforcement sites in different jurisdictions, that is, deploys corresponding model algorithms, and then realizes end-side data analysis and processing, which can greatly alleviate the resource allocation pressure of the central cloud server and improve the performance and stability of the system.

[0323] ①Sample image acquisition

[0324] Edge computing requires real-time access to sample images captured by surveillance cameras at law enforcement sites as input data. To meet the specific requirements for sample image processing, it is necessary to investigate and analyze different application scenarios before deploying the edge monitoring system. This will determine the characteristics and requirements of the current scenario samples and enable the installation of appropriate monitoring systems and image acquisition protocols. To meet the sample usage requirements of the law enforcement site management and supervision system, high-performance network or IP surveillance systems must be used as data sources at the law enforcement sites.

[0325] ②Sample image processing

[0326] To ensure the accuracy and effectiveness of sample collection, the sample images collected by surveillance must be analyzed and processed to transform them into valid data. Algorithmic models deployed on edge devices monitor and identify sample images and process them into sample data. This not only facilitates the operation of edge models, but also ensures the real-time and accuracy of recognition results, enhancing system robustness.

[0327] ③Recognition results are synchronized with the cloud

[0328] After processing the sample surveillance images, the client-side system needs to upload the data to the central cloud server. To avoid bandwidth pressure caused by massive data congestion and to ensure data security, accuracy, and integrity, reliable network protocols such as MQTT are used for data transmission.

[0329] ④Model upgrade and update

[0330] To adapt to the specific needs of different application scenarios and continuously optimize performance, models deployed on the edge need to be regularly upgraded and updated. By pushing updated system models to the cloud platform, the cloud platform automatically pushes the OTA models to edge devices, which then automatically update and upgrade the models. This process ensures that the models continuously learn new knowledge, adapt to environmental changes, and correct previous errors, resulting in better inspection results and higher accuracy.

[0331] ⑤ Equipment status cloud reporting

[0332] To maintain system stability, administrators regularly report the health status of edge devices to the cloud, including hardware working conditions, model execution status, and abnormal event records, to help cloud administrators monitor in real time, respond quickly to abnormalities, and ensure smooth system operation.

[0333] (2) Cloud platform computing

[0334] Due to the limitations of edge devices in law enforcement facilities, deeper, large-scale data analysis and data storage capabilities are unavailable. Therefore, data processed by edge computing platforms on edge devices needs to be synchronized to the cloud platform for further processing and analysis. The cloud platform computing module primarily provides functions such as intelligent analysis of sample data, data sharing, and synchronization services. It also completes tasks such as training video intelligent analysis models, security log mining and big data analysis, video sample annotation and generation, global monitoring, and edge node management. This fully leverages the cloud platform's powerful computing power and storage resources, while saving bandwidth, improving data security and privacy, and enhancing the responsiveness of law enforcement management and oversight systems.

[0335] ① Intelligent sample analysis

[0336] The central cloud primarily performs tasks such as model training, model upgrades and maintenance, data storage and analysis, sample labeling and generation, information mining and analysis, and edge node scheduling. Cloud administrators have access to object detection results, including generated sample data, and can also label and generate samples based on training samples in the cloud.

[0337] ②Data sharing and synchronization services

[0338] With the distributed storage architecture of cloud servers, the cloud platform can provide storage space for massive amounts of data, storing a large amount of critical data in the law enforcement system; through machine learning algorithms and high-performance computing resources, it can perform complex big data analysis and processing; at the same time, through multiple data replication strategies and audit logs, it ensures data security and reliability.

[0339] ③Functional services

[0340] The system can implement functions such as node management, component management, service management, and resource regulation. Administrators can view detailed information and operational status of resources such as edge computing devices and model training devices.

[0341] This embodiment uses image preprocessing technology and deep learning models to achieve intelligent monitoring and real-time warning of personnel behavior within law enforcement locations. The system can identify and monitor behaviors such as not wearing a police uniform, illegal mobile phone use, and illegal smoking, ensuring the standardization and legality of law enforcement.

[0342] Full-process law enforcement supervision: Utilizing the law enforcement site management and supervision system, unified oversight is achieved throughout the entire case filing and handling process. The system automatically records and stores video and related data from the entire law enforcement process, providing traceability and transparency, and enhancing the credibility of law enforcement.

[0343] Identification and Analysis of Illegal Behaviors: Based on structured active skeleton pose estimation technology, the system can accurately identify various illegal behaviors and generate detailed behavior reports. Through data analysis, the system can provide statistical analysis reports on illegal behaviors, providing law enforcement agencies with a basis for improvement.

[0344] Efficient Management and Security: A facial recognition vector library and visual model sample training are established to ensure the accuracy of law enforcement identity verification and behavior monitoring. Through automated monitoring and early warning, the system reduces the burden of manual inspections, improves supervision efficiency, and ensures the safety and standardized use of law enforcement sites.

[0345] These business objectives aim to address current regulatory challenges in law enforcement through intelligent and digital means, improve the standardization and transparency of the law enforcement process, and thereby effectively enhance the image and credibility of the public security force. Against the backdrop of increasingly stringent requirements for standardized law enforcement, this initiative provides strong technical support and assurance to ensure the compliance and legality of law enforcement activities.

[0346] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for managing and supervising law enforcement sites based on deep learning, characterized in that: include: Collect law enforcement process data within law enforcement venues; Based on the law enforcement process data, face recognition is performed using a face recognition model; wherein the face recognition model is constructed based on an improved Dense-MobileFaceNet network; Utilize a target recognition model to perform target recognition on the data after face recognition to identify abnormal behavior during law enforcement; wherein the target recognition model is built based on an improved YOLOv7-tiny network; Improvements to the YOLOv7-tiny network include: An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network; The enhanced pyramid segmentation attention mechanism is added again to the Neck part of the YOLOv7-tiny network to strengthen the feature network; The processing process of the enhanced pyramid segmentation attention mechanism is: Z i =SEWeight(F i ),i=0,1,2,…,S-1, Out=Cat([Y0,Y1,…,Y s-1 ]), Among them, Z i The weight vector representing the feature map channel attention, att i Represents the normalized weighting process of Softmax on the channel attention feature weight, 0ut is the feature map obtained by the final output, SEWeight(F i ) indicates that the SE attention module is used to perform channel-level weighted processing on the feature Fi, i represents the index of the feature map Fi, i.e., the i-th feature map, Softmax(Z i ) represents the application of Softmax function to vector Zi, Z i represents the channel weight vector of the i-th feature map Fi calculated by the SE attention module, S represents the number of groups into which the input feature map X is split by the SPC module, and Y S-1 Represents the last feature map after attention mechanism processing, and Cat() represents the concatenation operation of the feature map; During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network.

2. The method for managing and supervising law enforcement sites based on deep learning according to claim 1 is characterized in that: The law enforcement process data collected in law enforcement venues include: Collect video data of law enforcement processes in law enforcement locations; Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data; The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

3. The method for managing and supervising law enforcement sites based on deep learning according to claim 1 is characterized in that: Improvements to the Dense-MobileFaceNet network include: The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset. The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

4. The method for managing and supervising law enforcement sites based on deep learning according to claim 1 is characterized in that: Identifying abnormal behavior during law enforcement includes: Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model; According to the violation index, different levels of warning push are carried out; The violation index calculation model is: Among them, S i Indicates the severity score of the behavior, F i represents the behavior frequency factor, W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

5. The law enforcement site management and supervision system based on deep learning is characterized by: Used to implement the deep learning-based law enforcement site management and supervision method according to any one of claims 1 to 4, the system includes: a data acquisition module, a data processing module, a face recognition module, a target recognition module, a report generation module, a confidentiality transmission module and an edge computing module; The data collection module is used to collect law enforcement process data in the law enforcement site; The data processing module is used to perform data enhancement processing on the law enforcement process data; The face recognition module is used to perform face recognition on the enhanced law enforcement process data using a face recognition model; wherein the face recognition model is constructed based on an improved Dense-MobileFaceNet network; The target recognition module is used to use a target recognition model to perform target recognition on the data after face recognition and identify abnormal behavior during law enforcement; wherein the target recognition model is built based on an improved YOLOv7-tiny network; Improvements to the YOLOv7-tiny network include: An enhanced pyramid segmentation attention mechanism is added after the ELAN layers of different scales in the backbone network of the YOLOv7-tiny network; The enhanced pyramid segmentation attention mechanism is added to the Neck part of the YOLOv7-tiny network to enhance the feature network; The processing process of the enhanced pyramid segmentation attention mechanism is: Z i =SEWeight(F i ),i=0,1,2,…,S-1, Out=Cat([Y0,Y1,…,Y S-1 ]), Among them, Z i The weight vector representing the feature map channel attention, att i Represents the normalized weighting process of Softmax on the channel attention feature weight, 0ut is the feature map obtained by the final output, SEWeight(F i ) indicates that the SE attention module is used to perform channel-level weighted processing on the feature Fi, i represents the index of the feature map Fi, i.e., the i-th feature map, Softmax(Z i ) represents the application of Softmax function to vector Zi, Z i represents the channel weight vector of the i-th feature map Fi calculated by the SE attention module, S represents the number of groups into which the input feature map X is split by the SPC module, and Y S-1 Represents the last feature map after attention mechanism processing, and Cat() represents the concatenation operation of the feature map; During the training process of the YOLOv7-tiny network, SIoU Loss is used as the positioning loss function of the YOLOv7-tiny network; The report generation module is used to generate a report on the identified abnormal behavior; The confidentiality transmission module is used to encrypt data transmission between modules by adopting single sign-on and a transmission mechanism based on hash-salted encryption; The edge computing module is used to adopt edge computing methods for the data acquisition module, data processing module, face recognition module, and target recognition module.

6. The deep learning-based law enforcement site management and supervision system according to claim 5 is characterized in that: The data processing module: Separate key frames from video data, automatically generate samples, use deep convolutional neural networks to preprocess video frames, and extract high-quality image data; The low-light enhancement algorithm based on deep learning extracts semantic feature areas in the image through the semantic segmentation model and performs local contrast adjustment.

7. The deep learning-based law enforcement site management and supervision system according to claim 5 is characterized in that: Improvements to the Dense-MobileFaceNet network include: The CycleGAN network is added as a modality conversion module to the input of the Dense-MobileFaceNet network. The input infrared image is processed into a visible light image and then input into the Dense-MobileFaceNet network for recognition. CycleGAN is trained using a dual-light face dataset. The ArcFace loss function is used in the training process of the Dense-MobileFaceNet network.

8. The deep learning-based law enforcement site management and supervision system according to claim 5 is characterized in that: Identifying abnormal behavior during law enforcement includes: Based on the recognition result of the target recognition model, the violation index is calculated using the preset violation index calculation model; The violation index calculation model is: Among them, S i Indicates the severity score of the behavior, F i represents the behavior frequency factor, W i represents the identity weight of the person, T i Represents the time weight factor, R represents the site risk correction value, n represents the total types of illegal and irregular behavior, and i represents a certain illegal and irregular behavior type.

Images