Subway public area personnel safety identification and evaluation system and method based on deep learning
By adopting a deep learning-based personnel safety identification and evaluation system in subway public areas, the problems of strong dependence on traditional manual monitoring and low traceability efficiency are solved, efficient and accurate safety identification and evaluation are achieved, and the safety level of personnel in subway public areas is improved.
Patent Information
- Application Number
- CN202411549700.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-02-25
AI Technical Summary
The safety guarantee of personnel in traditional subway public areas relies on manual monitoring, resulting in strong artificial dependence, missed abnormal event detection and inefficient incident traceability.
The safety identification and evaluation system for the subway public areas based on deep learning is adopted, including the acquisition module, the preprocessing module, the feature labeling module, the algorithm adaptation module, the training evaluation module, the status identification module and the security evaluation module, and the deep learning algorithm is used to identify and evaluate the safety status of personnel in real time.
It has achieved extremely fast and accurate identification and evaluation, reduced the intensity of manual work, improved the safety level of personnel in public areas of the subway, and continuously improved its capabilities through self-learning mode.
Smart Images
Figure CN119515631A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision technology, and specifically relates to a system and method for assessing the safety of personnel in public areas of subways based on deep learning. Background Art
[0002] With the acceleration of urbanization, the safe operation of subways, as an important part of urban transportation, is directly related to the travel safety of the general public and social stability. The public areas of subways are densely populated and highly mobile. Once a safety accident occurs, the consequences will be disastrous. Therefore, the personnel safety requirements in the public areas of subways are an important aspect of ensuring the safety of subway operations, maintaining public order, and ensuring the safety of passengers' lives and property.
[0003] The safety of people in public areas of traditional subways mainly relies on video surveillance systems. The system captures and transmits images in real time through cameras installed in public areas of the subway, providing monitoring personnel with intuitive on-site images; this helps to detect and deal with abnormal situations in a timely manner.
[0004] At present, traditional video surveillance technology has the following shortcomings:
[0005] 1. Too much reliance on manual labor. Traditional video surveillance requires monitoring personnel to stare at the screen for a long time, which can easily cause problems such as fatigue and lack of concentration, resulting in missed detection of abnormal events, thus posing a public safety hazard.
[0006] 2. Data storage and retrieval are inconvenient. The large amount of data generated by traditional video surveillance requires a lot of storage space, and when it is necessary to retrieve and trace a specific security incident, the operation is cumbersome and inefficient.
[0007] With the continuous development of computer technology, image recognition technology has gradually matured and made certain progress, mainly based on machine learning methods such as neural networks and decision trees.
[0008] With the emergence and rapid development of deep learning technology, image recognition technology has made great breakthroughs and significantly improved the accuracy and efficiency of recognition. Summary of the invention
[0009] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a subway public area personnel safety identification and assessment system and method based on deep learning, so as to solve the problems of missed detection and low tracing efficiency of abnormal events for personnel safety requirements in subway public areas due to reliance on manual monitoring and event tracing using manual retrieval in the prior art.
[0010] To achieve the above object, the technical solution adopted by the present invention is as follows:
[0011] The present invention discloses a subway public area personnel safety identification and assessment system based on deep learning, comprising: an acquisition module, a preprocessing module, a feature annotation module, an algorithm adaptation module, a training and assessment module, a state identification module and a safety assessment module; wherein:
[0012] The acquisition module is used to collect real-time video stream data in the public areas of the subway;
[0013] The preprocessing module is used to extract the video stream data into time-sequential single-frame images and ensure the time sequence between frames of the video; process the time-sequential single-frame images one by one to achieve the unification of image samples, thereby generating training samples and evaluation samples;
[0014] The feature annotation module is used to annotate the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks;
[0015] Algorithm adaptation module, used to select 2D / 3D convolutional neural network algorithm according to actual tasks, and optimize parameters according to task conditions and training evaluation results;
[0016] The training and evaluation module is used to use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library;
[0017] The state recognition module is used to extract the features of human portraits and action behaviors from real-time sequential single-frame image samples using the optimal model, and match them with the optimal model to obtain the recognition results;
[0018] The safety assessment module is used to analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe.
[0019] Furthermore, the acquisition module is connected to multiple cameras via a network to acquire video stream data of the public area of the subway in real time.
[0020] Further, the pre-processing module includes: a video processing unit and an image processing unit;
[0021] The video processing unit sets the frame position and time interval to extract the video stream data into a time-sequential single-frame image. The time sequence of the frame is ensured by recording the timestamp and adding verification and adjustment. While extracting the frame, the timestamp of each frame is recorded. After the extraction is completed, the time sequence of the frame is verified by checking the timestamp or sequence number of the frame. If the time sequence of the frame is disordered, corresponding adjustments will be made.
[0022] The image processing unit improves the quality and efficiency of subsequent processing by preprocessing the image frames, including:
[0023] Image frame scaling: adjust the image frame size as needed to meet different processing requirements;
[0024] Grayscale: Convert color image frames into grayscale image frames;
[0025] Noise reduction: Reduce the noise points or noise areas in the image frame through filtering to make the image clear;
[0026] Image enhancement: Enhance the contrast and clarity of image frames by histogram equalization or sharpening;
[0027] Normalization: Normalize the pixel values of the image frame to eliminate the differences between different frames caused by lighting factors.
[0028] Furthermore, the feature annotation module includes a portrait feature annotation unit and a behavior feature annotation unit;
[0029] The portrait feature annotation unit marks and locates the face area by drawing a rectangular frame on a single frame image, and marks the positions of key points in the face area, including the left and right eyes, the tip of the nose, and the left and right corners of the mouth; annotates the attributes of the face, including gender and age; annotates the face identity, and associates the face with specific identity information;
[0030] The behavioral feature labeling unit labels the action categories in the video frames. The specific labeled action categories include holding weapons, fighting, chasing, falling, squatting, smoking, arson, and crying. The time sequence includes the start phase, continuation phase, and end phase of the action; the behavioral actions are assigned corresponding labels or categories.
[0031] Furthermore, the algorithm adaptation module determines the network structure and parameters according to the characteristics of the actual task; specifically, it is as follows:
[0032] When the actual task is a small-scale dataset, the LeNet two-dimensional convolutional neural network is used;
[0033] When the actual task is a large-scale image classification task, the AlexNet two-dimensional convolutional neural network is used;
[0034] When the actual task is an image classification task with high accuracy requirements, the ZFNet two-dimensional convolutional neural network is used;
[0035] When the actual task is image feature extraction, VGGNet two-dimensional convolutional neural network is used;
[0036] When the actual task is image recognition or image detection, the ResNet two-dimensional convolutional neural network is used;
[0037] When the actual task is to process the task of video recognition containing time information, a three-dimensional convolutional neural network is used;
[0038] Parameter optimization settings include:
[0039] Learning rate: Select the adaptive adjustment method to adjust the learning rate, automatically adjust the learning rate according to the indicator, and reduce the learning rate when the indicator stops improving;
[0040] Batch size: The batch size needs to be adjusted according to the actual task and dataset;
[0041] Convolution kernel size and number: Select different combinations of convolution kernel size and number according to the actual task to find the optimal solution;
[0042] Regularization method: adjust the Dropout ratio or regularization coefficient to observe the changes in model performance to prevent overfitting;
[0043] Optimizer: Select different optimizers for actual tasks and adjust their parameters. The optimizers include SGD, Adam, and RMSprop.
[0044] Furthermore, the training evaluation module includes a model training unit and a model evaluation unit; wherein,
[0045] The model training unit continuously adjusts the weights during the training process through the back-propagation algorithm to minimize the loss function of the training samples; monitors the performance of the model during the training process by setting a validation set to avoid overfitting, and uses regularization to prevent overfitting, and finally generates the result model through multiple rounds of training;
[0046] The model evaluation unit uses the labeled evaluation samples to evaluate the performance of the result model. The evaluation index depends on the task type. The prediction accuracy of the model on the evaluation samples is obtained by calculating the ratio of the number of correctly predicted evaluation samples to the total number of evaluation samples. The performance of the model on the evaluation samples is then compared with the preset benchmark to evaluate whether the model meets the standard. If the evaluation result is up to standard, the model is regarded as the optimal model and stored in the model library. If the evaluation result is not up to standard, the model is redesigned, the hyperparameters are adjusted, the training data is increased, and the training and evaluation are re-performed.
[0047] Furthermore, the state recognition module includes a feature extraction unit and a feature recognition unit; wherein,
[0048] A feature extraction unit, used to extract portrait features and action behavior features;
[0049] Portrait features: Use the specific convolutional neural network in the optimal model to extract facial features, including the X and Y coordinate values of the five key feature points of the left eye, right eye, nose tip, left corner of the mouth, and right corner of the mouth, the X and Y coordinate values of the face frame, and a facial feature value array with a length of 512;
[0050] Action behavior features: Use the specific three-dimensional convolutional neural network in the optimal model to analyze the changes in pixels or feature points between consecutive frames and extract behavioral features, including walking posture, gestures, and action trajectories; specifically: input multiple consecutive frames in the video sequence, stack the frame images into a three-dimensional array, and slide the three-dimensional convolution kernel in three dimensions to perform convolution operations on the input data; the size and step size of the convolution kernel affect the size of the feature map, and multiple convolution kernels can extract different types of features; through multi-layer convolution and pooling operations, extract features from low-level to high-level layer by layer; low-level features include: edge and corner point information; high-level features include: complex motion patterns and behavior patterns; the three-dimensional convolutional neural network integrates spatial and temporal information to extract features with spatiotemporal consistency;
[0051] The feature recognition unit is used to combine the extracted features with the optimal model for feature comparison and attribute behavior analysis; calculate the similarity value between the extracted facial feature value and the facial feature value of a specific identity in the database. When the similarity value is greater than the set facial recognition confidence threshold, the two are determined to be the same person and the identity information of the person is identified; calculate various types of portrait inference values and various types of action behavior inference values. When the inference value is greater than the set confidence threshold of a certain type of facial attribute or a certain type of behavior confidence threshold, the various attributes and specific action behaviors of the person are identified.
[0052] Furthermore, the safety assessment module assesses potential risks according to personnel attributes, action behavior, environmental conditions, and time factors to obtain risk assessment results.
[0053] The present invention provides a method for safety identification and assessment of personnel in public areas of subways based on deep learning, and the steps are as follows:
[0054] Collect real-time video streaming data in public areas of subways;
[0055] Extract the video stream data into time-series single-frame images, process the time-series single-frame images one by one to achieve the unification of image samples, and generate training samples and evaluation samples;
[0056] Label the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks;
[0057] Select 2D / 3D convolutional neural network algorithm according to the actual task, and optimize the parameters according to the task situation and training evaluation results;
[0058] Use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library;
[0059] The optimal model is used to extract the features of human portraits and action behaviors from the real-time sequential single-frame image samples, and then matched with the optimal model to obtain the recognition results;
[0060] Analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe.
[0061] Beneficial effects of the present invention:
[0062] The present invention can effectively improve the safety level of personnel in public areas of subways through artificial intelligence technology, reduce the corresponding manual work intensity, and also has the following characteristics:
[0063] 1. The present invention achieves extremely fast and accurate recognition by generating an optimal model;
[0064] 2. The present invention achieves continuous improvement of capabilities through a self-learning model of training-evaluation-retraining;
[0065] 3. The present invention realizes efficient police linkage through accurate assessment and early warning of the safety level of subway public areas. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 The figure is a functional block diagram of the system of the present invention. DETAILED DESCRIPTION
[0067] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.
[0068] Reference Figure 1 As shown, a subway public area personnel safety identification and assessment system based on deep learning of the present invention includes: an acquisition module, a preprocessing module, a feature annotation module, an algorithm adaptation module, a training and evaluation module, a state identification module and a safety assessment module; wherein,
[0069] The acquisition module is used to collect real-time video stream data in the public areas of the subway;
[0070] The acquisition module is connected to multiple cameras via a network to collect video stream data of the public area of the subway in real time.
[0071] The preprocessing module is used to extract the video stream data into time-sequential single-frame images and ensure the time sequence between frames of the video; process the time-sequential single-frame images one by one to achieve the unification of image samples, thereby generating training samples and evaluation samples;
[0072] Wherein, the pre-processing module includes: a video processing unit and an image processing unit;
[0073] The video processing unit sets the frame position and time interval to extract the video stream data into a time-sequential single-frame image. The time sequence of the frame is ensured by recording the timestamp and adding verification and adjustment. While extracting the frame, the timestamp of each frame is recorded. After the extraction is completed, the time sequence of the frame is verified by checking the timestamp or sequence number of the frame. If the time sequence of the frame is disordered (such as jumping, repeating, etc.), corresponding adjustments will be made (such as re-extracting, deleting or inserting the frame);
[0074] The image processing unit improves the quality and efficiency of subsequent processing by preprocessing the image frames, including:
[0075] Image frame scaling: adjust the image frame size as needed to meet different processing requirements;
[0076] Grayscale: Converting color image frames into grayscale image frames can reduce the amount of calculation and increase processing speed;
[0077] Noise reduction: Reduce the noise points or noise areas in the image frame through filtering to make the image clear;
[0078] Image enhancement: Enhance the contrast and clarity of image frames through histogram equalization or sharpening, which helps subsequent processing and analysis tasks to more accurately identify targets or features in the image;
[0079] Normalization: Normalize the pixel values of the image frames to eliminate the differences between different frames caused by lighting factors, which helps to improve the robustness and accuracy of subsequent processing algorithms.
[0080] The feature annotation module is used to annotate the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks;
[0081] Wherein, the feature annotation module includes a portrait feature annotation unit and a behavior feature annotation unit;
[0082] The portrait feature annotation unit marks and locates the face area by drawing a rectangular frame on a single frame image, and marks the positions of key points in the face area, including the left and right eyes, the tip of the nose, and the left and right corners of the mouth; annotates the attributes of the face, including gender and age; annotates the face identity, and associates the face with specific identity information (such as name, ID number);
[0083] The behavioral feature labeling unit labels the action categories in the video frames. The specific labeled action categories include holding weapons, fighting, chasing, falling, squatting, smoking, arson, and crying. The time sequence includes the start phase, duration phase, and end phase of the action; and the corresponding labels or categories are assigned to the behavioral actions.
[0084] Algorithm adaptation module, which is used to select two-dimensional / three-dimensional convolutional neural network (2D / 3DCNN) algorithms (including LeNet, AlexNet, ZFNet, VGGNet, ResNet, etc.) according to actual tasks, and optimize parameters according to task conditions and training evaluation results;
[0085] The algorithm adaptation module determines the network structure and parameters according to the characteristics of the actual task (such as image recognition, video analysis and recognition, etc.); specifically, it is as follows:
[0086] When the actual task is a small-scale dataset, the LeNet two-dimensional convolutional neural network is used;
[0087] When the actual task is a large-scale image classification task, the AlexNet two-dimensional convolutional neural network is used;
[0088] When the actual task is an image classification task with high accuracy requirements, the ZFNet two-dimensional convolutional neural network is used;
[0089] When the actual task is image feature extraction, VGGNet two-dimensional convolutional neural network is used;
[0090] When the actual task is image recognition or image detection, the ResNet two-dimensional convolutional neural network is used;
[0091] When the actual task is to process the task of video (dynamic image) recognition containing time information, a three-dimensional convolutional neural network is used;
[0092] Parameter optimization settings include:
[0093] Learning Rate: Select the adaptive adjustment method to adjust the learning rate, automatically adjust the learning rate according to the indicator (such as the loss of the evaluation sample set), and reduce the learning rate when the indicator stops improving;
[0094] Batch Size: The batch size needs to be adjusted according to the actual task and data set;
[0095] Convolution kernel size and number: Select different combinations of convolution kernel size and number according to the actual task to find the optimal solution;
[0096] Regularization method: adjust the Dropout ratio or regularization coefficient to observe the changes in model performance to prevent overfitting;
[0097] Optimizer: Select different optimizers for actual tasks and adjust their parameters. The optimizers include SGD, Adam, and RMSprop.
[0098] The training and evaluation module is used to use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library;
[0099] Wherein, the training evaluation module includes a model training unit and a model evaluation unit; wherein,
[0100] The model training unit continuously adjusts the weights during the training process through the back-propagation algorithm to minimize the loss function of the training samples; monitors the performance of the model during the training process by setting a validation set (sometimes directly dividing a small part of the evaluation samples as a validation set) to avoid overfitting, and uses regularization to prevent overfitting, and finally generates the result model through multiple rounds of training;
[0101] The model evaluation unit uses the labeled evaluation samples to evaluate the performance of the result model. The evaluation index depends on the task type. The prediction accuracy of the model on the evaluation samples is obtained by calculating the ratio of the number of correctly predicted evaluation samples to the total number of evaluation samples. The performance of the model on the evaluation samples is then compared with the preset benchmark (or called the compliance standard) to evaluate whether the model meets the standard. If the evaluation result is that it meets the standard, the model is regarded as the optimal model and stored in the model library. If the evaluation result is not up to standard, the model is redesigned, the hyperparameters are adjusted, the training data is increased, and the training and evaluation are re-performed.
[0102] The state recognition module is used to extract the features of human portraits and action behaviors from real-time sequential single-frame image samples using the optimal model, and match them with the optimal model to obtain the recognition results;
[0103] Wherein, the state recognition module includes a feature extraction unit and a feature recognition unit;
[0104] A feature extraction unit, used to extract portrait features and action behavior features;
[0105] Portrait features: Use the specific convolutional neural network (CNN) in the optimal model to extract facial features, including the X and Y coordinate values of the five key feature points of the left eye, right eye, nose tip, left corner of the mouth, and right corner of the mouth, the X and Y coordinate values of the face frame, and a facial feature value array with a length of 512;
[0106] Action behavior features: Use the specific three-dimensional convolutional neural network (3D-CNN) in the optimal model to analyze the changes in pixels or feature points between consecutive frames and extract behavioral features, including walking posture, gestures, and action trajectories; specifically: input multiple consecutive frames in the video sequence, stack the frame images into a three-dimensional array (height, width, time / number of frames), and slide the three-dimensional convolution kernel in three dimensions (width, height, time) to perform convolution operations on the input data; the size and step size of the convolution kernel affect the size of the feature map, and multiple convolution kernels can extract different types of features; through multi-layer convolution and pooling operations, extract features from low-level to high-level layer by layer; low-level features include: edge and corner point information; high-level features include: complex motion patterns and behavior patterns; the three-dimensional convolutional neural network integrates spatial and temporal information to extract features with spatiotemporal consistency;
[0107] The feature recognition unit is used to combine the extracted features with the optimal model for feature comparison and attribute behavior analysis; calculate the similarity value between the extracted facial feature value and the facial feature value of a specific identity in the database. When the similarity value is greater than the set facial recognition confidence threshold, the two are deemed to be the same person and the identity information of the person is identified; calculate various portrait inference values (including age, gender, expression) and various action behavior inference values (including holding weapons, fighting, chasing, falling, squatting, smoking, arson, crying). When the inference value is greater than the set confidence threshold of a certain type of facial attribute or a certain type of behavior confidence threshold, the various attributes and specific action behaviors of the person are identified.
[0108] The safety assessment module is used to analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe;
[0109] Potential risks are assessed based on personnel attributes (including age, gender, facial expressions and special personnel), action behaviors, environmental conditions (such as temperature, humidity, smoke concentration, etc.), and time factors (such as nighttime, holidays, etc.) to obtain risk assessment results.
[0110] The present invention provides a method for safety identification and assessment of personnel in public areas of subways based on deep learning, and the steps are as follows:
[0111] Collect real-time video streaming data in public areas of subways;
[0112] Extract the video stream data into time-series single-frame images, process the time-series single-frame images one by one to achieve the unification of image samples, and generate training samples and evaluation samples;
[0113] Label the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks;
[0114] Select 2D / 3D convolutional neural network algorithm according to the actual task, and optimize the parameters according to the task situation and training evaluation results;
[0115] Use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library;
[0116] The optimal model is used to extract the features of human portraits and action behaviors from the real-time sequential single-frame image samples, and then matched with the optimal model to obtain the recognition results;
[0117] Analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe.
[0118] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.
Claims
1. A subway public area personnel safety identification and assessment system based on deep learning, characterized in that: include: Acquisition module, preprocessing module, feature annotation module, algorithm adaptation module, training evaluation module, state recognition module and safety assessment module; among them, The acquisition module is used to collect real-time video stream data in the public areas of the subway; The preprocessing module is used to extract the video stream data into time-sequential single-frame images and ensure the time sequence between frames of the video; process the time-sequential single-frame images one by one to achieve the unification of image samples, thereby generating training samples and evaluation samples; The feature annotation module is used to annotate the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks; Algorithm adaptation module, used to select 2D / 3D convolutional neural network algorithm according to actual tasks, and optimize parameters according to task conditions and training evaluation results; The training and evaluation module is used to use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library; The state recognition module is used to extract the features of human portraits and action behaviors from real-time sequential single-frame image samples using the optimal model, and match them with the optimal model to obtain the recognition results; The safety assessment module is used to analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe.
2. According to the deep learning-based subway public area personnel safety identification and assessment system according to claim 1, it is characterized in that: The acquisition module is connected to multiple cameras via a network to collect video stream data of the public area of the subway in real time.
3. According to the deep learning-based subway public area personnel safety identification and assessment system according to claim 1, it is characterized in that: The pre-processing module includes: a video processing unit and an image processing unit; The video processing unit sets the frame position and time interval to extract the video stream data into a time-sequential single-frame image. The time sequence of the frame is ensured by recording the timestamp and adding verification and adjustment. While extracting the frame, the timestamp of each frame is recorded. After the extraction is completed, the time sequence of the frame is verified by checking the timestamp or sequence number of the frame. If the time sequence of the frame is disordered, corresponding adjustments will be made. The image processing unit improves the quality and efficiency of subsequent processing by preprocessing the image frames, including: Image frame scaling: adjust the image frame size as needed to meet different processing requirements; Grayscale: Convert a color image frame into a grayscale image frame; Noise reduction: Reduce the noise points or noise areas in the image frame through filtering to make the image clear; Image enhancement: Enhance the contrast and clarity of image frames by histogram equalization or sharpening; Normalization: Normalize the pixel values of the image frame to eliminate the differences between different frames caused by lighting factors.
4. According to the deep learning-based subway public area personnel safety identification and assessment system according to claim 1, it is characterized in that: The feature annotation module includes a portrait feature annotation unit and a behavior feature annotation unit; The portrait feature annotation unit marks and locates the face area by drawing a rectangular frame on a single frame image, and marks the positions of key points in the face area, including the left and right eyes, the tip of the nose, and the left and right corners of the mouth; annotates the attributes of the face, including gender and age; annotates the face identity, and associates the face with specific identity information; The behavioral feature labeling unit labels the action categories in the video frames. The specific labeled action categories include holding weapons, fighting, chasing, falling, squatting, smoking, arson, and crying. The time sequence includes the start phase, continuation phase, and end phase of the action; the behavioral actions are assigned corresponding labels or categories.
5. The subway public area personnel safety identification and assessment system based on deep learning according to claim 1 is characterized in that: The algorithm adaptation module determines the network structure and parameters according to the characteristics of the actual task; specifically: When the actual task is a small-scale dataset, the LeNet two-dimensional convolutional neural network is used; When the actual task is a large-scale image classification task, the AlexNet two-dimensional convolutional neural network is used; When the actual task is an image classification task with high accuracy requirements, the ZFNet two-dimensional convolutional neural network is used; When the actual task is image feature extraction, VGGNet two-dimensional convolutional neural network is used; When the actual task is image recognition or image detection, the ResNet two-dimensional convolutional neural network is used; When the actual task is to process the task of video recognition containing time information, a three-dimensional convolutional neural network is used; Parameter optimization settings include: Learning rate: Select the adaptive adjustment method to adjust the learning rate, automatically adjust the learning rate according to the indicator, and reduce the learning rate when the indicator stops improving; Batch size: The batch size needs to be adjusted according to the actual task and dataset; Convolution kernel size and number: Select different combinations of convolution kernel size and number according to the actual task to find the optimal solution; Regularization method: adjust the Dropout ratio or regularization coefficient to observe the changes in model performance to prevent overfitting; Optimizer: Select different optimizers according to the actual task and adjust their parameters. The optimizers include SGD, Adam, and RMSprop.
6. The subway public area personnel safety identification and assessment system based on deep learning according to claim 1 is characterized in that: The training and evaluation module includes a model training unit and a model evaluation unit; The model training unit continuously adjusts the weights during the training process through the back-propagation algorithm to minimize the loss function of the training samples; monitors the performance of the model during the training process by setting a validation set to avoid overfitting, and uses regularization to prevent overfitting, and finally generates the result model through multiple rounds of training; The model evaluation unit uses the labeled evaluation samples to evaluate the performance of the result model. The evaluation index depends on the task type. The prediction accuracy of the model on the evaluation samples is obtained by calculating the ratio of the number of correctly predicted evaluation samples to the total number of evaluation samples. The performance of the model on the evaluation sample is then compared with the preset benchmark to evaluate whether the model meets the standard. If the evaluation result is that the model meets the standard, the model is considered to be the optimal model and is stored in the model library. If the evaluation result is not up to standard, redesign the model, adjust hyperparameters, increase training data, and retrain and evaluate.
7. The subway public area personnel safety identification and assessment system based on deep learning according to claim 1 is characterized in that: The state recognition module includes a feature extraction unit and a feature recognition unit; wherein, A feature extraction unit, used to extract portrait features and action behavior features; Portrait features: Use the specific convolutional neural network in the optimal model to extract facial features, including the X and Y coordinate values of the five key feature points of the left eye, right eye, nose tip, left corner of the mouth, and right corner of the mouth, the X and Y coordinate values of the face frame, and a facial feature value array with a length of 512; Action behavior features: Use the specific three-dimensional convolutional neural network in the optimal model to analyze the changes in pixels or feature points between consecutive frames and extract behavioral features, including walking posture, gestures, and action trajectories; specifically: input multiple consecutive frames in the video sequence, stack the frame images into a three-dimensional array, and slide the three-dimensional convolution kernel in three dimensions to perform convolution operations on the input data; the size and step size of the convolution kernel affect the size of the feature map, and multiple convolution kernels can extract different types of features; through multi-layer convolution and pooling operations, extract features from low-level to high-level layer by layer; low-level features include: edge and corner point information; high-level features include: complex motion patterns and behavior patterns; the three-dimensional convolutional neural network integrates spatial and temporal information to extract features with spatiotemporal consistency; The feature recognition unit is used to combine the extracted features with the optimal model for feature comparison and attribute behavior analysis; calculate the similarity value between the extracted facial feature value and the facial feature value of a specific identity in the database. When the similarity value is greater than the set facial recognition confidence threshold, the two are determined to be the same person and the identity information of the person is identified; calculate various types of portrait inference values and various types of action behavior inference values. When the inference value is greater than the set confidence threshold of a certain type of facial attribute or a certain type of behavior confidence threshold, the various attributes and specific action behaviors of the person are identified.
8. The subway public area personnel safety identification and assessment system based on deep learning according to claim 1 is characterized in that: The safety assessment module assesses potential risks according to personnel attributes, action behavior, environmental conditions, and time factors to obtain risk assessment results.
9. A method for identifying and assessing personnel safety in subway public areas based on deep learning, characterized in that: Here are the steps: Collect real-time video streaming data in public areas of subways; Extract the video stream data into time-series single-frame images, process the time-series single-frame images one by one to achieve the unification of image samples, and generate training samples and evaluation samples; Label the portrait features and action behavior type features of the training samples and evaluation samples according to the actual tasks; Select 2D / 3D convolutional neural network algorithm according to the actual task, and optimize the parameters according to the task situation and training evaluation results; Use the labeled training samples to train and generate the result model, and use the labeled evaluation samples to evaluate the result model, obtain the optimal model that meets the evaluation requirements and store it in the model library; The optimal model is used to extract the features of human portraits and action behaviors from the real-time sequential single-frame image samples, and then matched with the optimal model to obtain the recognition results; Analyze the recognition results and evaluate whether the personnel and abnormal behaviors in different scenarios are safe.