Track traffic abnormal event identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a rail transit abnormal event recognition method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, the field of rail transit safety or other related fields, and the recognition method comprises the steps: collecting video stream data and sensor data from a rail transit system, obtaining target data, and carrying out the preprocessing; under the condition that the target data is determined to be valid data, performing feature extraction based on a deep learning algorithm to obtain a feature set; calling a pre-deployed target detection model, inputting the feature set into the target detection model, and outputting a detection result; and generating a final identification result of the rail transit abnormal event based on the probability value of the suspected abnormal event in the detection result, the event type, the position information and the model detection confidence, and triggering a system alarm program based on the final identification result. According to the invention, the technical problem of low abnormal alarm accuracy caused by high false alarm and missing alarm rate of abnormal event identification of the rail transit system in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for identifying abnormal rail transit events, an electronic device, and a storage medium. Background Art

[0002] As an important urban infrastructure, the safe and efficient operation of the Urban Rail Transit System (Rail Transit System) is of great significance to the operation of the city and the people's lives. In recent years, with the construction and development of the rail transit system, rail transit operation safety incidents have frequently occurred, causing inconvenience to urban traffic operations and citizens' lives. The operation and maintenance of the rail transit system faces many challenges, including not only monitoring the normal operation status of rail transit trains, but also timely identification and early warning of abnormal events in the rail transit system. Common abnormal events in the rail transit system, such as train failures, loose track components, and rail transit train derailments, pose hidden dangers to the operational safety of the rail transit system.

[0003] Currently, the safe operation of rail transit systems relies primarily on daily monitoring and maintenance by rail transit operators, requiring significant human and material resources. To address the safety challenges of rail transit systems, computer vision-based rail transit anomaly detection systems, such as those based on computer vision, primarily use moving object detection techniques, such as background subtraction, frame difference, and optical flow, to identify and warn of anomalies in the system. However, these methods suffer from the following limitations: 1. The accuracy of moving object detection is limited; 2. Moving object detection, which relies on a single data source for video streams, has very limited accuracy in identification and warning; 3. Abnormal events in rail transit systems are often sudden, and the detection results of moving object detection technology have a certain lag, which is detrimental to the safe operation and maintenance of rail transit systems.

[0004] In summary, due to various technical limitations, traditional identification methods have high false alarm and missed alarm rates for abnormal event identification, which is not conducive to precise detection and accurate warning.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present invention provide a method and device for identifying rail transit abnormal events, an electronic device, and a storage medium to at least solve the technical problem in the related art that the false alarm and missed alarm rate of abnormal event identification in rail transit systems is high, resulting in low abnormal alarm accuracy.

[0007] According to one aspect of an embodiment of the present invention, a method for identifying rail transit abnormal events is provided, comprising: collecting video stream data and sensor data from a rail transit system to obtain target data, and preprocessing the target data; when it is determined that the target data is valid data after preprocessing, extracting features from the valid data based on a deep learning algorithm to obtain a feature set; calling a pre-deployed target detection model, inputting the feature set into the target detection model, and outputting a detection result, wherein the detection result includes at least: a probability value, event type, location information, and model detection confidence of a suspected abnormal event; generating a final recognition result of the rail transit abnormal event based on the probability value, event type, location information, and model detection confidence of the suspected abnormal event, and triggering a system alarm program based on the final recognition result.

[0008] Furthermore, the preprocessing includes the following operations: data cleaning, data conversion and preliminary feature extraction.

[0009] Furthermore, for the video stream data in the target data, the data cleaning includes: removing / correcting video noise and detecting abnormal frames, the data conversion includes: normalizing the numerical format, adjusting the resolution and converting the color space, and the preliminary feature extraction includes: edge feature extraction and texture feature extraction.

[0010] Furthermore, for the sensor data in the target data, the data cleaning includes: noise filtering and detecting outliers, the data conversion includes: signal type conversion and numerical format normalization, and the preliminary feature extraction includes: numerical feature extraction and time series feature extraction.

[0011] Furthermore, the step of determining whether the target data after preprocessing is valid data includes: judging whether the video stream data is valid, wherein the validity detection items of the video stream data include: image integrity detection, lighting condition detection, motion blur detection, occlusion detection and key frame detection; judging whether the sensor data is valid, wherein the validity detection items of the sensor data include: data consistency detection, signal strength detection, timestamp detection, data integrity detection and noise level detection; when it is determined that the video stream data is valid and the sensor data is also valid, determining that the target data after preprocessing is valid data.

[0012] Furthermore, the step of extracting features from the valid data based on a deep learning algorithm to obtain a feature set includes: using a convolutional neural network to perform multi-layer convolution and pooling operations on the preliminary features of the valid video stream data to obtain high-level visual features, wherein the high-level visual features include: object shape features and motion pattern features; performing feature fusion processing on the high-level visual features of the video stream data and the preliminary features of the sensor data to obtain a feature set composed of fused features, wherein the preliminary features of the sensor data include: numerical features or time series features.

[0013] Furthermore, the training step of the target detection model includes: obtaining model training data through the rail transit system, wherein the model training data is obtained after abnormal event types are marked on historical operation data, and the historical operation data at least includes video stream data and sensor data during the historical operation process; performing data enhancement processing on the model training data to obtain enhanced training data, wherein the data enhancement processing refers to adjusting the perspective, lighting conditions and / or posture of the original model training data; obtaining an initial detection model, and iteratively training the initial detection model using the enhanced training data, and adjusting the model parameters according to the model detection accuracy of each generation of training, until the model detection accuracy is higher than a preset accuracy threshold, stopping the iterative training, and obtaining the target detection model.

[0014] Furthermore, the step of generating a final identification result of a rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event includes: determining that the suspected abnormal event is a rail transit abnormal event when it is determined that the probability value of the suspected abnormal event is higher than a preset probability value, the event type belongs to the abnormal type range, and the model detection confidence is higher than a preset confidence threshold; recording the probability value, event type, location information and model detection confidence in an abnormal event report, and using the abnormal event report as the final identification result of the rail transit abnormal event.

[0015] Furthermore, the step of triggering a system alarm program based on the final recognition result includes: determining the risk level of the rail transit abnormal event based on the event type and the location information in the final recognition result; determining an alarm channel based on the risk level, and triggering the system alarm program according to the determined alarm channel, wherein the types of the alarm channels include: sound signal alarm, light signal alarm, SMS notification, email notification, and system pop-up window.

[0016] According to another aspect of an embodiment of the present invention, a device for identifying rail transit abnormal events is also provided, including: an acquisition unit, used to acquire video stream data and sensor data from a rail transit system, obtain target data, and preprocess the target data; a feature extraction unit, used to extract features from the valid data based on a deep learning algorithm to obtain a feature set when it is determined that the target data is valid data after preprocessing; an output unit, used to call a pre-deployed target detection model, input the feature set into the target detection model, and output a detection result, wherein the detection result includes at least: a probability value of a suspected abnormal event, an event type, location information, and a model detection confidence; an alarm unit, used to generate a final recognition result of the rail transit abnormal event based on the probability value, event type, location information, and model detection confidence of the suspected abnormal event, and trigger a system alarm program based on the final recognition result.

[0017] Furthermore, the acquisition unit includes: a pre-processing module, and the pre-processing module includes the following operation sub-modules: a data cleaning sub-module, a data conversion sub-module and a preliminary feature extraction sub-module.

[0018] Furthermore, for the video stream data in the target data, the data cleaning submodule is used to remove / correct video noise and detect abnormal frames, the data conversion submodule is used to perform numerical format normalization, resolution adjustment and color space conversion, and the preliminary feature extraction submodule is used to perform edge feature extraction and texture feature extraction.

[0019] Furthermore, for the sensor data in the target data, the data cleaning submodule is used to perform noise filtering and detect outliers, the data conversion submodule is used to perform signal type conversion and numerical format normalization, and the preliminary feature extraction submodule is used to perform numerical feature extraction and time series feature extraction.

[0020] Furthermore, the device for identifying rail transit abnormal events also includes: a first determination module. Due to the step of determining that the target data after preprocessing is valid data, the first determination module includes: a first judgment submodule, used to determine whether the video stream data is valid, wherein the validity detection items of the video stream data include: image integrity detection, lighting condition detection, motion blur detection, occlusion detection and key frame detection; a second judgment submodule, used to determine whether the sensor data is valid, wherein the validity detection items of the sensor data include: data consistency detection, signal strength detection, timestamp detection, data integrity detection and noise level detection; a determination submodule, used to determine that the target data after preprocessing is valid data when the video stream data is valid and the sensor data is also valid.

[0021] Furthermore, the feature extraction unit includes: an operation module for using a convolutional neural network to perform multi-layer convolution and pooling operations on the effective preliminary features of the video stream data to obtain high-level visual features, wherein the high-level visual features include: object shape features and motion pattern features; a feature fusion module for performing feature fusion processing on the high-level visual features of the video stream data and the preliminary features of the sensor data to obtain a feature set composed of fused features, wherein the preliminary features of the sensor data include: numerical features or time series features.

[0022] Furthermore, the device for identifying rail transit abnormal events also includes: a training module for the training step of the target detection model, and the training module includes: an acquisition submodule for acquiring model training data through the rail transit system, wherein the model training data is obtained after the abnormal event type is marked on the historical operation data, and the historical operation data at least includes video stream data and sensor data during the historical operation process; a data enhancement submodule for performing data enhancement processing on the model training data to obtain enhanced training data, wherein the data enhancement processing refers to adjusting the perspective, lighting conditions and / or posture of the original model training data; an iterative training submodule for obtaining an initial detection model, and iteratively training the initial detection model using the enhanced training data, and adjusting the model parameters according to the model detection accuracy of each generation of training, until the model detection accuracy is higher than the preset accuracy threshold, then stopping the iterative training to obtain the target detection model.

[0023] Furthermore, the alarm unit includes: a second determination module, used to determine that the suspected abnormal event is a rail transit abnormal event when it is determined that the probability value of the suspected abnormal event is higher than a preset probability value, the event type belongs to the abnormal type range, and the model detection confidence is higher than a preset confidence threshold; a recording module, used to record the probability value, event type, location information and model detection confidence in an abnormal event report, and use the abnormal event report as the final identification result of the rail transit abnormal event.

[0024] Furthermore, the alarm unit also includes: a third determination module, used to determine the risk level of the rail transit abnormal event based on the event type and the location information in the final identification result; a trigger module, used to determine the alarm channel based on the risk level, and trigger the system alarm program according to the determined alarm channel, wherein the types of the alarm channels include: sound signal alarm, light signal alarm, SMS notification, email notification and system pop-up window.

[0025] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for identifying rail transit abnormal events.

[0026] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned methods for identifying abnormal rail transit events.

[0027] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the steps of any one of the above-mentioned methods for identifying rail transit abnormal events are implemented.

[0028] In the present invention, a method for identifying rail transit abnormal events is proposed. First, video stream data and sensor data are collected from the rail transit system to obtain target data, and the target data is preprocessed. Then, when it is determined that the target data after preprocessing is valid data, feature extraction is performed on the valid data based on a deep learning algorithm to obtain a feature set. Then, a pre-deployed target detection model is called, the feature set is input into the target detection model, and a detection result is output, wherein the detection result at least includes: the probability value of the suspected abnormal event, the event type, the location information and the model detection confidence. Finally, the final recognition result of the rail transit abnormal event is generated based on the probability value, the event type, the location information and the model detection confidence of the suspected abnormal event, and the system alarm program is triggered based on the final recognition result.

[0029] In the present invention, a comprehensive analysis method based on a deep learning algorithm is adopted to achieve the purpose of significantly improving the accuracy of rail transit abnormal event detection and response efficiency through precise preprocessing and intelligent feature extraction. Specifically, video stream data and sensor data are obtained from the rail transit system to ensure the comprehensiveness of the information. The collected target data is preprocessed to improve the quality of the data and the efficiency of subsequent analysis. On the premise that it is determined to be valid data, feature extraction is performed based on the deep learning algorithm to form a feature set, and a pre-deployed target detection model trained by deep learning is input. The probability value of the suspected abnormal event, the specific event type, the precise location information and the confidence level of the model detection are output to constitute a preliminary judgment of the abnormal event. Finally, the preliminary judgment information generates a final recognition result, and the system alarm program is immediately activated to ensure that the abnormal event can be responded to and processed in a timely manner. This not only improves the accuracy of abnormal event detection, but also greatly enhances the responsiveness of the system, effectively avoids false alarms and missed alarms, and ensures the high accuracy and reliability of the rail transit safety early warning system. This solves the technical problem of high false alarm and missed alarm rates in the identification of abnormal events in rail transit systems in related technologies, resulting in low accuracy of abnormal alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1 is a flowchart of an optional method for identifying abnormal rail transit events according to an embodiment of the present invention;

[0032] Figure 2 is a schematic diagram of an optional data collection process according to an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of an optional data preprocessing process according to an embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of an optional feature extraction process according to an embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of an optional AI model training process according to an embodiment of the present invention;

[0036] Figure 6 is a schematic diagram of an optional anomaly detection process according to an embodiment of the present invention;

[0037] Figure 7 is a schematic diagram of an optional alarm process according to an embodiment of the present invention;

[0038] Figure 8 is a schematic diagram of an optional device for identifying abnormal rail transit events according to an embodiment of the present invention;

[0039] Figure 9 The figure is a structural block diagram of an electronic device for executing a method for identifying abnormal rail transit events according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] To facilitate those skilled in the art to understand the present invention, some of the terms or nouns involved in the embodiments of the present invention are explained below:

[0043] Deep learning algorithms, a type of machine learning technology, primarily use neural network models to learn and analyze complex data to achieve high-precision predictions and classifications. Deep learning algorithms can process large amounts of data and automatically learn the underlying features of that data. They are suitable for analyzing image, video, and sensor data, and have significant advantages in pattern recognition and anomaly detection.

[0044] The following embodiments of the present invention can be applied to various systems, applications, and devices requiring video surveillance of rail transit environments and anomaly detection of sensor data, enabling the implementation of a deep learning-based intelligent recognition system for rail transit anomaly events. This invention uses deep learning algorithms to intelligently analyze video streams and sensor data, extract key features, and predict anomaly events, effectively reducing false alarms and missed alarms and improving the accuracy of anomaly alerts.

[0045] During specific implementation, the present invention first collects real-time video stream data and sensor data from the rail transit system, and ensures the accuracy and applicability of the data through a data preprocessing module; then, based on a deep learning algorithm, feature extraction is performed on the preprocessed valid data to obtain a feature set; then, a pre-deployed target detection model is called, the feature set is input into the model, and the probability value, event type, location information and model detection confidence of the suspected abnormal event are output; finally, the final recognition result of the rail transit abnormal event is generated based on this key information, and the system alarm program is immediately started to ensure that the abnormal event can be responded to and handled in a timely manner.

[0046] By combining deep learning algorithms with multi-source data processing technology, this invention overcomes the limitations of traditional monitoring systems in complex environments, solves the problems of false alarms and missed alarms of abnormal events caused by improper data processing and inaccurate feature recognition, and significantly improves the accuracy and response speed of abnormal event detection. This technology is not only suitable for real-time monitoring of rail transit operational safety, but also provides intelligent decision-making support for rail transit integrated management platforms, promoting the further development of rail transit's intelligent and digital transformation, and improving rail transit operation and maintenance efficiency and safety management.

[0047] The present invention will be described in detail below with reference to various embodiments.

[0048] Example 1

[0049] According to an embodiment of the present invention, an embodiment of a method for identifying abnormal rail transit events is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] The implementation subject of the embodiment of the present invention can be an intelligent rail transit monitoring system, which combines deep learning algorithms with multi-source data fusion processing technology and is used in rail transit abnormal event detection scenarios, especially to reduce the problems of false alarm and missed alarm rates. By means of comprehensive video and sensor data analysis, the following steps are specifically included: data acquisition, preprocessing, feature extraction, model prediction and event location, so as to achieve the purpose of improving the accuracy of rail transit abnormal event identification and response efficiency.

[0051] The embodiment of the present invention utilizes a deep learning algorithm to process video stream data and sensor data from the rail transit system. Through precise data preprocessing and intelligent feature extraction, it can effectively identify abnormal events such as catenary fireworks, pipeline heating, extremely strong crosswinds, and extremely heavy rainfall. This not only improves the accuracy of event detection, but also ensures the timeliness and reliability of alarms through model prediction and confidence analysis. It is particularly suitable for safety monitoring of elevated sections of rail transit, helping to avoid safety accidents caused by abnormal events not being discovered in time, and improving the safety of rail transit and the passenger experience. Through the embodiment of the present invention, rail transit operators are able to grasp abnormal situations in the first place and take prompt measures to ensure the stable operation of the rail system and the safety of public travel.

[0052] The embodiments of the present invention are described in detail below in conjunction with specific implementation steps.

[0053] Figure 1 is a flow chart of an optional method for identifying abnormal rail transit events according to an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps:

[0054] Step S101 : collecting video stream data and sensor data from a rail transit system, obtaining target data, and preprocessing the target data.

[0055] Specifically, the rail transit system refers to various railway transportation systems including subways, light rails, and suburban railways, involving fixed infrastructure (such as tracks, platforms, and bridges), mobile equipment (such as trains and carriages), and supporting monitoring and management systems.

[0056] Video stream data originates from the rail transit system's camera network, including but not limited to surveillance cameras installed along tracks, inside and outside stations, and within train cars. These cameras continuously record the actual conditions of the rail transit environment. Video stream data refers to the video signals continuously transmitted from the rail transit system's surveillance cameras. These signals contain visual information (such as images, pedestrian activity, and train movements), and record changes in the environment over a specific time period. This data is used to reflect the rail transit environment and operational status in real time, and serves as a crucial source of information for identifying abnormal events.

[0057] Sensor data is environmental and equipment status information collected in real time by various sensors in the rail transit system. It covers multiple dimensions such as temperature, humidity, air pressure, vibration, acceleration, and can comprehensively monitor the operating status of the rail transit environment and equipment. For example, temperature sensors can monitor the thermal state of track facilities to prevent safety hazards caused by overheating; vibration sensors can detect track vibration to help promptly detect possible track damage or foreign object intrusion. Sensor data. In embodiments of the present invention, sensor data includes but is not limited to: environmental and equipment status parameters such as temperature, humidity, pressure, and vibration. These are collected in real time by sensors installed on tracks, trains, or stations and are used to monitor the physical state and environmental changes of the rail transit system.

[0058] Target data refers to all information captured from video streams and sensor data that reflects the current state of the rail transit system. After a preprocessing phase, the target data is cleaned and formatted to ensure legible video clips and accurate sensor signals, resulting in a high-quality dataset suitable for deep learning algorithm analysis and model training.

[0059] An optional embodiment, Figure 2 FIG. 1 is a schematic diagram of an optional data acquisition process according to an embodiment of the present invention, which is executed by a data acquisition module that is directly connected to a data preprocessing module to transmit the raw data to the next processing stage. Figure 2 As shown, the process includes:

[0060] (1) Start: the starting point of the process;

[0061] (2) Start data collection equipment: activate cameras and sensors installed in rail transit systems (such as stations, trains, and tracks);

[0062] (3) Video data acquisition: collecting real-time video data from cameras;

[0063] (4) Sensor data acquisition: collecting data from various sensors (such as speed, temperature, and pressure sensors);

[0064] (5) Data synchronization: ensuring that video data and sensor data are synchronized in time;

[0065] (6) Data buffering: storing the collected data in a buffer, ready for transmission to the next processing stage;

[0066] (7) Data transmission: transferring the data in the buffer to the data preprocessing module;

[0067] (8) Is the data transmission successful? Check whether the data is transmitted successfully;

[0068] (9) Yes: If the data transmission is successful, the process ends;

[0069] (10) No: If data transmission fails, perform error checking and retry;

[0070] (11) Error checking and retry: Check if there are any errors during data transmission and try to retransmit;

[0071] (12) End: Complete the data collection process and wait for the next cycle.

[0072] In the embodiment of the present invention, preprocessing involves data cleaning, denoising, format unification and preliminary feature extraction. Optionally, preprocessing includes the following operations: data cleaning, data conversion and preliminary feature extraction.

[0073] Optionally, for the video stream data in the target data, data cleaning includes: removing / correcting video noise and abnormal frame detection, data conversion includes: numerical format normalization, resolution adjustment and color space conversion, and preliminary feature extraction includes: edge feature extraction and texture feature extraction.

[0074] It's important to note that data cleaning can cause noise in video streams due to ambient light, electronic interference, or interference during signal transmission. The present invention uses advanced filtering algorithms, such as median filtering or bilateral filtering, to remove this noise while preserving important details and ensuring that subsequent analysis is not disrupted. Image stabilization technology can be used to correct blur caused by camera shake or unstable shooting, ensuring the clarity of each frame and improving the accuracy of abnormal event detection.

[0075] Abnormal frames may be caused by hardware failure, signal transmission interruption, or sudden environmental changes. By calculating the difference between adjacent frames (e.g., using frame difference or optical flow methods), the present invention can automatically identify video frames that significantly deviate from the normal range and remove or mark them, preventing abnormal frames from misleading subsequent feature extraction and analysis.

[0076] In terms of data conversion, for the pixel values ​​in the video stream data, all pixel values ​​are unified into the same numerical range (such as the [0,1] interval) through normalization processing, which facilitates model training and feature comparison, helps to improve the convergence speed of deep learning models, and reduces the time required for model training.

[0077] Considering that different cameras may provide videos with different resolutions, unifying the video resolution to a preset standard (such as HD resolution 1920x1080) can ensure that all data has a consistent visual scale when processed, which helps the deep learning model perform consistently on data from different cameras.

[0078] Color space conversion refers to converting the color space of video data from RGB to a format more suitable for analysis, such as HSV (hue, saturation, brightness). These color spaces can better reflect lighting changes and color characteristics, which is beneficial for subsequent feature extraction and abnormal event identification.

[0079] In terms of preliminary feature extraction, edges are important structural information in images and can highlight the boundary between the target and the background. The embodiments of the present invention can use edge detection or advanced edge detection algorithms to extract edge features from video frames and identify abnormal events with clear contours, such as foreign objects on the track; texture is important information in the image that reflects the surface characteristics of the object. The embodiments of the present invention can use texture analysis technologies such as gray-level co-occurrence matrix and local binary pattern to extract texture features of various areas in the video and identify subtle changes or surface damage (such as wear or cracks on the contact network).

[0080] Optionally, for sensor data in the target data, data cleaning includes: noise filtering and outlier detection, data conversion includes: signal type conversion and numerical format normalization, and preliminary feature extraction includes: numerical feature extraction and time series feature extraction.

[0081] It should be noted that in terms of data cleaning, sensor data may be affected by electromagnetic interference or equipment noise. The embodiments of the present invention can effectively remove high-frequency noise components in sensor signals by using low-pass filtering or adaptive filtering algorithms, thereby ensuring data purity and improving the sensitivity of abnormal event detection. Sometimes sensors may also produce abnormal readings due to failures or extreme environmental conditions. The embodiments of the present invention can also identify and isolate these outliers through statistical analysis (such as the 3σ principle) to avoid their negative impact on model training and event evaluation.

[0082] In terms of data conversion, analog signals can be converted into digital signals, and an analog-to-digital converter (ADC) can be used to ensure that sensor data can be effectively processed by the computer system, while also facilitating subsequent numerical operations and storage. The values ​​in the sensor data can also be normalized to the same range, so that different types of sensor data can be compared and fused on the same scale, which is conducive to the comprehensive analysis of multi-source data and the comprehensive identification of abnormal events.

[0083] In terms of preliminary feature extraction, key numerical features in sensor data, such as mean, maximum, minimum, standard deviation, etc., are extracted to reflect the changing trend of system status and identify sudden anomalies (such as sudden increase in temperature or abnormally increased vibration); the trend of sensor data changing over time can also be analyzed. By calculating the mean and peak interval within the sliding window, the continuity of event development can be revealed and the probability of future events can be predicted.

[0084] Through the above-mentioned data cleaning, conversion and preliminary feature extraction process, the embodiment of the present invention not only significantly improves the overall quality of video stream data and sensor data, but also specifically extracts feature information closely related to abnormal event identification, not only reduces the false alarm and missed alarm rates in the deep learning model training and abnormal event detection stages, but also additionally achieves the beneficial effects of improving data processing efficiency, enhancing data compatibility, improving model training quality and optimizing abnormal event identification.

[0085] An optional embodiment, Figure 3 FIG is a schematic diagram of an optional data preprocessing process according to an embodiment of the present invention. The process is executed by the data preprocessing module, and the processed data is transmitted to the feature extraction module. Figure 3 As shown, the process includes:

[0086] (1) Start: the starting point of the process;

[0087] (2) Receive data: Receive the original video stream and sensor data from the data acquisition module;

[0088] (3) Data cleaning: remove outliers and noise from the data, for example, remove snowflake noise from an image;

[0089] (4) Data conversion: converting raw data into a format suitable for analysis, for example, adjusting image resolution or converting sensor data into a unified data structure;

[0090] (5) Feature extraction: extracting useful features from the transformed data, for example, extracting motion patterns from a video;

[0091] (6) Feature scaling: Normalize the extracted features so that they are within the same numerical range to facilitate model processing;

[0092] (7) Output preprocessed data: Output the preprocessed data for use by the feature extraction module or model training module;

[0093] (8) End: The end point of the process.

[0094] Step S102: When it is determined that the target data after preprocessing is valid data, feature extraction is performed on the valid data based on a deep learning algorithm to obtain a feature set.

[0095] It's important to note that the validity of the target data after preprocessing is verified based on a series of rigorous data quality standards and verifications, checking for data integrity and clarity, as well as the removal of noise and outliers. Specifically, video stream data must ensure that each frame is clearly legible, without excessive blur or distortion; sensor data must ensure time series continuity and a reasonable numerical range. Only when the target data meets these conditions is it considered valid and can be used for feature extraction.

[0096] Optionally, the step of determining whether the preprocessed target data is valid data includes: judging whether the video stream data is valid, wherein the validity detection items of the video stream data include: image integrity detection, lighting condition detection, motion blur detection, occlusion detection and key frame detection; judging whether the sensor data is valid, wherein the validity detection items of the sensor data include: data consistency detection, signal strength detection, timestamp detection, data integrity detection and noise level detection; when it is determined that the video stream data is valid and the sensor data is also valid, determining that the preprocessed target data is valid data.

[0097] Specifically, image integrity detection refers to checking whether each frame in the video stream is complete and intact, ensuring that the image data is not truncated or missing, and ensuring that event information is fully obtained; lighting condition detection refers to evaluating whether the lighting conditions of the video stream are appropriate to avoid overly dim or overly bright environments affecting the identification of abnormal events. Lighting condition detection can be performed based on the brightness distribution of the image to ensure that high-quality visual data can be obtained under all conditions; motion blur detection refers to detecting whether there is motion blur in the video frame, and by calculating the sharpness index of each frame (such as the Laplace operator response), ensuring image clarity in dynamic scenes and improving the accuracy of abnormal event identification; occlusion detection refers to identifying whether there are occlusions in the video stream that affect the field of view, using target detection algorithms to perform occlusion analysis, ensuring that key areas are unobstructed, and accurately identifying abnormal events; key frame detection refers to identifying key frames in the video stream, that is, frames containing important information or abnormal events, and reducing redundant data by analyzing inter-frame differences (such as the frame difference method) and dynamic features, thereby improving the efficiency of event detection.

[0098] Data consistency detection refers to checking whether sensor data is consistent, including whether the data format and numerical range are as expected, to ensure that data from different sensors can be effectively integrated and compared; signal strength detection refers to evaluating whether the strength of the sensor signal meets the minimum threshold requirement, avoiding data loss or signal interpretation errors due to weak signals, and ensuring the reliability of data transmission; timestamp detection refers to checking whether the timestamps of sensor data are continuous, eliminating data inconsistencies caused by time synchronization problems, and ensuring the time series integrity of the data and the time positioning accuracy of events; data integrity detection refers to ensuring that there are no missing values ​​in sensor data, maintaining the integrity of the data sequence through data completion or interpolation algorithms, and avoiding the accuracy of analysis results affected by incomplete data; noise level detection refers to measuring whether the noise level of sensor data is within an acceptable range, ensuring data quality by calculating the standard deviation or signal-to-noise ratio, and improving the sensitivity of abnormal event detection.

[0099] Deep learning algorithms are a type of machine learning technology particularly well-suited for processing datasets with high complexity and nonlinear relationships. In embodiments of the present invention, deep learning algorithms are used to automatically learn feature representations from preprocessed video stream data and sensor data, eliminating the need for manual design of complex features and enabling more efficient and accurate detection of abnormal events in rail transit environments. Applied to rail transit anomaly detection scenarios, embodiments of the present invention can employ any of the following deep learning algorithms: convolutional neural networks, long short-term memory networks, and generative adversarial networks.

[0100] Feature extraction is a key step in the learning process of deep learning models, which involves automatically identifying and selecting features that best represent abnormal event information from valid data.

[0101] For video stream data, feature extraction can include the following aspects: object detection and classification, that is, using CNN models to detect and classify objects in video frames, and identify specific objects related to rail transit anomalies, such as pedestrians, fireworks, etc.; behavior analysis, that is, analyzing the movement patterns of objects through time series models to identify abnormal behaviors, such as abnormal movement trajectories or speeds; environmental perception, extracting environmental information from the video, such as lighting conditions, weather conditions, and background details that may affect the identification of abnormal events.

[0102] For sensor data, feature extraction can include: numerical features, that is, extracting key statistical values ​​of sensor readings, such as mean, maximum, minimum and standard deviation, to reflect changes in the status of equipment or environment; dynamic features, that is, using time series models to analyze the trend of sensor readings over time and identify abnormal dynamic patterns; correlation features, that is, by analyzing the correlation between different sensor data, extracting a multi-source feature set that can comprehensively reflect the status of the rail transit system.

[0103] The deep learning model extracts a set of features representing data characteristics from preprocessed data, including object shape, motion pattern, numerical changes, and time series characteristics, which are used for model training and identification of abnormal events.

[0104] Preferably, the step of extracting features from valid data based on a deep learning algorithm to obtain a feature set includes: using a convolutional neural network to perform multi-layer convolution and pooling operations on the preliminary features of valid video stream data to obtain high-level visual features, wherein the high-level visual features include: object shape features and motion pattern features; performing feature fusion processing on the high-level visual features of the video stream data and the preliminary features of the sensor data to obtain a feature set composed of fused features, wherein the preliminary features of the sensor data include: numerical features or time series features.

[0105] When valid data has been confirmed, an embodiment of the present invention uses a convolutional neural network (CNN) to perform deep feature extraction on video stream data. CNN automatically learns and extracts hierarchical structural features in images through multi-layer convolution and pooling operations. The convolution layer of CNN uses different filters (or convolution kernels) to convolve the input image to extract local features in the image, such as edges, textures, and color patterns. As the depth of the network increases, the convolution layer can capture more complex visual information, such as the local shape and structure of objects. The pooling layer in CNN is responsible for downsampling the feature map output by the convolution layer, reducing the feature dimension while retaining the most important information. The pooling layer uses methods such as maximum pooling or average pooling to not only reduce the amount of calculation, but also enhance the model's robustness to changes in the position of objects in the image.

[0106] Convolutional neural networks (CNNs) can extract high-level visual features from valid video stream data. Object shape features are used to describe the shape, outline, and structure of objects in the video, identifying specific objects (such as pedestrians and foreign objects) and assessing their impact on rail transit safety. Motion pattern features are used to analyze the trajectory and speed of objects in the video and detect abnormal behavior (such as pedestrians moving at unusual speeds or objects moving erratically).

[0107] After obtaining high-level visual features from the video stream data, feature fusion processing is performed to more comprehensively represent the state of the rail transit environment and potential abnormal events. Feature fusion can include the following technologies:

[0108] Feature splicing: splicing feature vectors from different sources together to form a complete feature representation;

[0109] Feature weighting: weighting different features according to their importance and relevance to ensure that key features play an important role in model decision making;

[0110] Feature mapping: Using neural network models such as autoencoders, high-dimensional features are mapped to low-dimensional space, which not only retains feature information but also avoids the curse of dimensionality and improves the generalization ability of the model.

[0111] An optional embodiment, Figure 4 This is a schematic diagram of an optional feature extraction process according to an embodiment of the present invention. The process is executed by the feature extraction module, and the extracted features are fed into the AI ​​model training module. Figure 4 As shown, the process includes:

[0112] (1) Start: the starting point of the process;

[0113] (2) Receiving pre-processed data: receiving cleaned and formatted data from the data pre-processing module;

[0114] (3) Select feature extraction method: Select appropriate feature extraction algorithm based on the characteristics of the data and the analysis objectives;

[0115] (4) Feature extraction: Apply selected methods to extract key features from the data that can effectively represent the data and capture abnormal behavior;

[0116] (5) Feature selection: Select the most relevant and informative features from all extracted features to reduce model complexity and improve performance;

[0117] (6) Output features: Output the selected features for use in subsequent model training or analysis steps;

[0118] (7) End: The end point of the process.

[0119] Step S103: call a pre-deployed target detection model, input the feature set into the target detection model, and output a detection result, wherein the detection result at least includes: the probability value of the suspected abnormal event, the event type, the location information, and the model detection confidence.

[0120] It should be noted that the target detection model is a deep learning model specifically used in embodiments of the present invention to identify and locate specific objects in images or videos. It is used to identify abnormal events in video streams and output the probability value, type, location, and detection confidence of the event. Here, abnormal events refer to abnormal or potentially dangerous situations that occur in the rail transit system, such as overhead line fireworks, pipeline heating, excessive crosswinds, and heavy rainfall. These events may pose a threat to the safety of rail transit operations.

[0121] More specifically, the detection result is the information output by the target detection model after analyzing the feature set. The probability value of a suspected abnormal event is the model's estimated probability of the abnormal event occurring, reflecting the confidence that the detected event is an abnormal event. The event type is the specific type of abnormal event identified by the model, such as overhead line fireworks, pipeline heating, or pedestrians entering the tracks. The location information is the specific location of the detected abnormal event within the rail transit system, which can accurately locate the location of the event. The confidence level indicates the model's degree of certainty about a specific prediction or classification result and is a value between 0 and 1. Higher values ​​indicate greater confidence in the result.

[0122] Optionally, the training steps of the target detection model include: obtaining model training data through the rail transit system, wherein the model training data is obtained after anomaly event types are marked on historical operation data, and the historical operation data at least includes video stream data and sensor data during the historical operation process; performing data enhancement processing on the model training data to obtain enhanced training data, wherein data enhancement processing refers to adjusting the perspective, lighting conditions and / or posture of the original model training data; obtaining an initial detection model, and iteratively training the initial detection model using the enhanced training data, and adjusting the model parameters according to the model detection accuracy of each generation of training, until the model detection accuracy is higher than a preset accuracy threshold, stopping the iterative training, and obtaining the target detection model.

[0123] This embodiment of the present invention uses historical operational data collected from the rail transit system as a training basis for the model. This data includes video streams and sensor readings from both normal and abnormal conditions, providing the model with rich learning material. Professionals annotate the collected historical operational data, clearly indicating which data contain abnormal events and the specific types of events (such as catenary fireworks, pipeline heating, pedestrian intrusion, etc.), ensuring that the model can learn the typical characteristics of abnormal events.

[0124] In the data enhancement process, perspective adjustment refers to simulating different perspectives by randomly rotating, flipping or cropping images, so that the model can identify abnormal events from multiple angles and improve robustness in practical applications; lighting condition adjustment refers to changing the brightness and contrast of the image to simulate different lighting environments, ensuring that the model can accurately identify abnormal events under various lighting conditions; posture adjustment refers to adjusting the posture of moving objects in the video in the image (such as tilt angle or movement direction) so that the model can learn to correctly identify objects in different postures.

[0125] During the iterative training and parameter adjustment process, the model is trained using the augmented training data. After each round of training, model parameters are adjusted based on the model's detection accuracy, such as optimizing the learning rate, adding or removing certain layers, and adjusting weight initialization, to continuously optimize model performance. The model's detection accuracy is regularly evaluated using the validation set to ensure that the model's performance improvements meet expectations and can effectively distinguish different types of abnormal events. When the model's detection accuracy exceeds the preset accuracy threshold, the training process stops, indicating that the model has achieved sufficient recognition accuracy to meet the actual needs of rail transit abnormal event detection.

[0126] Consider a scenario where a catenary fireworks event is recorded in historical operational data on an elevated rail transit section, including both video stream data and sensor data. Professionals annotate the event, indicating its type and time of occurrence. Subsequently, through data augmentation, the model simulates the appearance of fireworks under different viewing angles, lighting conditions, and postures, enabling it to learn the comprehensive characteristics of fireworks events. During model training, parameters are continuously adjusted based on the detection accuracy assessment results until the model can accurately identify all types of fireworks appearing in the video, including faint sparks at night and intense flames during the day, and accurately correlate abnormal readings in the sensor data, such as sudden temperature increases. The resulting object detection model is capable of not only identifying catenary fireworks but also other types of abnormal events, such as crosswinds and precipitation, providing comprehensive safety monitoring support for rail transit operations.

[0127] An optional Figure 5 This is a schematic diagram of an optional AI model training process according to an embodiment of the present invention. The process is executed by the AI ​​model training module, and the trained model is deployed to the anomaly detection module. Figure 5 As shown, the process includes:

[0128] (1) Start: The starting point of the process.

[0129] (2) Prepare training data: Collect and preprocess the dataset used to train the AI ​​model, which may include images, videos, or sensor data.

[0130] (3) Select model architecture: Select a suitable AI model architecture based on the characteristics of rail transit anomaly detection.

[0131] (4) Configure training parameters: Set the parameters required during training, such as learning rate, batch size, etc.

[0132] (5) Training model: Use the prepared data and parameters to train the AI ​​model.

[0133] (6) Evaluate the model: Evaluate the performance of the model through the validation set to determine the accuracy and efficiency of the model.

[0134] (7) Adjust the model: Adjust the model architecture or training parameters based on the results of model evaluation to optimize model performance.

[0135] (8) Save the model: Save the final trained model for subsequent deployment to the production environment.

[0136] (9) End: The end point of the process.

[0137] After the model training is completed, it is deployed to the anomaly detection module, which executes the anomaly detection process and immediately passes the information to the alarm module once an anomaly is detected.

[0138] An optional Figure 6 is a schematic diagram of an optional anomaly detection process according to an embodiment of the present invention, such as Figure 6 As shown, the process includes:

[0139] (1) Start: The starting point of the process.

[0140] (2) Receive real-time data: Receive video streams and sensor data from the rail transit system in real time.

[0141] (3) Preprocessing data: Perform necessary preprocessing on the received data to adapt it to the input requirements of the AI ​​model.

[0142] (4) Feature extraction: Extract key features from the preprocessed data, which are crucial for anomaly detection.

[0143] (5) Model prediction: Use the trained AI model to predict the extracted features and determine whether anomalies occur.

[0144] (6) Result analysis: Analyze the model’s prediction results to determine whether further action is needed.

[0145] (7) Is it abnormal? It is used to determine whether the model prediction results indicate abnormal conditions.

[0146] (8) Triggering an alarm: If an anomaly is detected, the alarm mechanism is triggered and the abnormal event is recorded so that appropriate measures can be taken.

[0147] (9) End: The end point of the process.

[0148] Step S104 : generating a final recognition result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and triggering a system alarm program based on the final recognition result.

[0149] It should be noted that the final recognition result refers to the suspected abnormal event information based on the model output. The deterministic event information generated after further analysis and verification is the basis for triggering the alarm program; the system alarm program is a series of response mechanisms automatically triggered by the system after identifying an abnormal event, aiming to quickly notify relevant personnel to take countermeasures, including but not limited to: sound and light alarms, SMS notifications, email warnings and pop-up windows within the system.

[0150] An optional embodiment, Figure 7 This is a schematic diagram of an optional alarm process according to an embodiment of the present invention. The process is executed by the alarm module and is also the final output, responsible for transmitting abnormal information to relevant personnel or abnormality handling systems. Figure 7 As shown, the process includes:

[0151] (1) Start: The starting point of the process.

[0152] (2) Receiving anomaly detection signal: Receives an anomaly occurrence signal from the anomaly detection module.

[0153] (3) Verify the anomaly: Verify the anomaly signal to ensure that it is not a false alarm.

[0154] (4) Confirm the abnormality: If the verification result confirms that the abnormality has indeed occurred, the alarm triggering process will be entered.

[0155] (5) False positive: If the verification result is a false positive, the process ends.

[0156] (6) Trigger alarm: Once an abnormality is confirmed, the alarm mechanism is triggered, such as sound alarm, flashing light, etc.

[0157] (7) Notify relevant personnel: Notify operators or maintenance personnel via email, text messages, application push, etc.

[0158] (8) Record abnormal events: Record the detailed information of abnormal events in the log system for subsequent analysis and auditing.

[0159] (9) End: The end point of the process.

[0160] Preferably, the step of generating the final identification result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event in step S104 includes: determining that the suspected abnormal event is a rail transit abnormal event when it is determined that the probability value of the suspected abnormal event is higher than a preset probability value, the event type belongs to the abnormal type range, and the model detection confidence is higher than a preset confidence threshold; recording the probability value, event type, location information and model detection confidence in an abnormal event report, and using the abnormal event report as the final identification result of the rail transit abnormal event.

[0161] It should be noted that the probability value of an event output by the model being identified as an anomaly must exceed the threshold preset by the system. For example, if the threshold is set to 0.8, it will be considered a suspected abnormal event only when the model predicts a probability greater than 0.8. In addition, a set of abnormal event types can be predefined, such as fire, equipment overheating, extreme weather conditions (crosswind, heavy snow, rainstorm), pedestrian or obstacle intrusion, etc. Only when the event type identified by the model matches the predefined abnormal type will the event be further confirmed as a suspected abnormal event. In addition, the model's confidence in the recognition result must exceed the preset threshold. For example, a confidence level exceeding 0.9 can ensure the reliability of the model's recognition results.

[0162] Once a suspected abnormal event is confirmed as a rail transit abnormal event, an abnormal event report will be generated, which includes: the probability value of the suspected abnormal event, event type, location information (including geographic location and relative position in the rail transit system), and model detection confidence.

[0163] Optionally, the abnormal event report may also include: key characteristic values, duration and trend, recommended response measures, scope of impact, detailed description, historical event comparison analysis, etc.

[0164] It should be noted that key feature values ​​refer to the quantification of key features of identified abnormal events, such as the size, shape, color, temperature rise, and other values ​​of abnormal objects; the duration of abnormal events and their changing trends help to judge the urgency and possible development direction of the event. For example, the temperature change trend of a pipeline heating event helps to predict whether it may escalate into a fire.

[0165] Analysis based on event type and impact can also generate recommended response measures, such as immediately activating the fire sprinkler system for overhead line fireworks, and promptly notifying the near-end train to slow down or stop for pedestrian intrusion, which helps to respond quickly; and assessing the geographical scope and system components that may be affected by abnormal events, which helps to determine the area and impact of emergency response.

[0166] In addition to basic information about the event, the abnormal event report can also provide a detailed description of the event, such as a description of the specific scenario in which the event occurred, which is helpful for on-site verification and subsequent event analysis; comparing the current event with similar historical events can provide an assessment of the similarity of the event and possible pattern predictions, which is helpful for the prevention of abnormal events and long-term trend analysis.

[0167] Preferably, the step of triggering the system alarm program based on the final recognition result in step S104 includes: determining the risk level of the rail transit abnormal event based on the event type and location information in the final recognition result; determining the alarm channel based on the risk level, and triggering the system alarm program according to the determined alarm channel, wherein the types of alarm channels include: sound signal alarm, light signal alarm, SMS notification, email notification and system pop-up window.

[0168] Based on the final identification results, the system automatically analyzes the potential risks posed by the incident based on its type and location, and determines the risk level. Risk levels can be categorized into multiple levels, such as low, medium, and high, with each level triggering different alarm procedures and response measures.

[0169] The risk level can be determined based on the following factors:

[0170] 1. Event type: Determine the extent of damage and urgency based on the nature of the event (e.g., fire, equipment overheating, extreme weather, etc.);

[0171] 2. Location information: The specific location where the incident occurred in the rail transit system, considering the impact of this location on passenger safety, rail transit operations, and the surrounding environment.

[0172] Select the most appropriate alert channel to notify relevant personnel based on the risk level:

[0173] 1. Sound signal alarm: Applicable to high-risk events, immediately send sound signals to rail transit staff on site to convey emergency information in the fastest way.

[0174] 2. Light signal alarm: Also used for high-risk or nighttime events, it attracts the attention of on-site workers or rescue teams through strong flashing lights.

[0175] 3. SMS notification: used for all levels of events to ensure instant notification to the rail transit operation center and key decision makers.

[0176] 4. Email notification: Applicable to post-event analysis or low- to medium-risk events, providing detailed event reports and follow-up action plans.

[0177] 5. System pop-up window: Pop-up warnings are displayed in the interface of the rail transit monitoring room or operation center, applicable to all levels of events, ensuring visual reminders for monitoring personnel.

[0178] Once the alarm channel is determined, the corresponding alarm procedure is automatically triggered:

[0179] 1. Immediate execution: For high-risk events, the system will immediately send an alarm to the site and operation center through sound signals, light signals, and SMS notifications.

[0180] 2. On-demand notification: For low- and medium-risk events, the system notifies relevant personnel via SMS and email, and displays a pop-up warning in the monitoring system, giving them ample time to prepare and respond.

[0181] Through the above steps S101 to S104, video stream data and sensor data can be first collected from the rail transit system to obtain target data, and the target data can be preprocessed. Then, when it is determined that the target data after preprocessing is valid data, feature extraction is performed on the valid data based on the deep learning algorithm to obtain a feature set. Then, a pre-deployed target detection model is called, the feature set is input into the target detection model, and a detection result is output, wherein the detection result includes at least: the probability value of the suspected abnormal event, the event type, the location information and the model detection confidence. Finally, based on the probability value, the event type, the location information and the model detection confidence of the suspected abnormal event, the final recognition result of the rail transit abnormal event is generated, and the system alarm program is triggered based on the final recognition result.

[0182] In an embodiment of the present invention, a comprehensive analysis method using a deep learning algorithm is adopted to achieve the purpose of significantly improving the accuracy of rail transit abnormal event detection and response efficiency through precise preprocessing and intelligent feature extraction. Specifically, video stream data and sensor data are obtained from the rail transit system to ensure the comprehensiveness of the information. The collected target data is preprocessed to improve the quality of the data and the efficiency of subsequent analysis. After it is determined to be valid data, feature extraction is performed based on the deep learning algorithm to form a feature set, and a pre-deployed target detection model trained by deep learning is input. The probability value of the suspected abnormal event, the specific event type, the precise location information, and the confidence level of the model detection are output to constitute a preliminary judgment of the abnormal event. Finally, the preliminary judgment information generates a final recognition result, and the system alarm program is immediately activated to ensure that the abnormal event can be responded to and handled in a timely manner. This not only improves the accuracy of abnormal event detection, but also greatly enhances the response capability of the system, effectively avoids false alarms and missed alarms, and ensures the high accuracy and reliability of the rail transit safety warning system. This solves the technical problem of high false alarm and missed alarm rates in the identification of abnormal events in rail transit systems in related technologies, resulting in low accuracy of abnormal alarms.

[0183] The present invention is described below in conjunction with another optional embodiment.

[0184] Example 2

[0185] The device for identifying abnormal rail transit events provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0186] Figure 8 is a schematic diagram of an optional device for identifying abnormal rail transit events according to an embodiment of the present invention, such as Figure 8 As shown, the device may include: a collection unit 801, a feature extraction unit 802, an output unit 803, and an alarm unit 804.

[0187] The acquisition unit 801 is used to acquire video stream data and sensor data from the rail transit system, obtain target data, and pre-process the target data.

[0188] The feature extraction unit 802 is used to extract features from the valid data based on a deep learning algorithm to obtain a feature set when it is determined that the target data after preprocessing is valid data.

[0189] The output unit 803 is used to call a pre-deployed target detection model, input a feature set into the target detection model, and output a detection result, wherein the detection result includes at least: a probability value of a suspected abnormal event, an event type, location information, and a model detection confidence.

[0190] The alarm unit 804 is used to generate a final recognition result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and trigger the system alarm program based on the final recognition result.

[0191] The above-mentioned device for identifying rail transit abnormal events can first collect video stream data and sensor data from the rail transit system through the acquisition unit 801 to obtain target data, and pre-process the target data. Then, when it is determined that the target data after pre-processing is valid data, the feature extraction unit 802 extracts features from the valid data based on the deep learning algorithm to obtain a feature set. Then, through the output unit 803, it is used to call a pre-deployed target detection model, input the feature set into the target detection model, and output the detection result, wherein the detection result includes at least: the probability value of the suspected abnormal event, the event type, the location information and the model detection confidence. Finally, through the alarm unit 804, it is used to generate the final recognition result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and trigger the system alarm program based on the final recognition result.

[0192] In an embodiment of the present invention, a comprehensive analysis method using a deep learning algorithm is adopted to achieve the purpose of significantly improving the accuracy of rail transit abnormal event detection and response efficiency through precise preprocessing and intelligent feature extraction. Specifically, video stream data and sensor data are obtained from the rail transit system to ensure the comprehensiveness of the information. The collected target data is preprocessed to improve the quality of the data and the efficiency of subsequent analysis. After it is determined to be valid data, feature extraction is performed based on the deep learning algorithm to form a feature set, and a pre-deployed target detection model trained by deep learning is input. The probability value of the suspected abnormal event, the specific event type, the precise location information, and the confidence level of the model detection are output to constitute a preliminary judgment of the abnormal event. Finally, the preliminary judgment information generates a final recognition result, and the system alarm program is immediately activated to ensure that the abnormal event can be responded to and handled in a timely manner. This not only improves the accuracy of abnormal event detection, but also greatly enhances the response capability of the system, effectively avoids false alarms and missed alarms, and ensures the high accuracy and reliability of the rail transit safety warning system. This solves the technical problem of high false alarm and missed alarm rates in the identification of abnormal events in rail transit systems in related technologies, resulting in low accuracy of abnormal alarms.

[0193] Optionally, the acquisition unit includes: a preprocessing module, which includes the following operation submodules: a data cleaning submodule, a data conversion submodule and a preliminary feature extraction submodule.

[0194] Optionally, for the video stream data in the target data, the data cleaning submodule is used to remove / correct video noise and detect abnormal frames, the data conversion submodule is used to perform numerical format normalization, resolution adjustment and color space conversion, and the preliminary feature extraction submodule is used to perform edge feature extraction and texture feature extraction.

[0195] Optionally, for the sensor data in the target data, the data cleaning submodule is used to perform noise filtering and detect outliers, the data conversion submodule is used to perform signal type conversion and numerical format normalization, and the preliminary feature extraction submodule is used to perform numerical feature extraction and time series feature extraction.

[0196] Optionally, the device for identifying abnormal rail transit events also includes: a first determination module. Due to the step of determining that the pre-processed target data is valid data, the first determination module includes: a first judgment submodule, used to determine whether the video stream data is valid, wherein the validity detection items of the video stream data include: image integrity detection, lighting condition detection, motion blur detection, occlusion detection and key frame detection; a second judgment submodule, used to determine whether the sensor data is valid, wherein the validity detection items of the sensor data include: data consistency detection, signal strength detection, timestamp detection, data integrity detection and noise level detection; a determination submodule, used to determine that the pre-processed target data is valid data when the video stream data is valid and the sensor data is also valid.

[0197] Optionally, the feature extraction unit includes: an operation module for using a convolutional neural network to perform multi-layer convolution and pooling operations on the preliminary features of valid video stream data to obtain high-level visual features, wherein the high-level visual features include: object shape features and motion pattern features; a feature fusion module for performing feature fusion processing on the high-level visual features of the video stream data and the preliminary features of the sensor data to obtain a feature set composed of fused features, wherein the preliminary features of the sensor data include: numerical features or time series features.

[0198] Optionally, the device for identifying rail transit abnormal events also includes: a training module for the training steps of the target detection model, the training module includes: an acquisition sub-module for acquiring model training data through the rail transit system, wherein the model training data is obtained after the abnormal event type is marked on the historical operation data, and the historical operation data at least includes video stream data and sensor data during the historical operation process; a data enhancement sub-module for performing data enhancement processing on the model training data to obtain enhanced training data, wherein the data enhancement processing refers to adjusting the perspective, lighting conditions and / or posture of the original model training data; an iterative training sub-module for obtaining an initial detection model, and iteratively training the initial detection model using the enhanced training data, and adjusting the model parameters according to the model detection accuracy of each generation of training, until the model detection accuracy is higher than the preset accuracy threshold, then stopping the iterative training to obtain the target detection model.

[0199] Optionally, the alarm unit includes: a second determination module, used to determine that the suspected abnormal event is a rail transit abnormal event when it is determined that the probability value of the suspected abnormal event is higher than a preset probability value, the event type belongs to the abnormal type range, and the model detection confidence is higher than a preset confidence threshold; a recording module, used to record the probability value, event type, location information and model detection confidence in an abnormal event report, and use the abnormal event report as the final identification result of the rail transit abnormal event.

[0200] Optionally, the alarm unit also includes: a third determination module, used to determine the risk level of the rail transit abnormal event based on the event type and location information in the final identification result; a trigger module, used to determine the alarm channel based on the risk level, and trigger the system alarm program according to the determined alarm channel, wherein the types of alarm channels include: sound signal alarm, light signal alarm, SMS notification, email notification and system pop-up window.

[0201] The above-mentioned rail transit abnormal event identification device may also include a processor and a memory. The above-mentioned acquisition unit 801, feature extraction unit 802, output unit 803, alarm unit 804, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0202] The processor includes a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured. By adjusting kernel parameters, the kernel calls a pre-deployed target detection model, inputs a feature set into the target detection model, and outputs a detection result. The final recognition result of the rail transit abnormal event is generated based on the probability value of the suspected abnormal event, the event type, the location information, and the model detection confidence. The final recognition result triggers the system alarm program.

[0203] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0204] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: collecting video stream data and sensor data from a rail transit system to obtain target data, and preprocessing the target data; when it is determined that the target data is valid data after preprocessing, extracting features from the valid data based on a deep learning algorithm to obtain a feature set; calling a pre-deployed target detection model, inputting the feature set into the target detection model, and outputting a detection result, wherein the detection result includes at least: the probability value, event type, location information and model detection confidence of the suspected abnormal event; generating a final recognition result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and triggering a system alarm program based on the final recognition result.

[0205] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for identifying rail transit abnormal events of any one of the above-mentioned embodiments.

[0206] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method for identifying rail transit abnormal events of any one of the above-mentioned embodiments.

[0207] Figure 9 is a structural block diagram of an electronic device for executing a method for identifying abnormal rail transit events according to an embodiment of the present invention, such as Figure 9 As shown, the electronic device may include: one or more ( Figure 9 Only one is shown) processor 902, memory 904, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0208] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for identifying rail transit abnormal events in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method for identifying rail transit abnormal events. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] It can be understood by those skilled in the art that Figure 9 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (MID), or a PAD. Figure 9 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 9 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 9 Different configurations shown.

[0210] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0211] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0212] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0213] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0214] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0215] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0216] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0217] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for identifying abnormal rail transit events, characterized in that: include: Collecting video stream data and sensor data from a rail transit system to obtain target data, and preprocessing the target data; When it is determined that the target data is valid data after preprocessing, feature extraction is performed on the valid data based on a deep learning algorithm to obtain a feature set; Calling a pre-deployed target detection model, inputting the feature set into the target detection model, and outputting a detection result, wherein the detection result at least includes: a probability value of a suspected abnormal event, an event type, location information, and a model detection confidence level; A final recognition result of the rail transit abnormal event is generated based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and a system alarm program is triggered based on the final recognition result.

2. The method for identifying abnormal rail transit events according to claim 1, characterized in that: The preprocessing includes the following operations: data cleaning, data conversion and preliminary feature extraction, wherein: For the video stream data in the target data, the data cleaning includes: removing / correcting video noise and detecting abnormal frames; the data conversion includes: normalizing the numerical format, adjusting the resolution and converting the color space; the preliminary feature extraction includes: edge feature extraction and texture feature extraction; For the sensor data in the target data, the data cleaning includes: noise filtering and detecting outliers, the data conversion includes: signal type conversion and numerical format normalization, and the preliminary feature extraction includes: numerical feature extraction and time series feature extraction.

3. The method for identifying abnormal rail transit events according to claim 1, characterized in that: The step of determining that the target data after preprocessing is valid data includes: Determining whether the video stream data is valid, wherein the validity detection items of the video stream data include: image integrity detection, lighting condition detection, motion blur detection, occlusion detection, and key frame detection; Determining whether the sensor data is valid, wherein the validity detection items of the sensor data include: data consistency detection, signal strength detection, timestamp detection, data integrity detection and noise level detection; When it is determined that the video stream data is valid and the sensor data is also valid, the preprocessed target data is determined to be valid data.

4. The method for identifying abnormal rail transit events according to claim 2, characterized in that: The step of extracting features from the valid data based on a deep learning algorithm to obtain a feature set includes: Using a convolutional neural network to perform multi-layer convolution and pooling operations on the effective preliminary features of the video stream data to obtain high-level visual features, wherein the high-level visual features include: object shape features and motion pattern features; The high-level visual features of the video stream data are subjected to feature fusion processing with the preliminary features of the sensor data to obtain a feature set consisting of fused features, wherein the preliminary features of the sensor data include: numerical features or time series features.

5. The method for identifying abnormal rail transit events according to claim 1, characterized in that: The training steps of the target detection model include: Acquiring model training data through the rail transit system, wherein the model training data is obtained by annotating historical operation data with abnormal event types, and the historical operation data at least includes video stream data and sensor data during the historical operation process; Performing data enhancement processing on the model training data to obtain enhanced training data, wherein the data enhancement processing refers to performing perspective adjustment, lighting condition adjustment, and / or posture adjustment on the original model training data; An initial detection model is obtained, and the enhanced training data is used to iteratively train the initial detection model, and the model parameters are adjusted according to the model detection accuracy of each generation of training. When the model detection accuracy is higher than a preset accuracy threshold, the iterative training is stopped to obtain the target detection model.

6. The method for identifying abnormal rail transit events according to claim 1, characterized in that: The step of generating a final recognition result of a rail transit abnormal event based on the probability value, event type, location information, and model detection confidence of the suspected abnormal event includes: When it is determined that the probability value of the suspected abnormal event is higher than a preset probability value, the event type belongs to the abnormal type range, and the model detection confidence is higher than a preset confidence threshold, the suspected abnormal event is determined to be a rail transit abnormal event; The probability value, event type, location information, and model detection confidence are recorded in an abnormal event report, and the abnormal event report is used as the final recognition result of the rail transit abnormal event.

7. The method for identifying abnormal rail transit events according to claim 1, characterized in that: The step of triggering a system alarm program based on the final recognition result includes: Determining a risk level of the rail transit abnormal event based on the event type and the location information in the final recognition result; An alarm channel is determined based on the risk level, and the system alarm program is triggered according to the determined alarm channel, wherein the types of the alarm channel include: sound signal alarm, light signal alarm, SMS notification, email notification and system pop-up window.

8. A device for identifying abnormal rail transit events, characterized in that: include: An acquisition unit, configured to acquire video stream data and sensor data from a rail transit system, obtain target data, and pre-process the target data; A feature extraction unit is configured to, when determining that the target data after preprocessing is valid data, extract features from the valid data based on a deep learning algorithm to obtain a feature set; an output unit, configured to call a pre-deployed target detection model, input the feature set into the target detection model, and output a detection result, wherein the detection result includes at least: a probability value of a suspected abnormal event, an event type, location information, and a model detection confidence level; The alarm unit is used to generate a final recognition result of the rail transit abnormal event based on the probability value, event type, location information and model detection confidence of the suspected abnormal event, and trigger a system alarm program based on the final recognition result.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for identifying rail transit abnormal events according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The system comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for identifying rail transit abnormal events as described in any one of claims 1 to 7.

Images