Rail car safety state real-time monitoring and early warning system based on deep learning

Abstract

The invention relates to the field of rail traffic intelligent driving, and discloses a rail car safety state real-time monitoring and early warning system based on deep learning, which comprises a multi-modal data acquisition module for deploying sensors on key components of a rail car and acquiring time sequence data including vibration, temperature, pressure and operation parameters; the cross-modal feature conversion module encodes the collected time sequence data into a three-channel color image; the safety state evaluation module is used for performing deep modeling and feature learning on the three-channel color image, predicting a current state classification result of the rail car in real time, and obtaining a safety state evaluation result of the rail car; and the early warning and intelligent decision-making module is used for carrying out anomaly detection and alarm prompt when potential risks exist in the rail car according to the safety state evaluation result. According to the invention, high-precision early fault identification and automatic early warning are realized.

Description

Technical Field

[0001] The present invention relates to the field of rail transit intelligent driving technology, and in particular to a real-time monitoring and early warning system for the safety status of rail vehicles based on deep learning. Background Art

[0002] As rail transit systems evolve toward higher speeds and greater intelligence, the operational safety of rail vehicles has become a critical component of urban rail and trunk transportation systems. Traditional rail vehicle safety monitoring systems typically rely on fixed-point monitoring or regular manual inspections. These systems suffer from issues such as insufficient monitoring coverage, poor real-time performance, and delayed early warnings. These systems struggle to meet the requirements for early fault detection and rapid response during long-term, continuous rail vehicle operation.

[0003] In recent years, with the rapid development of sensor technology, multimodal data acquisition, and edge computing equipment, rail vehicles can now collect various types of data such as vibration, temperature, pressure, and speed in real time during operation. However, how to effectively integrate these heterogeneous time series data, mine potential operational anomaly information, and use deep learning models to achieve high-precision safety status identification and early warning remains a major challenge in the current field of intelligent monitoring. In particular, in the face of the high-dimensional, nonlinear, and time-dependent characteristics of rail vehicle operation data, traditional data processing methods have difficulty extracting key features, resulting in low fault diagnosis accuracy. Moreover, existing technologies generally only use a single channel, such as Gram angle field conversion time series data, which cannot capture state jumps and nonlinear characteristics, and rely on fully supervised training, making it difficult to adapt to small sample fault scenarios.

[0004] Therefore, there is an urgent need for a data modeling and intelligent recognition method with stronger feature expression and discrimination capabilities to improve the safety assurance capabilities of rail vehicles throughout their life cycle. Summary of the Invention

[0005] The present invention aims to provide a real-time monitoring and early warning system for the safety status of rail vehicles based on deep learning, to solve the problems of poor real-time performance and weak feature extraction capabilities of traditional monitoring methods, and to achieve high-precision early fault identification and automated early warning.

[0006] To achieve the above objectives, the following technical solutions are adopted:

[0007] A real-time monitoring and early warning system for rail vehicle safety status based on deep learning, including:

[0008] Multimodal data acquisition module, used to deploy sensors on key components of railcars and collect time series data including vibration, temperature, pressure, and operating parameters;

[0009] A cross-modal feature conversion module, configured to encode the collected time series data into a three-channel color image, comprising a Markov transition field channel constructed based on state transition probabilities, a recursive graph channel constructed based on state similarities, and a Gram angular field channel constructed based on angular relationships;

[0010] A safety status assessment module is used to perform deep modeling and feature learning on the three-channel color image, predict the classification result of the current state of the railcar in real time, and obtain a safety status assessment result of the railcar;

[0011] The early warning and intelligent decision-making module is used to detect anomalies and issue alarms when potential risks exist in rail vehicles based on the safety status assessment results.

[0012] Furthermore, the multimodal data acquisition module specifically includes:

[0013] Vibration sensors, temperature sensors, pressure sensors, and speed and acceleration sensors are deployed on wheels, axles, motors, brake systems, and vehicle body structures. The data collected by these sensors forms a multimodal raw data stream.

[0014] The multimodal original data stream is pre-processed by the edge device to form a data stream in a unified format.

[0015] Furthermore, the cross-modal feature conversion module performs the following processing:

[0016] The pre-processed time series data is discretized into a state sequence, and a Markov transition field is constructed to capture the state jump information as the first channel;

[0017] By setting a similarity threshold, a recurrence graph is constructed to characterize the nonlinear dynamic characteristics of the time series as the second channel;

[0018] The normalized data are mapped into polar coordinate angles, and the Gram angle field is constructed to characterize the local temporal correlation as the third channel;

[0019] The three channels are fused into a 224×224×3 three-channel color image.

[0020] Furthermore, the security status assessment module includes:

[0021] a feature extraction unit, which extracts temporal image features of the three-channel color images respectively through a shared feature extractor, performs dimensionality compression, and outputs a high-dimensional feature vector;

[0022] A dual-stream processing unit includes a main classification branch and an auxiliary comparison branch. For the high-dimensional feature vector output by the shared feature extractor, the main classification branch receives the feature vectors of similar samples in the support set and calculates the arithmetic mean of the high-dimensional feature vectors to generate prototype vectors for each category; the auxiliary comparison learning branch optimizes the feature space by aggregating features of similar samples and separating features of heterogeneous samples to generate an optimized feature space structure;

[0023] The meta-learning training unit is used to model the railcar safety assessment problem as an N-way K-shot small-sample classification problem, construct the small-sample classification task and train the model end-to-end. It optimizes the network parameters by weighted fusion of classification loss and contrastive loss, and outputs the optimal parameters for shared feature extractor training convergence.

[0024] Furthermore, the security status assessment module further includes:

[0025] A real-time classification decision unit is used to make fast predictions using only the main classification branch in new tasks. Specifically, it includes:

[0026] The sensor data collected in real time is mapped into the feature space using the trained and optimized shared feature extractor;

[0027] Query sample images in real time, convert them into high-dimensional feature vectors, and compare them with the Euclidean distance of prototype vectors of each category;

[0028] Select the prototype corresponding to the category with the smallest distance as the current state classification result, including the safety state label and class probability distribution vector;

[0029] When an abnormal state is detected, early warning information is automatically generated including the abnormality type, location and treatment suggestions.

[0030] Furthermore, the shared feature extractor includes:

[0031] A four-layer cascade convolution module is used to map the input three-channel color image into a 512-dimensional time-series image feature vector; wherein each convolution module in the four-layer cascade convolution module includes a convolution layer, a batch normalization layer and a ReLU activation function, which is used to extract time-frequency domain features of different scales.

[0032] Furthermore, the four-layer cascade convolution module specifically includes:

[0033] The first convolution module performs a 3×3 convolution operation to generate a 64-channel feature map. After ReLU activation and 2×2 maximum pooling, it outputs a 112×112×64 dimensional feature map for extracting the basic edge features of the railcar state jump;

[0034] The second convolution module performs grouped convolution operations to generate 128-channel feature maps, which are then batch normalized to output 56×56×128 dimensional features to enhance the ability to capture nonlinear dynamic behavior characteristics.

[0035] The third convolution module performs a 3×3 dilated convolution with a dilation rate of 2 to generate a 256-channel feature map. After spatial pyramid pooling, it outputs a 28×28×256 dimensional feature map for fusing multi-scale temporal structure features.

[0036] The fourth convolution module performs global average pooling dimensionality reduction and maps it to a 512-dimensional temporal image feature vector through a fully connected layer.

[0037] Furthermore, the main classification branch adopts a small-sample meta-learning mechanism to model the railcar safety status as an N-class K-sample support set task;

[0038] The operations performed by the auxiliary comparison branch include:

[0039] Label the training batch samples with similar / different relationship labels;

[0040] Construct positive and negative sample pairs and impose spatial constraints: impose a feature distance reduction constraint on similar sample pairs; impose a feature distance increase constraint on heterogeneous sample pairs, and set a minimum interval threshold. When the distance between heterogeneous samples is less than the minimum interval threshold, a separation penalty is triggered.

[0041] Output the optimized feature space structure.

[0042] Furthermore, the operations performed by the meta-learning training unit include:

[0043] A small sample task of sampling N categories and K samples from historical data, where each task contains a support set and a query set;

[0044] The training process jointly optimizes the cross entropy loss of the main classification branch and the supervised contrast loss of the auxiliary branch, and forms the final loss function through weighted fusion;

[0045] Iteratively update the shared feature extractor parameters to enable it to have both classification and discrimination capabilities and feature space structure optimization capabilities.

[0046] Furthermore, the early warning and intelligent decision-making module includes:

[0047] A hierarchical warning unit is used to match the preset risk level according to the safety status assessment results, generate warning information including the abnormality type, occurrence location and predicted consequences; and trigger a graphical interface warning and voice broadcast through the human-computer interaction interface;

[0048] A response strategy generation unit is used to automatically generate handling suggestions based on the abnormal risk level and operation priority, including at least one of deceleration, station parking for maintenance, and emergency braking; and send linkage control instructions to the vehicle control system;

[0049] The closed-loop optimization unit is used to record warning information and response execution results to the system log; based on the log data, the expert rule library and model training data set are updated.

[0050] Compared with the prior art, the present invention achieves the following beneficial effects:

[0051] 1. This paper proposes a time series visualization method based on Markov transition fields (MTFs), recurrence plots (RPs), and Gram's angle fields (GAFs) to encode raw railcar sensor time series data into three-channel color images. This method effectively captures dynamic amplitude changes, nonlinear evolutionary behavior, and local temporal structure during railcar operation, thereby enhancing feature representation and providing unified high-dimensional input for deep neural networks.

[0052] 2. This paper constructs a dual-stream feature enhancement network structure, consisting of a main classification branch and an auxiliary contrastive learning branch, which share a multi-scale feature extractor. This structure not only achieves efficient few-shot classification through the prototype network, but also fully utilizes the auxiliary branch to structurally optimize the feature embedding space, improving the model's discriminative performance for multiple complex operating states.

[0053] 3. This paper proposes an auxiliary supervised contrastive learning branch mechanism. By constructing positive and negative sample pairs and introducing supervisory signals, it enhances the model's ability to learn semantic boundaries between categories. This method effectively reduces the feature distance between samples in the same operating state and increases the embedding distance between samples in different states, thereby improving the model's recognition accuracy for subtle anomalies and critical states.

[0054] 4. This paper designs a supervised contrastive loss function and a final weighted loss function structure. By jointly optimizing classification accuracy and embedding space structure during model training, it guides the feature extractor to learn more discriminative embedding representations. The weighted loss combines cross-entropy loss and contrastive loss, ensuring strong generalization and robustness even with limited samples.

[0055] In summary, by combining time series visualization technology, prototype learning structure and supervised comparative learning mechanism, this paper constructs an efficient and scalable rail vehicle safety status monitoring system, which significantly improves the accuracy of fault identification and the timeliness of early warning response, and provides intelligent, data-driven technical support for the safe operation of rail transit.

[0056] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0058] Figure 1 1 is a module diagram of a rail vehicle safety status real-time monitoring and early warning system based on deep learning according to an embodiment of the present invention;

[0059] Figure 2 1 is a schematic diagram of the overall architecture of a rail vehicle safety status real-time monitoring and early warning system based on deep learning according to an embodiment of the present invention;

[0060] Figure 3 2 is a schematic diagram of a dual-stream network structure of a rail vehicle safety status real-time monitoring and early warning system based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0063] Figure 1 1 is a module diagram of a rail vehicle safety status real-time monitoring and early warning system based on deep learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall architecture of a real-time monitoring and early warning system for rail vehicle safety status based on deep learning according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, a rail vehicle safety status real-time monitoring and early warning system 100 based on deep learning includes:

[0064] Multimodal data acquisition module 110, for deploying sensors on key components of railcars and collecting time series data including vibration, temperature, pressure, and operating parameters;

[0065] Furthermore, the multimodal data acquisition module 110 specifically includes:

[0066] Various types of high-precision sensors are deployed on key components such as wheels, axles, motors, braking systems, and body structures to build an all-round, multi-modal perception system. Specifically, it includes:

[0067] Vibration sensors: installed on wheels, axles and chassis structures, monitor the structure in real time for abnormal vibrations, bearing wear or track inequality;

[0068] Temperature sensors: deployed in motors, cables, brake pads, and other locations to monitor heating trends and provide early warning of possible overload or short-circuit faults.

[0069] Pressure sensor: used to monitor the working status of the hydraulic brake system and suspension system, and determine whether there is leakage or abnormal pressure;

[0070] Speed ​​and acceleration sensors: used to collect operating parameter data of rail vehicles, analyze the motion status of rail vehicles during travel, and identify whether there are dangerous behaviors such as sudden acceleration and deceleration, and slipping;

[0071] The time series signals collected by these sensors form a multimodal raw data stream. This data is preprocessed by edge devices (such as denoising, normalization, and time synchronization), formatted uniformly, and uploaded to backend systems, providing high-quality data input for subsequent deep learning models.

[0072] A cross-modal feature conversion module 120 is used to encode the collected time series data into a three-channel color image, including a Markov transition field channel constructed based on state transition probability, a recursive graph channel constructed based on state similarity, and a Gram angle field channel constructed based on angular relationship;

[0073] To efficiently identify and warn of railcar safety status, it's necessary to perform cross-modal feature conversion on the large amount of time series data collected by sensors (such as wheel axle vibration, motor temperature, brake pressure, and operating speed). This data must be encoded into images to adapt to the input structure of deep learning networks. To this end, a time series visualization method based on Markov transition fields (MTFs), recurrence plots (RPs), and Gram's angle fields (GAFs) is proposed. The specific process executed by the cross-modal feature conversion module 120 is as follows:

[0074] (1) Constructing Markov transition field (MTF)

[0075] Discretize the preprocessed time series data into a state sequence. For example, discretize the preprocessed time series data into 10 state sequences using the equal-width binning method, and construct a Markov transition field to capture state jump information as the first channel.

[0076] MTF is used to capture the dynamic changes in the operating status of key components of railcars (such as vibration amplitude, temperature fluctuation, etc.) and encode the state transition probability of a one-dimensional time series into the R channel of the image:

[0077]

[0078] in, : The element in the i-th row and j-th column of the Markov transition field matrix represents the transition of the railcar state from discrete state to discrete state. Transition to The conditional probability of :from Transition to The statistical probability of ,reflects the state change trend; : The state value after the original sensor data is normalized and discretized.

[0079] This method enhances the "state jump" information during rail vehicle operation and is particularly effective in detecting abnormal states (such as sudden faults).

[0080] (2) Constructing a recursive graph (RP)

[0081] By setting a similarity threshold, a recurrence graph is constructed to characterize the nonlinear dynamic characteristics of the time series as the second channel;

[0082] RP is used to reveal the nonlinear dynamic characteristics of railcar operation data and is suitable for constructing the G channel of the image. This graph can represent the repeatability and trajectory similarity of the system state on the time axis:

[0083]

[0084] in, : The value of row i and column j in the recursive graph, representing the time point and Are the states similar enough? A step function that outputs 1 if the expression in the brackets is positive and 0 otherwise. : Similarity threshold, used to adjust the sensitivity to railcar state changes. Its optional value is 1.5 times the standard deviation of the time series (i.e., ϵ = 1.5σ) to adapt to the data fluctuation range under different railcar working conditions. : Time point and The Euclidean distance between them can be expanded to:

[0085]

[0086] Among them, k represents the different dimensions of the time series; Represents the reading of the kth sensor channel at the i-th time point (such as the k-th vibration axis or temperature sensor); Represents the reading of the kth sensor channel (such as the kth vibration axis or temperature sensor) at the jth time point.

[0087] This diagram helps to uncover the differences in railcar system behavior patterns under normal and abnormal operating conditions.

[0088] (3) Constructing Gram Angle Field (GAF)

[0089] The normalized data are mapped into polar coordinate angles, and the Gram angle field is constructed to characterize the local temporal correlation as the third channel;

[0090] GAF is used to express the correlation structure of railcar time series signals within a local time range, and can encode the angle information between time points into the B channel of the image:

[0091]

[0092] in, : The value at position (i, j) in the Gram angular field matrix, representing the time point and The angle relationship between them. : Angle representation of time series data, defined as:

[0093]

[0094] in, : The normalized value of the railcar sensor data at the i-th time point. X: The complete normalized time series data (such as a speed fluctuation series or a temperature change series). min(X): The minimum value in the time series X, used to normalize the data so that all data points are mapped to the same scale range. max(X): The maximum value in the time series X, used to normalize the data so that all data points are mapped to the same scale range. arccos(·): The inverse cosine function, which maps the normalized data to the polar coordinate projection of the angular range [0,π].

[0095] This method can effectively capture periodic changes or trend changes in operating status, and help improve the model's ability to perceive weak fault characteristics.

[0096] (4) Constructing a three-channel fusion image

[0097] Finally, the MTF, RP, and GAF ​​images from the three channels are fused into a three-channel color image with a dimension of 224×224×3, forming a unified data input format:

[0098]

[0099] : The fused color image is used The three channels encode dynamic amplitude changes, nonlinear behavior patterns, and local temporal structures in the time series. : The Markov transition field is used for the R channel to represent the dynamic amplitude changes of the time series. : The recurrence graph is used for the G channel to represent the nonlinear characteristics of the time series. : The Gram angle field is used for the B channel to represent the local temporal relationship of the time series.

[0100] The present invention utilizes these three complementary channels: MTF is more sensitive to sudden faults (such as bearing fracture), RP can identify nonlinear oscillations (such as abnormal wheel-rail friction), and GAF ​​can capture periodic degradation (such as motor wear). The integration of these three channels covers all railcar fault modes. Compared to the present invention, existing technologies use only single Gram angle field conversion time series data, which cannot capture state transitions and nonlinear characteristics. Furthermore, they rely on fully supervised training, making them difficult to adapt to small sample size fault scenarios.

[0101] Through the above-mentioned visualization process, the original multi-dimensional time series data of the rail vehicle is uniformly converted into image data, which not only retains the key characteristics of the signal but also enhances the spatial structure expression capability of the data, providing strong input support for subsequent safety status identification and fault warning.

[0102] A safety status assessment module 130 is configured to perform deep modeling and feature learning on the three-channel color image, predict the classification result of the current state of the railcar in real time, and obtain a safety status assessment result of the railcar;

[0103] The core objective of the safety status assessment module 130 is to perform deep modeling and feature learning on multi-source data (imaged time series data (three-channel color images)) during the operation of the railcar, to assess the safety status of the railcar in real time, and to perform accurate anomaly detection and alarm prompts when potential risks are discovered.

[0104] In order to achieve accurate safety status discrimination, this paper designs a dual-stream feature enhancement network, which includes a main classification branch and an auxiliary contrast learning branch, both of which share a multi-scale feature extractor. . Figure 3 2 is a dual-stream network structure diagram of an embodiment of the present invention.

[0105] Furthermore, the security status assessment module 130 includes:

[0106] The feature extraction unit 131 extracts the temporal image features of the three-channel color image through a shared feature extractor, performs dimensionality compression, and outputs a high-dimensional feature vector;

[0107] The shared feature extractor 1311 is used to embed the imaged time series data input into an M-dimensional feature space. Specifically, it maps the input three-channel color image with a dimension of 224×224×3 into a 512-dimensional time series image feature vector. Its structure includes a four-layer cascaded convolution module. Each convolution module in the four-layer cascaded convolution module includes a convolution layer, a batch normalization layer, and a ReLU activation function to extract time-frequency domain features of different scales. The four-layer cascaded convolution module specifically includes:

[0108] The first convolution module performs a 3×3 convolution operation to generate a 64-channel feature map. After ReLU activation and 2×2 maximum pooling, it outputs a 112×112×64 dimensional feature map for extracting the basic edge features of the railcar state jump;

[0109] The second convolution module performs grouped convolution operations to generate 128-channel feature maps, which are then batch normalized to output 56×56×128 dimensional features to enhance the ability to capture nonlinear dynamic behavior characteristics.

[0110] The third convolution module performs a 3×3 dilated convolution with a dilation rate of 2 (i.e., the convolution kernel element spacing is 2) to generate a 256-channel feature map. After spatial pyramid pooling, it outputs a 28×28×256 dimensional feature map for fusing multi-scale temporal structure features.

[0111] The fourth convolution module performs global average pooling dimensionality reduction and maps it to a 512-dimensional temporal image feature vector through a fully connected layer.

[0112] Dual stream processing unit 132, such as Figure 3 As shown, it includes a main classification branch and an auxiliary comparison branch. For the high-dimensional feature vector output by the shared feature extractor 1311, the main classification branch receives the feature vectors of similar samples in the support set and calculates the arithmetic mean of the high-dimensional feature vector to generate prototype vectors of each category; the auxiliary comparison learning branch optimizes the feature space by aggregating the features of similar samples and separating the features of heterogeneous samples to generate an optimized feature space structure;

[0113] The main classification branch is implemented based on the prototype network. The main classification branch adopts a small sample learning mechanism to model the railcar safety status as an N-class K-sample support set task. For example, let the sample set of a certain category n in the support set be , whose prototype vector is defined as:

[0114]

[0115] in, :category The prototype embedding vector of represents the central feature of the category. : The support set of category n, which contains all training samples belonging to category n, is mathematically expressed as: ; Indicates the number of samples of category n in the support set. For example, if category “normal” has 3 samples, then |S 正常 |=3; if there are 2 samples in the category “mild anomaly”, then |S 轻度异常 ∣=2. : The u-th sample in the support set. :sample The category label of . : Shared feature extractor Feature map of the input sample.

[0116] Assume that railcars have three safety states: normal (category 0), mild anomaly (category 1), and severe anomaly (category 2). The support set sample distribution is as follows: for the normal category (n=0), the number of samples is |3|, and the sample feature vectors are [0.2, -0.1], [0.3, 0.0], [0.1, 0.1] |; for the mild anomaly category (n=1), the number of samples is |2|, and the sample feature vectors are [1.8, 0.5], [2.0, 0.3]; for the severe anomaly category (n=2), the number of samples is |2|, and the sample feature vectors are [3.2, -1.0], [3.5, -0.8] . Prototype vector calculation process:

[0117] Normal class prototype (n=0):

[0118]

[0119] Mild abnormal prototype (n=1):

[0120]

[0121] Severe abnormal prototype (n=2):

[0122]

[0123] The auxiliary supervised contrastive learning branch is used to enhance the model's discriminative ability by constructing positive and negative sample pairs, narrowing the feature representations between similar samples and widening the distance between different samples. The auxiliary contrastive branch performs the following operations: labeling the training batch samples with similar / different-class labels; constructing positive and negative sample pairs and applying spatial constraints: for similar pairs, a constraint that reduces the feature distance; for different pairs, a constraint that increases the feature distance; and setting a minimum separation threshold. When the distance between different samples falls below this minimum separation threshold, a separation penalty is triggered; and finally, outputting the optimized feature space structure.

[0124] Specifically, the supervised contrast loss of the auxiliary supervised contrastive learning branch is expressed as follows:

[0125]

[0126] : A supervised contrast loss function that measures the model's ability to aggregate similar samples and separate heterogeneous samples in the current training batch; : A training sample pair, where are two samples sampled from the training set; : The label indicator value of the sample pair, if and Belong to the same category (i.e. the same operating status of the railcar), then ;otherwise ; : Shared feature extractor Feature mapping of input samples; :sample The Euclidean distance in the feature space reflects the similarity between their representations; : The interval hyperparameter is the minimum interval threshold (its value range is 0.5-1.0), which is used to set the minimum distance that should be maintained between samples of different categories in the feature space to prevent feature overlap. The optional m=1.0; :When the distance between samples of different categories is less than the interval When , a penalty term is generated to push the model to expand the distance between classes; : Encourage similar samples to be as close as possible in the feature space; : Penalize the situation where samples of different classes are too close in the feature space.

[0127] The meta-learning training unit 133 is used to model the rail vehicle safety assessment problem as an N-way K-shot small-sample classification problem, construct a small-sample classification task and perform end-to-end training on the model, optimize network parameters by weighted fusion classification loss and contrast loss, and output the optimal parameters for shared feature extractor training convergence.

[0128] During the meta-training phase:

[0129] The railcar safety assessment problem is modeled as an N-way K-shot small-sample classification problem. Historical fault data is split into thousands of N-category K-shot tasks, where N represents the number of railcar safety status categories (e.g., 5 categories: normal, mechanical failure, electrical failure, braking anomaly, and track anomaly), and K represents the number of support samples per category (3-5). Thousands of training tasks are constructed by randomly combining different fault fragments from the historical data, simulating real-world small-sample scenarios. This allows the model to quickly adapt with only a small number of new fault samples, overcoming the limitation of traditional models that require massive amounts of labeled data. During the meta-training phase, support and query set tasks are sampled from different safety status categories (e.g., normal, minor anomaly, and major anomaly), and the model is trained end-to-end.

[0130] The operations performed by the meta-learning training unit include: sampling small sample tasks of N categories and K samples (such as 5 categories and 3 samples) from historical data, each task contains a support set and a query set; the training process jointly optimizes the cross-entropy loss of the main classification branch and the supervised contrast loss of the auxiliary branch, and forms the final loss function through weighted fusion; iteratively updates the shared feature extractor parameters so that it has both classification discrimination capabilities and feature space structure optimization capabilities.

[0131] In the N-class K-sample task.

[0132] Specifically, the entire training goal is to minimize the following weighted loss function:

[0133]

[0134] : Final loss function; : Weight coefficient, balancing the importance of the classification branch and the contrastive learning branch. Optional, λ1 takes 0.6, λ2 takes 0.4. The ratio is determined by cross-validation to balance the classification accuracy and feature space separability. : Supervised contrastive loss for auxiliary contrastive learning branch. : Cross entropy loss of the main classification branch, used to optimize the shared feature extractor The parameter expression is:

[0135]

[0136] : Query sample Belong to category Through continuous iterative training, the optimized feature extractor parameters are obtained. :

[0137]

[0138] : The optimal parameters of the optimized shared feature extractor are trained and used to initialize the model in the meta-test phase; : Shares the current parameter variables of the feature extractor and continuously updates them during training to optimize model performance; T: A small sample classification task constructed from the historical operation data of the railcar, including the support set and the query set, representing a meta-training iteration; : The sampling distribution of the training task T, which is used to extract task samples from the task space in each round of meta-training to ensure that the model has good generalization ability; : The expected value operation for all possible training tasks T, which represents the average loss under all task distributions; :On the current task T, the parameter is The loss function value calculated by the model takes into account the classification accuracy and feature embedding discrimination ability.

[0139] The real-time classification decision unit 134 is used to perform rapid prediction using only the main classification branch in a new task, specifically including: mapping the sensor data collected in real time to the feature space using a trained and optimized shared feature extractor; querying the sample image in real time, converting it into a high-dimensional feature vector and comparing it with the Euclidean distance of each category prototype vector; selecting the prototype corresponding to the category with the smallest distance as the current state classification result, including the safety state label and the class probability distribution vector; and automatically generating early warning information containing the abnormality type, occurrence location and disposal suggestions when an abnormal state is detected.

[0140] During the meta-testing phase:

[0141] The real-time classification decision unit 134 uses the optimal parameters obtained by meta-training , only the main classification branch is used for fast prediction in the new task. For a given query sample , calculate the distance between it and various prototype vectors, and predict its category:

[0142]

[0143] : Final prediction result, that is, query sample The railcar is judged to be in a safe state (e.g., normal operation, slight abnormality, severe abnormality, etc.); : The query samples to be classified come from the real-time collected rail vehicle operation data (such as imaged sensor signals); : The optimal parameters of the shared feature extractor after meta-training optimization Perform feature embedding mapping on the input sample and transform the input data Convert to high-dimensional feature vector; : The prototype vector of category n, which represents the average feature representation of the railcar safety status level (normal, mild abnormality, severe abnormality), is obtained by the support samples of this category through the optimized shared feature extractor Extraction obtained; : Similarity measurement function, using Euclidean distance, is used to measure the distance between the query sample and the prototype of each category in the embedding space; : Indicates that among all known categories n, the prototype vector closest to the query sample is selected , as the basis for the final prediction category.

[0144] Safety status labels include: normal, mild abnormality, and severe abnormality. The class probability distribution vector is a three-dimensional probability array. For example, the class probability distribution vector format: [0.02, 0.15, 0.83] indicates an 83% probability of severe abnormality. For a normal status level, the fault type is none, and the action recommended is "continuous monitoring." For a mild abnormality, the fault type is, for example, axle imbalance, and the action recommended is "slow down to 80 km / h and proceed to the next station for inspection." For a severe abnormality, the fault type is, for example, brake pad overheating, and the action recommended is "stop the vehicle immediately and activate the cooling system."

[0145] The present invention improves the accuracy of small sample scenarios by jointly optimizing classification and feature space through a dual-stream network. The meta-learning framework achieves rapid adaptation of fault types, which is significantly better than traditional supervised learning.

[0146] The early warning and intelligent decision-making module 140 is used to detect anomalies and issue alarms when potential risks exist in the railcar based on the safety status assessment results.

[0147] After completing the assessment of the railcar's current operating status and detecting anomalies, the system will enter the early warning and intelligent decision-making support stage. When the anomaly risk reaches the set threshold, the system automatically generates an early warning message and prompts the driver or control center through a graphical interface or voice broadcast. The warning content includes the type of anomaly, the location of occurrence, the predicted consequences, and response suggestions. Combined with the railcar's operating tasks and operational priorities, the system can automatically generate scheduling suggestions, such as: recommending deceleration and entering low-power mode; recommending parking for maintenance at the nearest station; and linking the control system to trigger the emergency braking mechanism. All early warning and response suggestions will be recorded in the system log as the basis for subsequent model optimization and expert rule updates, forming a data-driven closed-loop intelligent early warning system. Through this module, the railcar safety monitoring system can not only detect current anomalies, but also assist in formulating reasonable response measures, effectively improving the safety and operational efficiency of the rail transit system.

[0148] Specifically, the early warning and intelligent decision-making module 140 includes:

[0149] The hierarchical warning unit 141 is used to match the preset risk level according to the safety status assessment results, generate warning information including the abnormality type, occurrence location and predicted consequences, and trigger graphical interface warnings and voice broadcasts through the human-computer interaction interface;

[0150] The response strategy generation unit 142 is used to automatically generate a handling suggestion based on the abnormal risk level and the operation priority, including at least one of deceleration, station parking for maintenance, and emergency braking; and send a linkage control instruction to the vehicle control system;

[0151] The closed-loop optimization unit 143 is used to record warning information and response execution results to the system log; and update the expert rule base and model training data set based on the log data.

[0152] Optionally, the closed-loop optimization unit 143 further expands the support set samples through a semi-supervised algorithm based on the false positive / missing negative samples in the log, and updates the prototype vector database once a month; the expert rule base adopts a decision tree model to dynamically adjust the risk level threshold.

[0153] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0154] It should also be noted that, in the embodiments of the present application, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device comprising the elements.

[0155] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the embodiments of the present application may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown in the embodiments of the present application, but rather will conform to the widest scope consistent with the principles and novel features disclosed in the embodiments of the present application.

Claims

1. A real-time monitoring and early warning system for rail vehicle safety status based on deep learning, characterized in that: include: Multimodal data acquisition module, used to deploy sensors on key components of railcars and collect time series data including vibration, temperature, pressure, and operating parameters; A cross-modal feature conversion module, configured to encode the collected time series data into a three-channel color image, comprising a Markov transition field channel constructed based on state transition probabilities, a recursive graph channel constructed based on state similarities, and a Gram angular field channel constructed based on angular relationships; A safety status assessment module is used to perform deep modeling and feature learning on the three-channel color image, predict the classification result of the current state of the railcar in real time, and obtain a safety status assessment result of the railcar; The early warning and intelligent decision-making module is used to detect anomalies and issue alarms when potential risks exist in rail vehicles based on the safety status assessment results.

2. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The multimodal data acquisition module specifically includes: Vibration sensors, temperature sensors, pressure sensors, and speed and acceleration sensors are deployed on wheels, axles, motors, brake systems, and vehicle body structures. The data collected by these sensors forms a multimodal raw data stream. The multimodal original data stream is pre-processed by the edge device to form a data stream in a unified format.

3. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 2 is characterized in that: The cross-modal feature conversion module performs the following processing: The pre-processed time series data is discretized into a state sequence, and a Markov transition field is constructed to capture the state jump information as the first channel; By setting a similarity threshold, a recurrence graph is constructed to characterize the nonlinear dynamic characteristics of the time series as the second channel; The normalized data are mapped into polar coordinate angles, and the Gram angle field is constructed to characterize the local temporal correlation as the third channel; The three channels are fused into a 224×224×3 three-channel color image.

4. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 3 is characterized in that: The security status assessment module includes: a feature extraction unit, which extracts temporal image features of the three-channel color images respectively through a shared feature extractor, performs dimensionality compression, and outputs a high-dimensional feature vector; A dual-stream processing unit includes a main classification branch and an auxiliary comparison branch. For the high-dimensional feature vector output by the shared feature extractor, the main classification branch receives the feature vectors of similar samples in the support set and calculates the arithmetic mean of the high-dimensional feature vectors to generate prototype vectors for each category; the auxiliary comparison learning branch optimizes the feature space by aggregating features of similar samples and separating features of heterogeneous samples to generate an optimized feature space structure; The meta-learning training unit is used to model the railcar safety assessment problem as an N-way K-shot small-sample classification problem, construct the small-sample classification task and train the model end-to-end. It optimizes the network parameters by weighted fusion of classification loss and contrastive loss, and outputs the optimal parameters for shared feature extractor training convergence.

5. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 4 is characterized in that: The security status assessment module further includes: A real-time classification decision unit is used to make fast predictions using only the main classification branch in new tasks. Specifically, it includes: The sensor data collected in real time is mapped into the feature space using the trained and optimized shared feature extractor; Query sample images in real time, convert them into high-dimensional feature vectors, and compare them with the Euclidean distance of prototype vectors of each category; Select the prototype corresponding to the category with the smallest distance as the current state classification result, including the safety state label and class probability distribution vector; When an abnormal state is detected, early warning information is automatically generated including the abnormality type, location and treatment suggestions.

6. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 5 is characterized in that: The shared feature extractor comprises: A four-layer cascade convolution module is used to map the input three-channel color image into a 512-dimensional time-series image feature vector; wherein each convolution module in the four-layer cascade convolution module includes a convolution layer, a batch normalization layer and a ReLU activation function, which is used to extract time-frequency domain features of different scales.

7. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 6 is characterized in that: in, The four-layer cascade convolution module specifically includes: The first convolution module performs a 3×3 convolution operation to generate a 64-channel feature map. After ReLU activation and 2×2 maximum pooling, it outputs a 112×112×64 dimensional feature map for extracting the basic edge features of the railcar state jump; The second convolution module performs grouped convolution operations to generate 128-channel feature maps, which are then batch normalized to output 56×56×128 dimensional features to enhance the ability to capture nonlinear dynamic behavior characteristics. The third convolution module performs a 3×3 dilated convolution with a dilation rate of 2 to generate a 256-channel feature map. After spatial pyramid pooling, it outputs a 28×28×256 dimensional feature map for fusing multi-scale temporal structure features. The fourth convolution module performs global average pooling dimensionality reduction and maps it to a 512-dimensional temporal image feature vector through a fully connected layer.

8. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 7 is characterized in that: in, The main classification branch adopts a small-sample meta-learning mechanism to model the railcar safety status as an N-class K-sample support set task; The operations performed by the auxiliary comparison branch include: Label the training batch samples with similar / different relationship labels; Construct positive and negative sample pairs and impose spatial constraints: impose a feature distance reduction constraint on similar sample pairs; impose a feature distance increase constraint on heterogeneous sample pairs, and set a minimum interval threshold. When the distance between heterogeneous samples is less than the minimum interval threshold, a separation penalty is triggered. Output the optimized feature space structure.

9. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 8 is characterized in that: The operations performed by the meta-learning training unit include: A small sample task of sampling N categories and K samples from historical data, where each task contains a support set and a query set; The training process jointly optimizes the cross entropy loss of the main classification branch and the supervised contrast loss of the auxiliary branch, and forms the final loss function through weighted fusion; Iteratively update the shared feature extractor parameters to enable it to have both classification and discrimination capabilities and feature space structure optimization capabilities.

10. The rail vehicle safety status real-time monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The early warning and intelligent decision-making module includes: A hierarchical warning unit is used to match the preset risk level according to the safety status assessment results, generate warning information including the abnormality type, occurrence location and predicted consequences; and trigger a graphical interface warning and voice broadcast through the human-computer interaction interface; A response strategy generation unit is used to automatically generate handling suggestions based on the abnormal risk level and operation priority, including at least one of deceleration, station parking for maintenance, and emergency braking; and send linkage control instructions to the vehicle control system; The closed-loop optimization unit is used to record warning information and response execution results to the system log; based on the log data, the expert rule library and model training data set are updated.

Images