Metro vehicle state evaluation system based on trackside data fusion

Abstract

The invention relates to the technical field of metro vehicle state evaluation, in particular to a metro vehicle state evaluation system based on trackside data fusion, which is characterized in that a visual image feature extraction module generates rim structural features from rim visual images detected by a visual image acquisition device through a rim feature extraction model based on YOLO; the infrared image feature extraction module is used for generating overheat region features of a rim infrared image detected by the infrared image acquisition device through an overheat region feature extraction model based on Unet; the data fusion module is used for generating fusion features from the rim structure features, the overheating area features and the time-frequency domain vibration features through an Attention-based feature fusion model; and the state evaluation module is used for predicting the rim remaining life through a state judgment model based on an improved CNN according to the fusion features, and performing subway vehicle state management according to the rim remaining life. According to the invention, accurate evaluation of the state of the metro vehicle bogie rim is realized.

Description

Technical Field

[0001] This invention relates to the field of metro vehicle condition assessment technology, and in particular to a metro vehicle condition assessment system based on trackside data fusion. Background Technology

[0002] The rapid development of intelligent technology has brought new opportunities for the technological transformation of rail transit vehicles, including intelligent operation and maintenance technology for subway vehicles. From the perspective of technology application, the core change is to transform the planned maintenance model into a condition-based maintenance model, so as to realize the transformation from the previous extensive operation and maintenance of "casting a wide net" to the precise operation and maintenance of "focusing on key targets".

[0003] Among these, the condition detection of subway vehicles is of paramount importance. Currently, however, the reliability assessment of subway vehicles, especially the condition judgment of the critical bogie rims in contact with the track, lacks the integration of multiple trackside data for detection. Previously, trackside data fusion detection was a comprehensive cross-disciplinary problem involving vehicles, tracks, and civil engineering structures. Intelligent operation and maintenance technologies, with their algorithmic models, lay the foundation for deep integration, thus providing intelligent theoretical guidance for overcoming the technical bottlenecks of single-modal detection, such as weak feature capture and high error rates.

[0004] Currently, for example, patent document with application number 202410853351.0 discloses a method for guiding wheelset maintenance by fusion analysis of onboard and trackside data of subway trains. However, it does not adopt intelligent operation and maintenance technology, and therefore the structural features it inputs cannot dynamically match the actual detection situation. Furthermore, its fusion process cannot perform intelligent deep fusion judgment based on a large amount of prior data. Therefore, its detection accuracy is inevitably insufficient.

[0005] In summary, how to accurately assess the operational status of subway vehicles by combining intelligent operation and maintenance technologies is a technical problem that needs to be solved. Summary of the Invention

[0006] To this end, the present invention provides a subway vehicle condition assessment system based on trackside data fusion. It extracts data features from visual image acquisition devices, infrared image acquisition devices, and vibration source strength data acquisition devices through multiple recognition models / algorithms, and realizes the construction of judgment criteria features that integrate mechanical, thermodynamic, and dynamic dimensions. By utilizing the improved CNN model to capture the deep features of the data, it achieves accurate assessment of the condition of subway vehicle bogie rims by combining intelligent operation and maintenance technology.

[0007] To achieve the above objectives, this invention proposes a subway vehicle condition assessment system based on trackside data fusion. The management platform communicates with visual image acquisition devices, infrared image acquisition devices, and vibration source intensity data acquisition devices installed along the track. The management platform is equipped with:

[0008] The visual image feature extraction module is used to generate rim structural features from the rim visual image detected by the visual image acquisition device through a YOLO-based rim feature extraction model;

[0009] The infrared image feature extraction module is used to generate overheated area features from the infrared image of the wheel rim detected by the infrared image acquisition device through the overheated area feature extraction model based on Unet.

[0010] The vibration signal feature extraction module is used to extract time-frequency domain vibration features from the vibration source strength data detected by the vibration source strength data acquisition device through a vibration feature conversion algorithm;

[0011] The data fusion module is connected to the visual image feature extraction module, the infrared image feature extraction module, and the vibration signal feature extraction module, and is used to generate fused features from the wheel rim structure features, overheated area features, and time-frequency domain vibration features through an attention-based feature fusion model.

[0012] The state assessment module is used to predict the remaining life of the rim by using the fused features through a state judgment model based on an improved CNN, and to perform subway vehicle state management based on the remaining life of the rim.

[0013] Furthermore, the feature fusion model includes a channel attention unit, a spatial attention unit, and a multi-head attention unit;

[0014] The channel attention unit is used to enhance the rim structural features, overheated area features and time-frequency domain vibration features through multiple channels, including rim surface discoloration defects, rim surface contaminant defects, rim surface damage defects, infrared variation defects, bearing inner ring fault frequency band and rim out-of-roundness impact frequency band, to generate channel enhancement features.

[0015] The spatial attention unit is used to identify the associated region location of each channel based on the channel enhancement features and generate associated location features.

[0016] The multi-head attention unit is used to perform cross-modal fusion of the associated localization features to generate the fused features.

[0017] Furthermore, the multi-head attention unit includes a visual vibration enhancement subunit and a visual temperature enhancement subunit;

[0018] The visual vibration enhancement subunit is used to identify the time-frequency domain vibration characteristics of the area where the surface damage defect of the rim is located based on the channel enhancement features, and generate the corresponding fusion features.

[0019] The visual heating enhancement subunit is used to identify the time-frequency domain vibration characteristics of the region where the infrared variation defect is located by the channel enhancement feature, and generate the corresponding fusion feature, wherein the infrared variation defect includes long-wave infrared variation defect and mid-wave infrared variation defect.

[0020] Furthermore, the channel attention unit includes a global average pooling layer, a global max pooling layer, a feature operation layer, and a reweighting layer;

[0021] The global average pooling layer is used to extract the global average response of the rim structural features, overheated region features, and time-frequency domain vibration features in a single channel, and generate an overall feature distribution.

[0022] The global max pooling layer is used to extract the local salient features of the rim structure features, overheated region features, and time-frequency domain vibration features in a single channel, and generate local features.

[0023] The feature operation layer is used to concatenate the overall feature distribution and the local features, perform batch normalization, and then perform convolution activation operations to generate channel weights.

[0024] The reweighted layer is used to generate the channel enhancement features based on the channel weights, the rim structure features, the overheated region features, and the time-frequency domain vibration features.

[0025] In the above scheme, the attention-based feature fusion model achieves the generation of cross-modal feature correlation strength by learning fault coupling relationships through data-driven learning without prior knowledge through channel attention, spatial attention, and multi-head attention. This enables GPU acceleration suitable for subway train management platforms and meets the real-time requirements of side track data fusion.

[0026] Furthermore, the state evaluation module includes an improved LeNet-5 architecture and a multilayer perceptron architecture;

[0027] The improved LeNet-5 architecture includes a first convolutional pooling layer to a third convolutional pooling layer, which are used sequentially to extract the edge features, temporal-related features, and semantic-related features of the fused features;

[0028] The multilayer perceptron architecture is used to combine the fused features with the edge features, temporal correlation features and semantic correlation features to perform regression prediction and generate the remaining life of the wheel rim.

[0029] Furthermore, the management platform also includes a training strategy module;

[0030] The training strategy module is used to construct a contrastive loss term based on the attention weights of the multi-head attention unit, construct a joint loss term based on the generated value of the remaining life of the rim and the fusion features, and generate a total loss function by weighted summation of the contrastive loss term and the joint loss term. The total loss function is used to optimize the data fusion module and the state evaluation module.

[0031] Furthermore, the rim feature extraction model includes an intersection-union scaling unit, a loss term calculation unit, and a loss function adjustment unit;

[0032] The cross-union ratio scaling unit is used to calculate the scaling cross-union ratio based on the difference between the cross-union ratio and the set parameters;

[0033] The loss term calculation unit is used to calculate the loss term based on the difference between the cross-union ratio and the scaled cross-union ratio;

[0034] The loss function adjustment unit is used to calculate the acceleration loss function based on the loss term and the original loss function.

[0035] The above scheme achieves high-precision state assessment, strong robustness and industrial-grade efficiency through multi-level feature extraction, cross-modal attention fusion and joint loss optimization, thus enabling the overall model to be used for real-time inference on various edge devices and adapt to complex working conditions.

[0036] Furthermore, the vibration signal feature extraction module includes a resonance demodulation analysis unit and a wavelet packet energy entropy analysis unit;

[0037] The resonance demodulation analysis unit is used to separate the vibration source intensity data into the fault characteristic frequencies corresponding to the bearing inner ring fault frequency band.

[0038] The wavelet packet energy entropy analysis unit is used to separate the vibration source strength data into fault characteristic frequencies corresponding to the rim out-of-roundness impact frequency band.

[0039] Furthermore, the overheated region feature extraction model is a Unet architecture with a residual network, used to transmit the overheated region features.

[0040] Furthermore, the management platform includes an AI middleware platform, a big data platform built on Kafka, and a processing platform built on Flink.

[0041] The AI ​​platform is equipped with the visual image feature extraction module and the infrared image feature extraction module;

[0042] The processing platform is equipped with a data fusion module and a status assessment module;

[0043] The big data platform is equipped with a vibration signal feature extraction module, which is used to store wheel rim structural features, overheated area features, time-frequency domain vibration features, and predict the remaining life of the wheel rim, and to manage the remaining life of the wheel rim in the subway vehicle status.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1. By extracting data features from visual image acquisition devices, infrared image acquisition devices, and vibration source strength data acquisition devices through various recognition models / algorithms, a judgment basis feature integrating mechanical, thermodynamic, and dynamic dimensions was constructed. By utilizing the improved CNN model to capture the deep features of the data, the accurate assessment of the rim condition of the subway vehicle bogie was achieved.

[0046] 2. The attention-based feature fusion model achieves data-driven learning of fault coupling relationships and generation of cross-modal feature correlation strength without prior knowledge through channel attention, spatial attention, and multi-head attention. This enables GPU acceleration suitable for subway train management platforms and meets real-time requirements for side track data fusion.

[0047] 3. Through multi-level feature extraction, cross-modal attention fusion, and joint loss optimization, high-precision state assessment, strong robustness, and industrial-grade efficiency are achieved, enabling the overall model to be used for real-time inference on various edge devices and adapt to complex working conditions. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the structure of the subway vehicle condition assessment system based on trackside data fusion according to an embodiment of the present invention;

[0049] Figure 2 This is a flowchart illustrating the subway vehicle condition assessment system based on trackside data fusion, according to an embodiment of the present invention.

[0050] Figure 3 This is a schematic diagram of the model structure of a subway vehicle condition assessment system based on trackside data fusion, according to an embodiment of the present invention.

[0051] Figure 4 This is a schematic diagram of the management platform architecture of the subway vehicle condition assessment system based on trackside data fusion, according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0053] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0054] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0055] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0056] like Figures 1 to 4 As shown, this invention provides a subway vehicle condition assessment system based on trackside data fusion. It extracts data features from visual image acquisition devices, infrared image acquisition devices, and vibration source strength data acquisition devices through multiple recognition models / algorithms, and realizes the construction of judgment criteria features that integrate mechanical, thermodynamic, and dynamic dimensions. By utilizing an improved CNN model to capture the deep features of the data, it achieves accurate assessment of the condition of the subway vehicle bogie rim.

[0057] like Figures 1 to 4 As shown, this embodiment proposes a subway vehicle condition assessment system based on trackside data fusion. The management platform communicates with visual image acquisition devices, infrared image acquisition devices, and vibration source intensity data acquisition devices installed along the track. The management platform is equipped with:

[0058] The visual image feature extraction module is used to generate rim structural features from the rim visual image detected by the visual image acquisition device through a YOLO-based rim feature extraction model;

[0059] The infrared image feature extraction module is used to generate overheated area features from the infrared image of the wheel rim detected by the infrared image acquisition device through the overheated area feature extraction model based on Unet.

[0060] The vibration signal feature extraction module is used to extract time-frequency domain vibration features from the vibration source strength data detected by the vibration source strength data acquisition device through a vibration feature conversion algorithm;

[0061] The data fusion module is connected to the visual image feature extraction module, the infrared image feature extraction module, and the vibration signal feature extraction module, and is used to generate fused features from the wheel rim structure features, overheated area features, and time-frequency domain vibration features through an attention-based feature fusion model.

[0062] The state assessment module is used to predict the remaining life of the rim by using the fused features through a state judgment model based on an improved CNN, and to perform subway vehicle state management based on the remaining life of the rim.

[0063] It is understandable that, such as Figure 3 As shown, the YOLO-based rim feature extraction model can quickly and in real-time locate rim surface defects. The Unet-based overheating area feature extraction model can accurately segment small overheating areas, avoiding missed detections. The vibration feature conversion algorithm can capture dynamic anomalies, and its output features are converted into standard codes with positional codes through one-hot encoding. Then, after deep fusion through the attention-based feature fusion model, it is fed into the state judgment model based on the improved CNN to achieve cross-modal interactive modeling and lightweight dynamic enhancement of the remaining rim life. This achieves high-precision and high-robust industrial-grade prediction capabilities, providing an efficient and reliable technical path for multimodal industrial prediction tasks based on trackside data features, and promoting the evolution of the intelligent operation and maintenance system of subway trains towards lightweight, adaptive, and interpretable directions.

[0064] Furthermore, such as Figure 1 , 2 As shown in Figure 3, the feature fusion model includes a channel attention unit, a spatial attention unit, and a multi-head attention unit;

[0065] The channel attention unit is used to enhance the rim structural features, overheated area features and time-frequency domain vibration features through multiple channels, including rim surface discoloration defects, rim surface contaminant defects, rim surface damage defects, infrared variation defects, bearing inner ring fault frequency band and rim out-of-roundness impact frequency band, to generate channel enhancement features.

[0066] The spatial attention unit is used to identify the associated region location of each channel based on the channel enhancement features and generate associated location features.

[0067] The multi-head attention unit is used to perform cross-modal fusion of the associated localization features to generate the fused features.

[0068] It is understandable that, such as Figure 3 As shown, the rim structural features, overheated region features, and time-frequency domain vibration features each represent a modal data. The overall architecture of the feature fusion model is as follows: each modal data independently passes through a channel attention unit to enhance key feature channels; the enhanced features of each modal data are input into a spatial attention unit to locate key regions; the spatial attention outputs of all modal data are spliced ​​together and input into a multi-head attention mechanism for cross-modal interaction, learning the correlation between different modalities, and generating fused features.

[0069] Specifically, for the rim structural features, the channel attention unit enhances the green channel characterizing rim surface discoloration defects, the blue channel characterizing rim surface contaminant defects, and the edge gradient channel characterizing rim surface damage defects, while suppressing the red channel for reflective noise caused by changes in illumination. For the overheated area features, the channel attention unit enhances the high-temperature response channel and suppresses the low-temperature background channel. For the time-frequency domain vibration features, the channel attention unit enhances the bearing inner ring fault frequency band (140-170Hz) and the rim out-of-roundness impact frequency band (10-50Hz), while suppressing high-frequency noise (greater than 1kHz). It is understood that the green channel (G) is more sensitive to rim surface oxidation and discoloration (such as rust), potentially indicating material degradation, while the blue channel (B) is more effective at detecting contaminants such as oil stains and lubricant residues.

[0070] Specifically, the spatial attention unit is used to locate the positions of surface discoloration defects, rim surface contaminant defects, and rim surface breakage defects, generate corresponding spatial weight masks, and focus on the overheated area of ​​the bearing outer ring to generate corresponding spatial weight masks.

[0071] Furthermore, the multi-head attention unit includes a visual vibration enhancement subunit and a visual temperature enhancement subunit;

[0072] The visual vibration enhancement subunit is used to identify the time-frequency domain vibration characteristics of the area where the surface damage defect of the rim is located based on the channel enhancement features, and generate the corresponding fusion features.

[0073] The visual heating enhancement subunit is used to identify the time-frequency domain vibration characteristics of the region where the infrared variation defect is located by the channel enhancement feature, and generate the corresponding fusion feature, wherein the infrared variation defect includes long-wave infrared variation defect and mid-wave infrared variation defect.

[0074] Specifically, the visual vibration enhancement subunit calculates the correlation between the visual query of the rim structure features and the time-frequency domain vibration features of the vibration through a multi-head attention mechanism, deriving attention weights. These attention weights are then used to weight the associated localization features corresponding to the vibration, generating weighted features, which are then used as elements in the multi-head stitching process. The visual heating enhancement subunit operates similarly.

[0075] The visual vibration enhancement subunit can be represented by the following formula:

[0076]

[0077] F vsb =AttentionV vib

[0078] F = Concat(F) vis ,F vib ,F vsb …)W

[0079] In the formula, Attention represents the attention weight, Q vis Queries describing the structural features of wheel rims. The key represents the time-frequency domain vibration characteristics, d represents the query dimension of the rim structure characteristics used to reduce its dimensionality, and V vib F represents the value of the vibration characteristic in the time-frequency domain, and F represents the weighted characteristic. vis ,F vib ,F vsb These represent the rim structural features, time-frequency domain vibration features, and associated positioning features, respectively, while W represents the learnable parameter matrix.

[0080] Furthermore, the channel attention unit includes a global average pooling layer, a global max pooling layer, a feature operation layer, and a reweighting layer;

[0081] The global average pooling layer is used to extract the global average response of the rim structural features, overheated region features, and time-frequency domain vibration features in a single channel, and generate an overall feature distribution.

[0082] The global max pooling layer is used to extract the local salient features of the rim structure features, overheated region features, and time-frequency domain vibration features in a single channel, and generate local features.

[0083] The feature operation layer is used to concatenate the overall feature distribution and the local features, perform batch normalization, and then perform convolution activation operations to generate channel weights.

[0084] The reweighted layer is used to generate the channel enhancement features based on the channel weights, the rim structure features, the overheated region features, and the time-frequency domain vibration features.

[0085] Specifically, the global average pooling layer is used to extract the global average response of the channel, reflecting the overall feature distribution, such as the average wear depth and average temperature of the rim. The global max pooling layer is used to extract local salient features of the channel, such as high gradient regions at crack tips and local hot spots, thereby achieving complementary enhancement of local and global features.

[0086] In the above scheme, the feature fusion model based on Attention achieves the generation of cross-modal feature correlation strength by learning fault coupling relationships through data-driven learning without prior knowledge through channel attention, spatial attention and multi-head attention. This enables GPU acceleration suitable for subway train management platforms and meets the real-time requirements of side track data fusion. In actual tests, the accuracy of composite fault detection was increased from 68% to 92%.

[0087] like Figure 3 As shown, the state evaluation module further includes an improved LeNet-5 architecture and a multilayer perceptron (MLP) architecture;

[0088] The improved LeNet-5 architecture includes a first convolutional pooling layer to a third convolutional pooling layer, which are used sequentially to extract the edge features, temporal-related features, and semantic-related features of the fused features;

[0089] The multilayer perceptron architecture is used to combine the fused features with the edge features, temporal correlation features and semantic correlation features to perform regression prediction and generate the remaining life of the wheel rim.

[0090] Specifically, the parameter configurations of the first to third convolutional pooling layers are as follows: First convolutional pooling layer: Conv5×5, filters=6, Pool2×2, MaxPooling; Second convolutional pooling layer: Conv3×3, filters=16, dilation=2, Pool2×2; Third convolutional pooling layer: Conv3×3, filters=32, Pool2×2. The first convolutional pooling layer captures local edge gradients through convolutional kernels; the second convolutional pooling layer expands the receptive field through dilation (dilation=2) to capture temporal dynamics; and the third convolutional pooling layer integrates multi-frame temporal features to generate high-level semantics, such as the peeling of the inner ring of a bearing or severe wear of a wheel rim.

[0091] More specifically, a multilayer perceptron architecture is set after the third convolutional pooling layer. The hidden layer of the multilayer perceptron architecture is used to combine the fused features with the edge features, temporal related features and semantic related features. The output layer of the multilayer perceptron architecture preferably uses the Softplus loss function to generate the remaining life of the wheel rim.

[0092] Furthermore, the management platform also includes a training strategy module;

[0093] The training strategy module is used to construct a contrastive loss term based on the attention weights of the multi-head attention unit, construct a joint loss term based on the generated value of the remaining life of the rim and the fusion features, and generate a total loss function by weighted summation of the contrastive loss term and the joint loss term. The total loss function is used to optimize the data fusion module and the state evaluation module.

[0094] Specifically, the total loss function is:

[0095]

[0096] In the formula, Loss is the total loss function, and λ is the weighting coefficient.

[0097] These are the joint loss term and the comparative loss term, respectively, where F m F and F' are the rim remaining life generated by the state assessment module and the weighted feature output by the multi-head attention unit, respectively. i ,F j Let A represent the features of two different samples H, both belonging to the hard sample H, and let ε be the threshold parameter. The hard sample H is based on the same multimodal feature A of the wheel rim. i Features A of different states of wheel rim j The product of these factors is determined by comparing it with the attention-related threshold τ.

[0098] Furthermore, the rim feature extraction model includes an intersection-union scaling unit, a loss term calculation unit, and a loss function adjustment unit;

[0099] The cross-union ratio scaling unit is used to calculate the scaling cross-union ratio based on the difference between the cross-union ratio and the set parameters;

[0100] The loss term calculation unit is used to calculate the loss term based on the difference between the cross-union ratio and the scaled cross-union ratio;

[0101] The loss function adjustment unit is used to calculate the acceleration loss function based on the loss term and the original loss function.

[0102] Specifically, the process by which the cross-union ratio scaling unit calculates the scaling cross-union ratio is as follows:

[0103]

[0104] f iou =max(0,min(1,f′))

[0105] In the formula, f′, f iou These represent the prescaled cross-union ratio and the scaled cross-union ratio, respectively. d and u are the scaling parameters, and oiou is the cross-union ratio.

[0106] The process by which the loss calculation unit calculates the loss term is as follows:

[0107] f foc =mean(oiou-f iou )

[0108] In the formula, oiou is the intersection-union ratio, f iou It is the scaling ratio of intersection to union, f foc It is a loss item.

[0109] The acceleration loss function is:

[0110] L=L′+f foc

[0111] In the formula, L and L′ represent the acceleration loss function and the original loss function, respectively, and f foc It is a loss item.

[0112] The above scheme achieves high-precision state assessment, strong robustness and industrial-grade efficiency through multi-level feature extraction, cross-modal attention fusion and joint loss optimization, thus enabling the overall model to be used for real-time inference on various edge devices and adapt to complex working conditions.

[0113] Furthermore, the vibration signal feature extraction module includes a resonance demodulation analysis unit and a wavelet packet energy entropy analysis unit;

[0114] The resonance demodulation analysis unit is used to separate the vibration source intensity data into the fault characteristic frequencies corresponding to the bearing inner ring fault frequency band.

[0115] The wavelet packet energy entropy analysis unit is used to separate the vibration source strength data into fault characteristic frequencies corresponding to the rim out-of-roundness impact frequency band.

[0116] It is understandable that the bearing inner ring fault frequency band represents the high-frequency inherent resonance of the mechanical system, belonging to periodic impacts. This manifests as a high-frequency carrier signal being modulated by a low-frequency fault characteristic frequency. Therefore, by analyzing the resonance demodulation and extracting the envelope of the modulated signal, the fault characteristic frequency can be separated. The rim out-of-roundness impact frequency band can increase the non-stationarity of the signal, leading to an increase in energy entropy. Therefore, by analyzing the wavelet packet energy entropy and quantifying the complexity of the vibration signal, the fault characteristic frequency can be extracted.

[0117] Furthermore, the overheated region feature extraction model is a Unet architecture with a residual network, used to transmit the overheated region features.

[0118] Understandably, Residual Networks (ResNet) were primarily proposed to address the degradation problem in deep neural networks. Belonging to deep convolutional neural networks, ResNet addresses the issue that in traditional networks, gradients tend to shrink during backpropagation as the number of layers increases, leading to training difficulties. By introducing short connections between the outputs of each convolutional layer in the ResNet architecture, the input and output are summed, making it easier for the network to learn residual information. This effectively solves the problems of vanishing and exploding gradients during network training.

[0119] Furthermore, such as Figure 4 As shown, the management platform includes an AI middleware platform, a big data platform built on Kafka, and a processing platform built on Flink.

[0120] The AI ​​platform is equipped with the visual image feature extraction module and the infrared image feature extraction module;

[0121] The processing platform is equipped with a data fusion module and a status assessment module;

[0122] The big data platform is equipped with a vibration signal feature extraction module, which is used to store wheel rim structural features, overheated area features, time-frequency domain vibration features, and predict the remaining life of the wheel rim, and to manage the remaining life of the wheel rim in the subway vehicle status.

[0123] Specifically, fault data is accessed via Kafka: the trackside sensing system sends fault diagnosis data to Kafka, and the vehicle intelligent analysis system consumes the fault diagnosis data from Kafka and writes it to IoTDB for persistent storage. The data format is uniformly defined by the subway company's big data platform. Through interface integration and unified permissions, the trackside sensing system is integrated into the platform.

[0124] like Figure 4As shown, the entire data flow is divided into three paths: visual data and infrared data are directly sent to the image storage of the AI ​​platform via the upload interface (confirmed by the base group). Visual and infrared features push diagnostic results to Kafka, and then the data is transposed using a data mapping table and stored in the big data platform IoTDB. The wireless transmission device sends vibration data to the Netty server in real time, removes the packet header, puts the data into Kafka, and then uses the streaming engine Flink to parse, transpose, and compute the data. The transposed data is placed in IoTDB, and the computed model results are placed in Kafka. The work order system subscribes to data acquisition and then sends the work order data back as historical data.

[0125] In this embodiment, multiple recognition models / algorithms are used to extract data features from visual image acquisition devices, infrared image acquisition devices, and vibration source strength data acquisition devices. This enables the construction of judgment criteria features that integrate mechanical, thermodynamic, and dynamic dimensions. By utilizing an improved CNN model to capture deep features of the data, accurate assessment of the rim condition of the metro vehicle bogie is achieved. An attention-based feature fusion model, through channel attention, spatial attention, and multi-head attention, achieves fault coupling relationship learning without prior knowledge and generates cross-modal feature correlation strength through data-driven learning. This enables GPU acceleration suitable for metro train management platforms, meeting real-time requirements for side-track data fusion. Through multi-level feature extraction, cross-modal attention fusion, and joint loss optimization, high-precision state assessment, strong robustness, and industrial-grade efficiency are achieved. This allows the overall model to be used for real-time inference on various edge devices, adapting to complex operating conditions.

[0126] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0127] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A subway vehicle condition assessment system based on trackside data fusion, characterized in that, The management platform communicates with visual image acquisition devices, infrared image acquisition devices, and vibration source strength data acquisition devices installed beside the track. The management platform is equipped with: The visual image feature extraction module is used to generate rim structural features from the rim visual image detected by the visual image acquisition device through a YOLO-based rim feature extraction model; The infrared image feature extraction module is used to generate overheated area features from the infrared image of the wheel rim detected by the infrared image acquisition device through the overheated area feature extraction model based on Unet. The vibration signal feature extraction module is used to extract time-frequency domain vibration features from the vibration source strength data detected by the vibration source strength data acquisition device through a vibration feature conversion algorithm; The data fusion module is connected to the visual image feature extraction module, the infrared image feature extraction module, and the vibration signal feature extraction module, and is used to generate fused features from the wheel rim structure features, overheated area features, and time-frequency domain vibration features through an attention-based feature fusion model. The status assessment module is used to predict the remaining life of the rim by using the fused features through a status judgment model based on an improved CNN, and to manage the status of the subway vehicle based on the remaining life of the rim.

2. The subway vehicle condition assessment system based on trackside data fusion according to claim 1, characterized in that, The feature fusion model includes a channel attention unit, a spatial attention unit, and a multi-head attention unit; The channel attention unit is used to enhance the rim structural features, overheated area features and time-frequency domain vibration features through multiple channels, including rim surface discoloration defects, rim surface contaminant defects, rim surface damage defects, infrared variation defects, bearing inner ring fault frequency band and rim out-of-roundness impact frequency band, to generate channel enhancement features. The spatial attention unit is used to identify the associated region location of each channel based on the channel enhancement features and generate associated location features. The multi-head attention unit is used to perform cross-modal fusion of the associated localization features to generate the fused features.

3. The subway vehicle condition assessment system based on trackside data fusion according to claim 2, characterized in that, The multi-head attention unit includes a visual vibration enhancement subunit and a visual temperature enhancement subunit; The visual vibration enhancement subunit is used to identify the time-frequency domain vibration characteristics of the area where the surface damage defect of the rim is located based on the channel enhancement features, and generate the corresponding fusion features. The visual heating enhancement subunit is used to identify the time-frequency domain vibration characteristics of the region where the infrared variation defect is located by the channel enhancement feature, and generate the corresponding fusion feature, wherein the infrared variation defect includes long-wave infrared variation defect and mid-wave infrared variation defect.

4. The subway vehicle condition assessment system based on trackside data fusion according to claim 2, characterized in that, The channel attention unit includes a global average pooling layer, a global max pooling layer, a feature operation layer, and a reweighted layer; The global average pooling layer is used to extract the global average response of the rim structural features, overheated region features, and time-frequency domain vibration features in a single channel, and generate an overall feature distribution. The global max pooling layer is used to extract the local salient features of the rim structure features, overheated region features, and time-frequency domain vibration features in a single channel, and generate local features. The feature operation layer is used to concatenate the overall feature distribution and the local features, perform batch normalization, and then perform convolution activation operations to generate channel weights. The reweighted layer is used to generate the channel enhancement features based on the channel weights, the rim structure features, the overheated region features, and the time-frequency domain vibration features.

5. The subway vehicle condition assessment system based on trackside data fusion according to claim 1, characterized in that, The state evaluation module includes an improved LeNet-5 architecture and a multilayer perceptron architecture; The improved LeNet-5 architecture includes a first convolutional pooling layer to a third convolutional pooling layer, which are used sequentially to extract the edge features, temporal-related features, and semantic-related features of the fused features; The multilayer perceptron architecture is used to combine the fused features with the edge features, temporal correlation features and semantic correlation features to perform regression prediction and generate the remaining life of the wheel rim.

6. The subway vehicle condition assessment system based on trackside data fusion according to claim 2, characterized in that, The management platform also includes a training strategy module; The training strategy module is used to construct a contrastive loss term based on the attention weights of the multi-head attention unit, construct a joint loss term based on the generated value of the remaining life of the rim and the fusion features, and generate a total loss function by weighted summation of the contrastive loss term and the joint loss term. The total loss function is used to optimize the data fusion module and the state evaluation module.

7. The subway vehicle condition assessment system based on trackside data fusion according to claim 1, characterized in that, The wheel rim feature extraction model includes an intersection-union scaling unit, a loss term calculation unit, and a loss function adjustment unit; The cross-union ratio scaling unit is used to calculate the scaling cross-union ratio based on the difference between the cross-union ratio and the set parameters; The loss term calculation unit is used to calculate the loss term based on the difference between the cross-union ratio and the scaled cross-union ratio; The loss function adjustment unit is used to calculate the acceleration loss function based on the loss term and the original loss function.

8. The subway vehicle condition assessment system based on trackside data fusion according to claim 1, characterized in that, The vibration signal feature extraction module includes a resonance demodulation analysis unit and a wavelet packet energy entropy analysis unit. The resonance demodulation analysis unit is used to separate the vibration source intensity data into the fault characteristic frequencies corresponding to the bearing inner ring fault frequency band. The wavelet packet energy entropy analysis unit is used to separate the vibration source strength data into fault characteristic frequencies corresponding to the rim out-of-roundness impact frequency band.

9. The subway vehicle condition assessment system based on trackside data fusion according to claim 1, characterized in that, The overheated region feature extraction model is a Unet architecture with a residual network, used to transmit the overheated region features.

10. The subway vehicle condition assessment system based on trackside data fusion according to any one of claims 1 to 9, characterized in that, The management platform includes an AI middleware platform, a big data platform built on Kafka, and a processing platform built on Flink. The AI ​​platform is equipped with the visual image feature extraction module and the infrared image feature extraction module; The processing platform is equipped with a data fusion module and a status assessment module; The big data platform is equipped with a vibration signal feature extraction module, which is used to store wheel rim structural features, overheated area features, time-frequency domain vibration features, and predict the remaining life of the wheel rim, and to manage the remaining life of the wheel rim in the subway vehicle status.

Images