Railway infrastructure health state prediction method, device and platform

Abstract

The invention discloses a railway infrastructure health state prediction method, device and platform. The method comprises the following steps: collecting multi-mode sensor data reflecting the state of the railway infrastructure; the data of the multi-mode sensor are preprocessed; constructing and training a multi-modal data fusion deep learning model; and inputting preprocessed data collected in real time into the trained multi-modal data fusion deep learning model to obtain a health state prediction result of the railway infrastructure, and comparing the health state prediction result with a preset health threshold. According to the technical scheme, data standardization is achieved by collecting the data of the multi-modal sensor and adopting the preprocessing technology; a multi-task deep neural network self-organizing mapping and a graph neural network are used for fusing modal features, and a multi-task long-short-term memory network and a Bayesian uncertainty quantification module are used for assisting, so that the railway health state is accurately predicted.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation, and particularly to a method, device, and platform for predicting the health status of railway infrastructure. Background Art

[0002] With the continuous increase in railway operating mileage, the continuous improvement of network density, and the continuous increase in train speed and load level, the potential safety risks during operation have become increasingly prominent. Therefore, the issue of railway safe operation has received extensive attention. In this context, as an important link to ensure train operation safety, the stable and efficient operation of railway infrastructure is crucial. The traditional manual inspection method relies on the observation and experience of on-site staff, which is easily interfered by subjective factors, resulting in inspection lags, misjudgments, or missed judgments, and even leading to over-maintenance or under-maintenance phenomena, bringing greater challenges to railway maintenance departments and making it difficult to maximize the benefits of railway full-life cycle management.

[0003] The operation and maintenance of railway infrastructure mainly rely on manual inspections and regular maintenance. Manual inspection refers to trained inspection personnel who use visual observation or simple detection equipment to check the track and related components. However, this method has limitations such as strong subjectivity, low detection efficiency, poor night operation conditions, limited inspection frequency, being restricted by the comprehensive maintenance time window, narrow inspection scope, and insufficient automated analysis capabilities. In addition, in the alpine regions of the west, railway lines pass through a large number of uninhabited areas with harsh environments, which not only increases the inspection risks but also threatens the safety of personnel. Regular maintenance has a long cycle and limited coverage, making it difficult to detect potential hidden dangers in a timely manner. Once a fault occurs, due to the lack of real-time monitoring and precise positioning means, it is difficult to locate and repair the fault in a timely manner, seriously affecting the train operation safety and punctuality rate. How to properly solve the above problems has become an urgent issue in the industry. Summary of the Invention

[0004] The present invention provides a method, device, and platform for predicting the health status of railway infrastructure, which uses multi-modal deep learning technology to monitor the health status of railway facilities in real time, accurately warn of fault risks, optimize maintenance strategies, and ensure train operation safety and operational benefits.

[0005] According to the first aspect of the present invention, there is provided a method for predicting the health status of railway infrastructure, the method for predicting the health status of railway infrastructure including: Collecting multi-modal sensor data reflecting the status of railway infrastructure, the multi-modal sensor data including any one or more of vibration signals, image data, and train operation parameters, and adjusting the sampling frequency of each sensor according to a real-time risk assessment using an adaptive dynamic sampling strategy; Preprocess the multi-modal sensor data, where the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation for missing data, feature extraction, and normalization, and the feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; Construct and train a multi-modal data fusion deep learning model. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which incorporates a Bayesian uncertainty quantification module to obtain the health state prediction result and the confidence interval; Input the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model to obtain the health state prediction result of the railway infrastructure, and compare the health state prediction result with a preset health threshold to generate a warning message; Collect maintenance feedback data, and perform online adaptive update on the multi-modal data fusion deep learning model through a closed-loop feedback mechanism using reinforcement learning or meta-learning methods. At the same time, use the attention mechanism to construct an interpretable artificial intelligence module to perform root cause analysis of the prediction result, and output any one or more of the health state level, confidence interval, warning message, and maintenance suggestion.

[0006] In one embodiment, the collection of multi-modal sensor data reflecting the state of railway infrastructure includes: Collect track vibration signal data through vibration sensors installed on the railway track structure; Collect image data of railway infrastructure through monitoring cameras; Obtain train operation parameter data from the train operation control system; Dynamically adjust the sampling frequency of each sensor according to the risk index calculated by the real-time risk assessment model, and synchronize each data according to a unified time reference.

[0007] In one embodiment, the preprocessing of the multi-modal sensor data includes: Perform noise filtering, outlier removal, and interpolation processing for missing data on the multi-modal sensor data; Align data from different sources based on timestamps, and extract features including time-domain features, frequency-domain features, and spatial topology features; Perform normalization processing on the extracted feature parameters to form standardized data for subsequent model training.

[0008] In one embodiment, the construction and training of the multi-modal data fusion deep learning model includes: Construct a multi-modal data fusion deep learning model that integrates a multi-task deep neural network self-organizing mapping module and a graph neural network module, where each data modality is processed by an independent feature extraction sub-network, and the spatial topological association between railway monitoring points is established through the graph neural network; Embed a Bayesian uncertainty quantification module in a multi-task long short-term memory network (MTL-LSTM), and use historical multi-modal data and corresponding health status labels to train the model; Evaluate the prediction accuracy of the model on the validation dataset, and adjust the model parameters or structure according to the evaluation results until the model converges to improve the prediction accuracy and model adaptability.

[0009] In one embodiment, the output of any one or more of the health status level, confidence interval, warning information, and maintenance suggestions includes: Input the real-time collected and preprocessed multi-modal data into the trained multi-modal data fusion deep learning model to obtain the health status prediction result and confidence interval of the railway infrastructure; Compare the prediction result with a preset health threshold to determine the health status level of the railway infrastructure; When the prediction result exceeds the normal range, generate warning information and maintenance suggestions, and use the attention mechanism to quantify the relevance between the multi-modal data and the prediction result, and output a root cause analysis report of the fault; Output any one or more of the health status level, confidence interval, warning information, and maintenance suggestions to provide a basis for maintenance decision-making.

[0010] In one embodiment, it further includes: Based on any one or more of the output health status level, confidence interval, warning information, and maintenance suggestions, conduct targeted inspections or maintenance on the railway infrastructure, and collect the actual status data after inspection or maintenance as feedback data; Compare the feedback data with the corresponding prediction result to evaluate the accuracy of the model prediction; Use the feedback data to perform online adaptive update on the multi-modal data fusion deep learning model through a closed-loop feedback mechanism using reinforcement learning or meta-learning algorithms; Dynamically adjust the health threshold, sensor sampling strategy, and maintenance plan according to the results of the updated multi-modal data fusion deep learning model, so as to further optimize the health status prediction of the railway infrastructure.

[0011] According to the second aspect of the present invention, a railway infrastructure health status prediction device is provided, including: The acquisition module is used to acquire multi-modal sensor data reflecting the status of railway infrastructure. The multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and adjusts the sampling frequencies of each sensor according to real-time risk assessment using an adaptive dynamic sampling strategy; The preprocessing module is used to preprocess the multi-modal sensor data. The preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation for missing values, feature extraction, and normalization. The feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; The training module is used to construct and train a multi-modal data fusion deep learning model. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which embeds a Bayesian uncertainty quantification module to obtain the health status prediction result and the confidence interval; The prediction module is used to input the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model, obtain the health status prediction result of the railway infrastructure, and compare the health status prediction result with a preset health threshold to generate a warning message; The adaptive module is used to collect maintenance feedback data, and perform online adaptive update on the multi-modal data fusion deep learning model by using a reinforcement learning or meta-learning method through a closed-loop feedback mechanism. At the same time, an interpretable artificial intelligence module is constructed by using an attention mechanism to perform root cause analysis of the prediction result, and output any one or more of the health status level, confidence interval, warning message, and maintenance suggestion.

[0012] In one embodiment, the device is deployed on a railway infrastructure health status prediction platform, and the railway infrastructure health status prediction platform includes any one or more of a platform large screen, a platform home page, dynamic monitoring, warning analysis, fault diagnosis, health assessment, fault prediction, maintenance decision-making, comprehensive analysis, equipment management, basic information, configuration management, and system management.

[0013] According to the third aspect of the present invention, an electronic device is provided. The electronic device includes: a communication interface, a processor, and a memory; Wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor communicatively connected to the memory through the communication interface, the above-mentioned any railway infrastructure health status prediction method is implemented.

[0014] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a computer (e.g., a processor in the computer), implement any one of the above-described railway infrastructure health state prediction methods.

[0015] In summary, the present invention provides a method, apparatus, and platform for predicting the health state of railway infrastructure. The method includes: collecting multi-modal sensor data reflecting the state of railway infrastructure, where the multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and adjusting the sampling frequency of each sensor according to a real-time risk assessment using an adaptive dynamic sampling strategy; preprocessing the multi-modal sensor data, where the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation and filling, feature extraction, and normalization, and the feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; constructing and training a multi-modal data fusion deep learning model, where the multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which is embedded with a Bayesian uncertainty quantification module to obtain a health state prediction result and a confidence interval; inputting the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model to obtain a health state prediction result of the railway infrastructure, and comparing the health state prediction result with a preset health threshold to generate a warning message; collecting maintenance feedback data, and online adaptively updating the multi-modal data fusion deep learning model by using a reinforcement learning or meta-learning method through a closed-loop feedback mechanism, and at the same time using an attention mechanism to construct an interpretable artificial intelligence module to perform a root cause analysis of the prediction result, and output any one or more of a health state level, a confidence interval, a warning message, and a maintenance suggestion.

[0016] The technical solution of this application collects multi-modal sensor data such as track vibration, images, and train operation, and realizes data standardization by using adaptive dynamic sampling, noise filtering, time series alignment, and feature extraction preprocessing techniques; uses a multi-task deep neural network self-organizing map and a graph neural network to fuse the features of each modality, supplemented by a multi-task long short-term memory network and a Bayesian uncertainty quantification module to accurately predict the health state of the railway; and assists decision-making through closed-loop feedback, real-time warning, and root cause analysis of faults, significantly reducing the accident risk.

[0017] Other features and advantages of the present invention will be set forth in the following description, and in part will be obvious from the description, or may be learned by practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the structure particularly pointed out in the written description and drawings.

[0018] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 A flowchart of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 2 A flowchart of step S11 of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 3 A flowchart of step S12 of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 4 A flowchart of step S13 of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 5 A flowchart of step S15 of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 6 A flowchart of another method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 7 A schematic diagram of the integrated intelligent monitoring platform system structure of a method for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 8 A structural diagram of a device for predicting the health status of railway infrastructure provided for an embodiment of the present invention; Figure 9 A structural diagram of an electronic device provided for an embodiment of the present invention. Detailed Embodiments

[0021] The features and exemplary embodiments of various aspects of the present application will be described in detail below. To make the objectives, technical solutions, and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0022] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0023] As Figure 1 shown, the present invention provides a method for predicting the health status of railway infrastructure, and the method for predicting the health status of railway infrastructure includes S11 - S15: In step S11, multi-modal sensor data reflecting the status of railway infrastructure is collected. The multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and the sampling frequency of each sensor is adjusted according to real-time risk assessment using an adaptive dynamic sampling strategy; In step S12, the multi-modal sensor data is preprocessed. The preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation for missing values, feature extraction, and normalization. The feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; In step S13, a multi-modal data fusion deep learning model is constructed and trained. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which is embedded with a Bayesian uncertainty quantification module to obtain the health status prediction result and the confidence interval; In step S14, the preprocessed data collected in real time is input into the trained multi-modal data fusion deep learning model to obtain the prediction result of the health state of the railway infrastructure, and the prediction result of the health state is compared with a preset health threshold to generate a warning message; In step S15, maintenance feedback data is collected, and the multi-modal data fusion deep learning model is updated online adaptively by using reinforcement learning or meta-learning method through a closed-loop feedback mechanism. At the same time, an interpretive artificial intelligence module is constructed by using an attention mechanism to perform root cause analysis of the prediction result, and any one or more of the health state level, confidence interval, warning message and maintenance suggestion are output.

[0024] In one embodiment, through links such as multi-modal sensor data acquisition, preprocessing, feature fusion, health state prediction, warning generation and closed-loop feedback update, the real-time monitoring of the railway infrastructure state and the prediction of fault risks are realized. The entire technical solution adopts a feature extraction module combining multi-task deep neural network self-organizing mapping (MTDNN-SOM) and graph neural network (GNN), and a multi-task long short-term memory network (MTL-LSTM) embedded with a Bayesian uncertainty quantification module, which can not only fully extract and fuse multi-modal data features, but also perform uncertainty assessment on the prediction result, and finally output the health state level, confidence interval, warning message and maintenance suggestion. Through the closed-loop feedback mechanism and with the help of reinforcement learning or meta-learning method to update the model parameters online, it is ensured that the system has good adaptability and can continuously improve the prediction accuracy.

[0025] The track vibration signal, image data and train operation parameter data are collected respectively through the vibration sensors, monitoring camera devices and train operation control systems installed on the railway track structure. To improve the real-time performance and effectiveness of the collected data, the system dynamically adjusts the sampling frequency of each sensor according to the risk index calculated by the real-time risk assessment model, so as to ensure the timeliness and representativeness of the data.

[0026] The multi-modal sensor data collected need to be preprocessed before entering the deep learning model. The preprocessing process includes noise filtering, outlier removal, time alignment and interpolation filling, and feature extraction and normalization. Noise filtering refers to using filtering algorithms to suppress high-frequency noise in the original data to ensure the accuracy of the subsequent processed data. Outlier removal refers to using statistical analysis or machine learning methods to identify and remove abnormal data to avoid interference with model training. Time alignment and interpolation filling are to align data from different sensors based on a unified timestamp, and use interpolation algorithms to fill in the missing data. Feature extraction and normalization refer to extracting time-domain, frequency-domain, and spatial topology features from the preprocessed data and performing normalization processing to form standardized data, providing high-quality input for subsequent deep learning model training.

[0027] Two-level deep learning modules are adopted to achieve data fusion and health state prediction. The first module is a combination of multi-task DNN-SOM and graph neural network, mainly responsible for feature extraction and fusion of the preprocessed multi-modal data. Specifically, a multi-task deep neural network self-organizing map (DNN-SOM) is used to extract local features of each data modality; a graph neural network is used to establish spatial topology associations between railway monitoring points, effectively integrating the spatial relationships of each sensor data to form a global feature description. The feature vector output by the first module is used as the input of the second module. The second module is a multi-task long short-term memory network (MTL-LSTM), which performs health state prediction based on the feature vector provided by the first module. The multi-task LSTM embedded with a Bayesian uncertainty quantification module can simultaneously output the health state prediction value and the corresponding confidence interval during the prediction process, ensuring that the prediction results have high credibility and interpretability.

[0028] The real-time collected and preprocessed data are predicted through the above-trained deep learning model to obtain the health state prediction results of the railway infrastructure. The health state prediction results are compared with the preset health thresholds. When the predicted value exceeds the normal range, warning information is automatically generated, and the relevance between the multi-modal data and the prediction results is quantified by combining the attention mechanism, and a root cause analysis report of the fault is output, and maintenance suggestions are given at the same time, providing a basis for railway maintenance decision-making.

[0029] In order to continuously improve the prediction accuracy and adaptability of the system, a closed-loop feedback mechanism is introduced. Specifically, after the prediction is completed, the feedback data after actual on-site maintenance or inspection are collected and compared with the prediction results, and the deep learning model is updated online adaptively through reinforcement learning or meta-learning algorithms. The updated model can dynamically adjust the health thresholds, sensor sampling strategies, and maintenance plans according to the latest data, realizing the health management of the entire life cycle of the railway infrastructure.

[0030] Based on the risk indicators calculated by the real-time risk assessment model for the complex railway operation environment, the sampling frequencies of each sensor are dynamically adjusted. For example, in high-risk areas or under adverse weather conditions, the system can automatically increase the sampling frequency of vibration sensors to ensure that sudden abnormal signals can be captured; while in low-risk areas, the sampling frequency can be appropriately reduced to save storage resources and computational burden. The preprocessing not only includes basic noise filtering and outlier removal, but also ensures data integrity through time alignment and interpolation to fill in missing values. The normalization process ensures the consistency of different modality data in the numerical scale, which helps to improve the convergence speed and prediction accuracy of the deep learning model. At the same time, the multi-angle extraction of time-domain, frequency-domain, and spatial topology features can comprehensively reflect all dimensions of the state of railway infrastructure. The first module uses a multi-task DNN-SOM combined with a graph neural network, which can capture the local information of the data and the spatial correlation between monitoring points during the feature extraction stage, so as to generate a globally consistent feature representation. The second module is based on a long short-term memory network, which has the advantage of capturing temporal information. At the same time, by embedding a Bayesian uncertainty quantification module, the uncertainty of the prediction results is quantified, providing a credibility reference for subsequent early warning decisions. The introduction of a closed-loop feedback mechanism realizes the closed-loop management of prediction, maintenance, feedback, and model update. The feedback data after on-site maintenance is compared with the model prediction results, which can not only evaluate the model performance, but also provide training samples for online model update. The adaptive update mechanism ensures that the system can respond in a timely manner to environmental changes and equipment state changes, improving the real-time performance and accuracy of railway facility health management. Taking a certain high-speed railway section as an example, based on the installation of vibration sensors, monitoring cameras, and train operation parameter collection devices, multi-modal data is automatically collected every day. The preprocessing module first filters out noise, removes abnormal data, and aligns the time of the data, and then performs feature extraction and normalization. The processed data is input into the deep learning model. The first module extracts the features of each monitoring point through DNN-SOM and the graph neural network to generate a global feature vector; the second module predicts the health status through multi-task LSTM and outputs the predicted value and the corresponding confidence interval. When the prediction result exceeds the preset threshold, an early warning message is automatically generated, and the main reasons affecting the health status are analyzed through the attention mechanism. After receiving the early warning, the maintenance personnel quickly conduct on-site inspections on the railway facilities in this section and feedback the actual inspection results to the system. The model is updated online according to the feedback data through reinforcement learning, making the next prediction more accurate. This ensures that abnormal situations of railway facilities can be quickly captured and processed, greatly improving the safety and maintenance efficiency of railway operation.

[0031] Through the monitoring data of railway facilities, accurate prediction of the health status is realized, potential fault hazards are discovered in advance, and a scientific basis is provided for preventive maintenance, thereby significantly improving the safety and reliability of railway operation.

[0032] A feature engineering method for the Multi-Task Deep Neural Networks Self-Organizing Map (MTDNN-SOM) is proposed. Aiming at the multi-modal characteristics of railway infrastructure monitoring data, various features are adaptively fused through the self-organizing mapping mechanism, thereby improving the accuracy and real-time performance of fault diagnosis and early warning and state time series prediction.

[0033] For the operating condition parameters of high-speed railways, in-depth analysis and identification are carried out, and the K-means algorithm is used to classify and identify the operating conditions. On this basis, a prediction model based on operating condition identification and multi-task deep learning - the Multi-Task Learning Long Short-Term Memory (MTL-LSTM) is constructed. Through the multi-task learning mechanism, the MTL-LSTM simultaneously trains multiple related tasks, fully excavates the state evolution laws contained in historical monitoring data, and comprehensively evaluates the health status of high-speed railway infrastructure according to the difference between the predicted value and the actual value of the MTL-LSTM model based on the statistical quality control theory.

[0034] Domestically, it is the first to propose the application of advanced technologies such as artificial intelligence, big data, and the Internet of Things to break the boundaries of on-site maintenance in traditional specialties such as engineering, electricity, and power supply, and achieve the deep integration of cross-system data. Through big data analysis and processing means, the correlation of the health status of cross-professional infrastructure in space and time is deeply studied, the decoupling of faults and risk factors in multi-professional coupled data is realized, and the root cause analysis of fault and risk characterization is completed. It can realize the joint processing of cross-professional facility and equipment data and the linkage analysis of health status, providing comprehensive and intelligent decision-making support for the comprehensive maintenance of railways and other infrastructure.

[0035] This embodiment not only realizes the accurate prediction of the health status of railway infrastructure, but also constructs an efficient and intelligent data fusion and analysis platform. Through adaptive feature engineering, multi-task learning, and cross-system data fusion, it not only improves the effect of fault diagnosis and early warning, but also can dynamically adjust monitoring and maintenance strategies, thus realizing the intelligent management of preventive maintenance of railway infrastructure. A unified standardized data interface and a consistent naming system are adopted between modules to ensure the consistency and efficient cooperation of internal system and cross-professional data processing, and it has good engineering application prospects and promotion value.

[0036] The operation data of railway infrastructure is collected in real time through special sensor equipment, and an edge computing controller is equipped to perform preliminary processing on the collected data, and then the processed data is transmitted back to the central monitoring system through the transmission network. The equipment fault data and operation status data are integrated, and according to the characteristics of the operation status and fault sample data, the fault features extracted by the multi-task deep neural network are combined with the self-organizing feature mapping (SOM) method to construct a multi-task deep learning model for equipment fault diagnosis based on deep neural networks.

[0037] Specifically, this embodiment uses a multi-task deep neural network self-organizing map model (Multi-task DeepNeural Networks Self-Organizing Map, MTDNN-SOM) to perform fault diagnosis and prediction on the status data and maintenance data received in real time, realize status-based equipment maintenance, and achieve adaptive optimization of the model. Through this model, potential fault hazards can be quickly identified, providing a scientific basis for preventive maintenance, thereby improving the operational safety and reliability of railway infrastructure. In addition, multi-task learning (Multi-Task Learning, MTL) effectively reduces the excessive correlation between model training data and the main task. At the same time, according to the training characteristics of each auxiliary task, it fully explores and utilizes the potential features implicit in the data to improve the data processing and feature extraction capabilities of the overall model. After multi-task joint training, the model can better realize knowledge sharing between tasks, thereby improving the accuracy of fault diagnosis and the generalization ability of prediction. The multi-task joint training process is as follows: Assuming that the total number of tasks is T, the training data of the tth task is recorded as ( ),in , , N is the total number of training samples. , are the feature vector and annotation label of the i-th sample respectively. Then the multi-task objective function can be expressed as: (1) in, is the input feature vector and weight parameters The mapping function, L() is the loss function shown in formula (1), is the regularization value of the weight parameter, is the regularization coefficient factor.

[0038] Self-Organizing Feature Mapping (SOM) is an unsupervised learning algorithm. The main purpose of SOM is to transform the input signal of any dimension into a one-dimensional or two-dimensional discrete mapping through computational mapping, and achieve this process in an adaptive manner. The steps of the SOM learning algorithm are as follows: (1) Initialization of the SOM network Initialize the connection weights between the input layer and the mapping layer. Due to the self-learning feature of the algorithm, in principle, it can be initialized as random numbers, but considering the computational efficiency of the algorithm, it is usually set to smaller weights.

[0039] (2) Vector input of the SOM neural network Assign the input vector to each input neuron according to the corresponding dimension.

[0040] (3) Determine the generalized distance between the input vector and the weight vector The calculation of the distance adopts the representation form of Euclidean distance. Taking the calculation of the j-th neuron in the mapping layer as an example, the distance between it and the input vector is calculated as follows: (2) In the above formula (2), is the weight between the i-th neuron in the input layer and the j-th neuron in the mapping layer. After the above calculation, by comparing the distance values, select the neuron with the smallest distance in the mapping layer, which is called the winning neuron, denoted as j. According to j, determine a certain processing unit k, which satisfies for any j , and accordingly determine the corresponding set of adjacent neurons.

[0041] (4) Weight learning Adjust the weights of the winning neuron j and its adjacent neurons determined in the following formula (3): (3) In the above formula (3), is a constant, 0 < < 1, and it decays to 0 over time . Analyze and identify the operating condition parameters of high-speed railways, and apply the K-means algorithm to classify and identify the operating conditions of high-speed railways; then, based on condition identification and multi-task deep learning, construct a multi-task long short-term memory neural network (LSTM) infrastructure prediction model. Finally, according to the theory of statistical quality control, use the difference between the predicted value and the true value of the MTL-LSTM model to evaluate the health status of high-speed railway infrastructure. Multi-task learning improves the ability of machine learning by training multiple related tasks simultaneously, and constructs a prediction model based on condition identification and multi-task learning (Multi-TaskLearning-Long Short-Term Memory, abbreviated as MTL-LSTM).

[0042] 1. Data acquisition Monitoring point and sensor selection: Combine the types of railway infrastructure (such as tracks, bridges, tunnels, etc.) and key stress-bearing parts to reasonably arrange sensors. For example, install strain sensors at track fasteners to monitor the stress of fasteners; arrange displacement and stress sensors at the mid-span and bearings of bridges. Select corresponding types of sensors according to different monitoring requirements, such as using acceleration sensors to monitor track vibration and thermocouple sensors to monitor temperature changes, etc.

[0043] Start data acquisition: Start the sensors according to the preset sampling frequency to obtain various types of operating data in real time. For example, during train operation, the track vibration sensor can be set to a sampling frequency of 500Hz to record dynamic information such as track vibration acceleration and displacement; the ambient temperature and humidity sensor can collect surrounding environment data every 10 minutes to monitor the operating environment of the infrastructure.

[0044] 2. Data transmission and storage Data transmission: The real-time data collected by the sensors can be transmitted to the data acquisition terminal through wired networks (such as optical fibers) or wireless networks (such as 4G, LoRa, etc.). For railway sections in remote locations, wireless methods are usually preferred to ensure the stability and reliability of data transmission.

[0045] Data storage: The data acquisition terminal stores the received multi-modal data in the database uniformly. For different data types, relational databases (such as MySQL) or non-relational databases (such as MongoDB) can be selected. For example, structured sensor numerical data can be stored in a relational database, while large-scale unstructured image or video data is suitable for storage in a non-relational database.

[0046] 3. Data preprocessing Data cleaning: After reading the original data from the database, methods such as the 3σ principle are used to identify and remove outliers from numerical data. For example, for the strain data collected by track strain sensors, if a value exceeds the mean ± 3 times the standard deviation, it is marked as an outlier and removed. For consecutive abnormal data caused by sensor failures, it can be repaired and complemented through comparison with adjacent time periods or similar sensors, combined with smoothing algorithms (such as the moving average method), to ensure the accuracy and consistency of the data required for subsequent analysis and model training.

[0047] Data normalization: Normalize data with different dimensions. For data such as track displacement and stress, the min-max normalization method is used to scale it to the [0,1] interval, and the formula is as shown in formula 4 below: (4) Where x is the original data, and are the minimum and maximum values of this feature data respectively.

[0048] For some data that needs to be mapped to the [-1,1] interval, the formula is as shown in formula 5 below: (5) Numerical interpolation: For data with missing values, when the data is a time series and the number of missing values is small, the linear interpolation method is used. For example, for the missing temperature data within a certain short period of time, linear fitting can be performed based on the temperature values at the previous and subsequent times to calculate and fill in the missing values; while when the number of missing values is large and the data has spatial correlation (such as the data of pressure sensors at different positions in a tunnel), the K-nearest neighbor interpolation method is used to reasonably fill in the data by referring to the data of adjacent sensors.

[0049] 4. Feature engineering Feature extraction includes time series data and feature selection Time series data: For sequence data such as track vibration and strain that change over time, it can be input into a long short-term memory network (LSTM). LSTM relies on gating mechanisms such as forget gates, input gates, and output gates to effectively handle the long-term dependence problems in time series data, capture the trends, periodicities, and other key dynamic features of the data changing over time, and then output representative feature representations.

[0050] Feature Selection: The mutual information method is used to measure the degree of association between each feature and the label of the health state of railway infrastructure. For example, the mutual information values between features such as the change rate of track strain and vibration frequency and the track health state can be calculated, and features with higher mutual information values are preferentially selected. At the same time, redundant features (for example, features with a correlation coefficient exceeding 0.9) are removed through correlation analysis to reduce the data dimension and improve the efficiency and stability of subsequent model training.

[0051] 5. Model Construction and Training Dataset Division: The data after preprocessing and feature engineering is divided into a training set, a validation set, and a test set according to the ratio of 7:2:1. The training set is used for learning model parameters, the validation set is used for hyperparameter tuning, and the test set is used for finally evaluating the model performance.

[0052] Model Training: Optimization algorithms such as Adam are selected, and appropriate learning rates (for example, 0.001) and the number of iterations (such as 100 times) are set. For the classification task of predicting the health state level of railway infrastructure, the cross-entropy loss function is adopted; while for the regression task of predicting numerical values such as track deformation, the mean squared error (MSE) is used as the loss function. During the training process, the training set data is input into the model in batches, the gradient of the loss function is calculated through the backpropagation algorithm, and the model parameters are continuously updated; at the same time, the model performance is monitored in real-time on the validation set, and the hyperparameters are adjusted according to the validation results to effectively prevent overfitting.

[0053] 6. Model Evaluation Test Data Loading: After the model training is completed, the test set data is loaded into the trained model for performance evaluation.

[0054] Calculation of Evaluation Metrics: For the classification task, metrics such as accuracy, recall rate, and F1 value need to be calculated. For example, when predicting whether there is structural damage to a railway bridge, the accuracy can be obtained by statistically calculating the ratio of the number of correctly predicted samples to the total number of samples; the recall rate can be obtained by calculating the ratio of the number of samples with actual damage and correctly predicted to the number of samples with actual damage, and then the F1 value is calculated. For the regression task, the mean squared error (MSE) and the mean absolute error (MAE) are calculated. For example, when predicting the track deformation, the MSE is obtained by calculating the mean of the squared errors between the predicted value and the true value, and the MAE is obtained by calculating the mean of the absolute values of the errors. The model performance is judged based on these evaluation metrics. If the metrics do not meet the expectations, return to the model construction and training steps, adjust the model architecture or hyperparameters, and retrain.

[0055] 7. Health State Prediction Real-time data processing: In actual operation, by collecting multi-modal data of railway infrastructure in real time, steps such as data collection, transmission, preprocessing, and feature engineering are completed in sequence, and the processed data is input into the trained model.

[0056] Obtaining prediction results: The model outputs the health status prediction results of railway infrastructure. For example, if the predicted track health level is "warning", it indicates that there may be potential problems with the track; or the predicted bridge stress value is output to provide a decision-making basis for bridge maintenance. Maintenance personnel can formulate corresponding maintenance plans based on the prediction results, such as increasing the inspection frequency of tracks in a warning state, or taking structural reinforcement measures for bridges with predicted stress approaching the threshold, so as to ensure the safe and stable operation of railway infrastructure.

[0057] III. Development of the integrated intelligent monitoring platform for railway infrastructure Through the construction of an integrated intelligent monitoring platform, the present invention realizes the effective integration of sensing devices, the efficient construction of a data transmission network, and the deep application of artificial intelligence and big data technologies in the monitoring of railway infrastructure, so as to digitally display the operating environment and status of the infrastructure. The platform enables operation and maintenance personnel to remotely and real-time master the equipment operation situation, improve the operation and maintenance efficiency and fault handling ability, and at the same time complete data collection and cleaning, construct a large model for fault prediction and health management for infrastructure, and realize intelligent operation and maintenance and decision support.

[0058] The integrated monitoring platform mainly consists of the following modules, and each module has several sub-modules. Its overall system structure is as Figure 7 shown: 1. Platform large screen The platform large screen provides an intuitive display of the key information of the system for users and supports the rapid browsing and jumping of data. Its functions include equipment status monitoring, historical data statistics, health prediction analysis, monitoring terminal display, warning information presentation, health ranking and prediction data display, and line data visualization simulation, etc.

[0059] 2. Platform home page The system home page serves as a framework for content display and reflects the management of system function permissions. The page is divided into top information (user information), left menu (function permissions), and middle content display area, forming a complete information interaction interface.

[0060] 3. Dynamic monitoring The dynamic monitoring module is used to count the operation status and warning information of monitoring devices, and real-time display the equipment status, data indicators and trends of each monitoring point. Its sub-modules include monitoring overview, equipment overview, monitoring terminal type overview, monitoring list, monitoring details, and monitoring history, etc.

[0061] 4. Warning analysis For the early warning data generated by the monitoring equipment, the early warning analysis module conducts data statistics and visual display, simulates the occurrence location of the early warning event, and supports the query of early warning history. The main sub-modules include early warning monitoring, early warning list, early warning details, and early warning history, etc.

[0062] 5. Fault Diagnosis According to the threshold values and limit value ranges of different devices, using the neural network deep learning method, analyze and converge the fault diagnosis data, construct a fault information database, and realize the automatic diagnosis of equipment faults.

[0063] 6. Health Assessment Based on the monitoring data and early warning information, the health assessment module uses a variety of algorithms and multi-dimensional analysis to evaluate and score the health status of each monitoring point, and statistically analyzes the health development trend of the equipment. The functions include health overview, assessment list, health details, and health history, etc.

[0064] 7. Fault Prediction Based on parameters such as the bridge health index and its deterioration rate, this module conducts fault prediction on the facilities, analyzes the deteriorated equipment indicators, provides maintenance suggestions, and recommends the best maintenance time window.

[0065] 8. Maintenance Decision According to the equipment health index and fault prediction results, the system automatically generates maintenance work orders, tracks the execution status of the maintenance, continuously evaluates the success rate of the predicted maintenance, and forms a closed-loop management. This module directly guides the on-site maintenance work, reduces the manual labor intensity, and improves the overall maintenance efficiency.

[0066] 9. Comprehensive Analysis The comprehensive analysis module conducts fusion processing on different data in the platform, and uses a variety of analysis methods to display the comprehensive evaluation results, including health trend prediction, facility comprehensive evaluation, and operation report output. Its sub-modules cover prediction overview, prediction list, prediction history, facility analysis, facility analysis details, and terminal status statistics, etc.

[0067] 10. Equipment Management The equipment management module centrally manages and maintains the monitoring terminal equipment, measuring points, and edge controllers. The management results are directly reflected in modules such as dynamic monitoring, early warning analysis, health assessment, and comprehensive analysis. The main functions include measuring point management, terminal management, and edge controller management.

[0068] 11. Basic Information The basic information module mainly displays the ledger data of all facilities and equipment. Through unified warehousing and synchronization updates with other systems, the ledger data is bound to the measuring point information to improve the equipment information. The functions include facility ledger management and ledger list, etc.

[0069] 12. Configuration Management The configuration management module provides the basic configurations required for system operation and business. Through the association of various configuration information, the overall system framework is constructed. The sub-modules cover professional facility types, professional configurations, terminal configurations, edge controller configurations, warning rules and levels, acquisition protocol configurations, etc.

[0070] 13. System Management The system management module is mainly responsible for the management of users, roles, permissions, menus, etc., ensuring the safe and efficient operation of the platform. Its sub-modules include user management, role management, menu management, department management, parameter settings, notice announcements, and log management, etc.

[0071] Through this integrated intelligent monitoring platform, the operating status of railway infrastructure is digitally presented, and seamless connection is achieved in the links of data acquisition, fault diagnosis, health assessment, fault prediction, and maintenance decision-making, forming a set of comprehensive monitoring and operation and maintenance systems with intelligent and closed-loop management, effectively improving the safety and maintenance efficiency of railway operation.

[0072] The technical solution in this embodiment realizes the comprehensive monitoring and intelligent prediction of the health status of railway infrastructure through steps such as data acquisition, preprocessing, deep learning model construction, health status prediction, warning generation, and closed-loop feedback update. It can not only capture key information in multi-modal data in real time, but also improve the prediction accuracy by using advanced deep learning and uncertainty quantification technologies, and continuously optimize the model parameters through the closed-loop feedback mechanism, with high practical application value and promotion prospects. Through the detailed description of the above specific implementation manners, technicians can further improve and expand the application of the railway infrastructure health status prediction system according to the solution provided by the present invention to achieve intelligent and refined management of the entire railway life cycle.

[0073] In one embodiment, as Figure 2 shown, step S11 includes the following steps S21-S24: In step S21, track vibration signal data is collected through vibration sensors installed on the railway track structure; In step S22, image data of railway infrastructure is collected through monitoring camera devices; In step S23, train operation parameter data is obtained from the train operation control system; In step S24, the sampling frequencies of each sensor are dynamically adjusted according to the risk indicators calculated by the real-time risk assessment model, and the data is synchronized according to a unified time benchmark.

[0074] In one embodiment, it mainly includes modules such as vibration signal acquisition, image data acquisition, train operation parameter acquisition, real-time risk assessment, and dynamic sampling frequency adjustment. Data synchronization is achieved among the modules through a unified time reference to comprehensively monitor and warn the state of railway track structures and ancillary facilities. The sensor subsystem consists of a vibration sensor module, an image acquisition module, a train operation parameter acquisition module, a risk assessment and sampling dynamic adjustment module, and a data synchronization processing module. Further, the vibration sensor module uses highly sensitive vibration sensors installed on the railway track structure to collect track vibration signals in real time. The vibration signal data reflects the impact, vibration, and potential abnormal vibration information suffered by the track structure when the train passes, providing an important basis for evaluating the track health status. The image acquisition module uses monitoring cameras installed along the railway line and at key sections to obtain high-definition image data of railway infrastructure (such as sleepers, ballast beds, bridges, etc.). The image data can not only show the external state of the equipment but also detect subtle changes such as cracks and settlements. The train operation parameter acquisition module is connected to the train operation control system through an interface to obtain real-time relevant parameters of train operation, such as vehicle speed, load, acceleration and deceleration conditions, track pressure, etc., providing operation background data for subsequent dynamic risk assessment. The real-time risk assessment model comprehensively considers vibration signals, image data, and train operation parameters, and uses various data fusion algorithms to evaluate the health status of railway infrastructure and output risk indicators. When the risk indicator reaches a preset threshold, the system will dynamically adjust the sampling frequency of each sensor according to the risk level. For example, when abnormal vibration is detected or cracks appear in the image, the sampling frequencies of the vibration sensor and the camera device will be automatically increased to obtain more detailed information and achieve fine monitoring of abnormal events. The data synchronization processing module collects and processes the data of each sensor according to a unified time reference to ensure the temporal consistency among multi-source data. The annotation of a unified timestamp not only facilitates data fusion but also supports subsequent data mining and historical comparison, providing a reliable basis for accident cause analysis and trend prediction.

[0075] During the implementation process, vibration sensors are evenly arranged on the railway track structure to ensure coverage of the main stress-bearing parts; at the same time, monitoring camera devices are installed along the railway line and at key nodes to obtain image data in real time; in addition, through the communication interface with the train operation control system, real-time transmission of train operation parameters is achieved. Each sensor device ensures that the collected data is attached with a unified timestamp through the built-in time synchronization module or GPS signal. After the collected multi-modal data is transmitted to the data processing center, preprocessing is performed on each data, including signal denoising, image enhancement, and abnormal data screening. Through a preset real-time risk assessment model (such as an algorithm model based on neural network or fuzzy logic), fusion calculations are performed on various data to obtain real-time risk indicators. This real-time risk indicator reflects the comprehensive health status and potential risk level of the current railway infrastructure. During the assessment process, historical data can be used as a comparison benchmark to further improve the accuracy of the assessment results. The risk indicator output by the risk assessment module is used as a feedback signal and transmitted to each sensor module through the control unit. According to the comparison result between the risk indicator and the preset threshold, the sampling frequencies of devices such as vibration sensors and monitoring camera devices are dynamically adjusted. For example, when the risk indicator exceeds the warning value, the control unit instructs the sampling frequency of the vibration sensor to increase from the normal 10Hz to 20Hz; similarly, the monitoring camera device will also increase from collecting 24 frames of images per second to collecting 60 frames per second to capture subtle changes. Conversely, in a low-risk or stable state, each sensor returns to the low-sampling mode to save system resources. The data collected by each module is transmitted back to the data processing center through the communication network, and in the data synchronization module, the data from different sensors is corrected for time sequence using the unified timestamp. The corrected data is sent into the subsequent data fusion algorithm to generate a comprehensive dataset that comprehensively reflects the status of the railway infrastructure. This comprehensive dataset can not only be used for real-time monitoring but also stored in the database for subsequent historical data analysis and trend prediction.

[0076] Through the above steps, this embodiment realizes the efficient collection and dynamic management of multi-modal data. The multi-source data collected by different types of sensors can comprehensively monitor the railway status from three perspectives: vibration, vision, and operation parameters; the introduction of the risk assessment model enables the system to have an adaptive adjustment ability, which can automatically increase the sampling frequency when abnormal situations occur, timely capture possible fault information, and thus reduce the probability of accidents; the application of a unified time reference solves the problem of inconsistent time sequences between different data, ensuring the accuracy and effectiveness of data fusion.

[0077] It also includes other types of sensors, such as temperature sensors, stress sensors, etc., to achieve real-time monitoring of the temperature changes in the railway environment and the stress changes in the structure. At the same time, combined with big data analysis and artificial intelligence algorithms, trend prediction can be carried out on the long-term collected data to further improve the preventive maintenance system of railway infrastructure. In addition, to cope with emergencies, an emergency data transmission module can be added to achieve local data caching and delayed transmission when the communication network is limited, so as to ensure that the monitoring data is not lost under extreme conditions. Through modular design, while maintaining high real-time and high-precision data acquisition, it also has good scalability and compatibility, and can be flexibly configured according to different railway operation environments and technical requirements.

[0078] In this embodiment, through technical means such as multi-modal sensor data acquisition, risk assessment and dynamic sampling frequency adjustment, and data synchronization processing, accurate monitoring and timely warning of the state of railway infrastructure are realized, which not only improves the real-time performance and reliability of the monitoring system, but also reduces the maintenance cost and operation risk.

[0079] In one embodiment, as Figure 3 shown, step S12 includes the following steps S31 - S23: In step S31, noise filtering, outlier removal, and interpolation processing of missing data are performed on the multi-modal sensor data; In step S32, data from different sources are aligned based on timestamps, and features including time-domain features, frequency-domain features, and spatial topology features are extracted; In step S33, the extracted feature parameters are normalized to form standardized data for subsequent model training.

[0080] In one embodiment, the preprocessing process mainly includes noise filtering, outlier removal, and interpolation processing of missing data on the original data; aligning data from different sensors based on a unified time reference; extracting multi-dimensional features including time-domain features, frequency-domain features, and spatial topology features; and normalizing the extracted features to form standardized data.

[0081] In multi-modal sensor data, noise and outliers are common problems. Digital filtering techniques (such as low-pass filtering, mean filtering, or wavelet transform, etc.) are used to filter the noise of vibration, image, and other sensor signals, removing high-frequency noise interference to ensure data stability. For outliers, statistical methods, such as the mean ± 3 times standard deviation method, box plot method, or outlier detection based on clustering algorithms, are used to automatically identify and remove outlier data points. Taking the railway vibration signal as an example, when the vibration amplitude collected by the sensor far exceeds the normal operation range, the data will be automatically marked and removed to avoid affecting the accuracy of subsequent feature extraction due to noise or sudden anomalies.

[0082] In practical applications, due to reasons such as transmission delays or equipment failures, data loss may occur. To ensure data continuity, an interpolation algorithm is used to complete the missing data. Common interpolation methods include linear interpolation, spline interpolation, and nearest neighbor-based interpolation methods. For different types of data, an appropriate interpolation algorithm can be automatically adapted. For example, for time series data, linear interpolation is used, which is simple and can better restore the data trend; for the case of missing image data, an interpolation method based on spatial correlation can be used to ensure the continuity of image texture information.

[0083] Multimodal data often comes from different sensors, and there are differences in their acquisition times. To achieve data fusion, it is necessary to align according to the timestamps attached to each data. After preprocessing each data stream, extract their respective timestamp information; according to a preset unified time reference, align the sensor data on the time axis through interpolation or resampling methods. It can ensure that data from different sources have consistent time identifiers at the same moment, thus providing an accurate time reference for subsequent feature extraction.

[0084] For the preprocessed data, further extract multi-dimensional features. Conduct statistical analysis on the original time series data, such as parameters like mean, standard deviation, maximum value, minimum value, skewness, and kurtosis. Time-domain features reflect the overall distribution and change trend of data over time, and can directly reflect the railway operation state or equipment vibration characteristics. Use methods such as Fourier transform and wavelet transform to convert the time-domain data to the frequency domain, and extract parameters such as spectral energy, main frequency components, and frequency band energy distribution. For image data or multi-sensor network layouts, extract their spatial structure information, such as edges, texture features in images, or spatial correlations between sensors. By extracting spatial features, structural defects or local damage conditions can be identified, improving the sensitivity of the monitoring system to local abnormal changes.

[0085] For feature parameters of different scales and dimensions, a normalization algorithm is used for data standardization processing. The main role of normalization processing is to eliminate the influence caused by different dimensions between features and avoid biases caused by numerical scale problems during subsequent model training. Taking Z-score normalization as an example, by subtracting the mean and dividing by the standard deviation, the feature data follows a standard normal distribution with a mean of 0 and a variance of 1, thereby improving the convergence speed and prediction accuracy of model training.

[0086] Noise filtering and outlier rejection significantly reduce the interference information in the original data, making subsequent feature extraction more accurate and reliable. The alignment based on timestamps ensures the temporal consistency of different sensor data, providing a solid foundation for multi-modal data fusion. Meanwhile, extracting time-domain, frequency-domain, and spatial topology features can not only comprehensively reflect the operating state of the device but also capture hidden fault information, enhancing the diagnostic ability of the model. Normalization effectively eliminates the scale differences between data, providing stable and efficient training data for machine learning models and improving the accuracy of subsequent prediction and recognition.

[0087] For example, in a certain railway operation monitoring system, there is a certain amount of random noise and occasional abnormal vibrations in the original data collected by vibration sensors. After low-pass filtering and outlier rejection based on the mean ± 3 times the standard deviation, the noise in the data is significantly reduced. At the same time, due to missing data caused by network transmission problems during a certain period, the system uses linear interpolation to fill in the data for that period. Subsequently, the vibration data, image monitoring data, and train operation parameters are aligned using a unified timestamp, and time-frequency features such as the mean, standard deviation, and frequency-domain energy distribution are extracted from the vibration signal; edge, texture, and local shape features are extracted from the image. After Z-score standardization of these feature parameters, a set of standardized data is formed for the training and prediction of subsequent fault diagnosis models. This example not only proves the practicality of this preprocessing method but also demonstrates its significant advantages in improving the data quality and stability of the monitoring system.

[0088] In this embodiment, by performing noise filtering, outlier rejection, missing data interpolation, data alignment, feature extraction, and normalization on multi-modal sensor data, not only the quality and reliability of the original data are improved, but also a stable standardized data foundation is provided for subsequent model training, which has wide applicability and practical application value. For those skilled in the art, by improving and optimizing the above preprocessing scheme, the overall performance of the monitoring system can be further enhanced to meet the higher requirements for data processing and intelligent analysis.

[0089] In one embodiment, as Figure 4 shown, step S13 includes the following steps S41 - S43: In step S41, a multi-modal data fusion deep learning model integrating a multi-task deep neural network self-organizing mapping module and a graph neural network module is constructed, where each data modality is processed by an independent feature extraction sub-network, and spatial topological associations between railway monitoring points are established through the graph neural network; In step S42, a Bayesian uncertainty quantification module is embedded in a multi-task long short-term memory network (MTL-LSTM), and the model is trained using historical multi-modal data and corresponding health status labels; In step S43, the prediction accuracy of the model is evaluated on the validation dataset, and the model parameters or structure are adjusted according to the evaluation results until the model converges, so as to improve the prediction accuracy and model adaptability.

[0090] In one embodiment, for the multimodal data fusion deep learning model for railway monitoring data, its main purpose is to make full use of data from different sensors, and through the collaborative work of deep neural networks, graph neural networks and multi-task long short-term memory networks (MTL-LSTM), achieve accurate prediction and uncertainty assessment of the railway health state.

[0091] Each data modality (such as vibration signals, image data, train operation parameters, etc.) is processed by an independent feature extraction sub-network. Specifically, for vibration signals, a convolutional neural network (CNN) or an autoencoder is used to extract time-domain and frequency-domain features; for image data, a deep convolutional network is used to extract edge, texture, and local shape features; and for operation parameters, the time correlation is extracted through a fully connected layer or a recurrent network. Each sub-network undergoes noise filtering, outlier removal, and data interpolation, etc. in the preprocessing stage to ensure that the input data has high quality and consistency. To achieve effective fusion of multimodal features, a fusion multi-task deep neural network self-organizing mapping module is constructed. Through self-organizing mapping, the features of different modalities are adaptively adjusted in the high-dimensional feature space, and the feature representations under different health state labels are learned simultaneously through multi-task learning. After the feature vectors output by each feature extraction sub-network are uniformly mapped, they enter the subsequent fusion layer to form a comprehensive representation. Considering that the distribution of railway monitoring points has a clear spatial topological relationship, a graph neural network is used to model the spatial association between each monitoring point. Each monitoring point is regarded as a node in the graph, and the connection relationship between nodes is constructed based on geographical location, equipment type, and historical data correlation. The graph neural network realizes the aggregation and update of node information through the message passing mechanism, thereby capturing local and global spatial structure information and improving the accuracy of fault prediction. To further explore the potential patterns in the time-series data, a multi-task long short-term memory network (MTL-LSTM) is embedded at the input end of the fused features to capture the dynamic change trends in the time series. At the same time, a Bayesian uncertainty quantification module is introduced into the multi-task long short-term memory network to quantitatively analyze the uncertainty in the model prediction process. Through Bayesian modeling of the network weights, the confidence interval of the prediction result is calculated, so as to provide risk assessment information for decision-makers and play a regularization role in model training.

[0092] During the model training stage, a training dataset is constructed using historical multi-modal data and corresponding health status labels. The dataset contains sensor data under different operating conditions and corresponding fault, anomaly, or health status labels, providing sufficient training samples for the model. A joint training strategy is adopted to jointly construct a self-organizing mapping module, a graph neural network module, and an MTL-LSTM into an integrated model. During the training process, different tasks (such as predicting the occurrence of faults and evaluating the health of equipment) share the underlying feature representations and are jointly optimized through a multi-task loss function. A Bayesian uncertainty quantification module is also embedded to estimate the uncertainty of model parameters using methods such as Monte Carlo sampling or variational inference, enabling the model to give a probability distribution rather than a single numerical output during prediction. During the model training process, the prediction accuracy of the model is evaluated on the validation dataset to guide parameter adjustment. If the error on the validation set exceeds a preset threshold, the network structure (such as increasing or decreasing the number of hidden layer nodes, adjusting the learning rate, changing the number of samplings, etc.) is optimized according to the feedback results until the model converges. Strategies such as early stopping and cross-validation are adopted in this process to prevent overfitting and ensure that the model has good generalization ability.

[0093] By using independent feature extraction sub-networks for different data modalities, the advantages of each sensor can be fully utilized to achieve efficient decoupling and fusion of data. The self-organizing mapping module realizes the adaptive integration of cross-modal features during the joint learning process, improving the model's discrimination ability for complex fault patterns. The graph neural network module not only improves the detection rate of local faults by modeling the spatial relationships between railway monitoring points but also captures the hidden connections between different monitoring points, facilitating information sharing and local collaboration in a distributed monitoring environment and significantly enhancing the overall reliability of the system. In traditional deep learning models, the prediction results are usually deterministic outputs, while in practical applications, uncertainties may be caused by factors such as data noise and model bias. Through the Bayesian uncertainty quantification module, while ensuring prediction accuracy, a prediction confidence interval can be given to assist engineering decision-makers in quantitatively evaluating risks, which has important practical application significance.

[0094] Taking a railway monitoring system as an example, the system is deployed at multiple key monitoring points to collect multimodal data such as vibration, images, and operating parameters. After the data at each monitoring point undergoes independent feature extraction, it is input into the self-organizing mapping module for fusion, and then the graph neural network constructs the spatial topological relationship between each monitoring point. At this time, the fused feature data captures the time series dynamics through MTL-LSTM and is embedded in the Bayesian module to evaluate the uncertainty of the prediction results. During the training process, using historical data and health status labels, through the optimization of the multi-task loss function, the model continuously adjusts the parameters on the validation set until convergence. Accurately predict the possible failure risks at a certain monitoring point in the future for a period of time, and output the corresponding prediction confidence interval to assist maintenance personnel in formulating preventive maintenance plans.

[0095] In this embodiment, by constructing a multimodal data fusion deep learning model integrating a self-organizing mapping module, a graph neural network module, and MTL-LSTM, it not only realizes the comprehensive extraction and fusion of railway monitoring data, but also provides a reliable risk assessment for the prediction results through the Bayesian uncertainty quantification module. Through joint training, multi-task learning, and parameter optimization, the model continuously iterates and updates on the validation data set until the expected convergence effect is achieved, significantly improving the prediction accuracy and model adaptability. It has the advantages of comprehensive data processing, accurate fault prediction, and strong system robustness, and is suitable for various complex monitoring scenarios.

[0096] In one embodiment, as Figure 5 shown, step S15 includes the following steps S51 - S53: In step S51, input the real-time collected and preprocessed multimodal data into the trained multimodal data fusion deep learning model to obtain the health status prediction results and confidence intervals of the railway infrastructure; In step S52, compare the prediction results with the preset health threshold to determine the health status level of the railway infrastructure; In step S53, when the prediction results exceed the normal range, generate warning information and maintenance suggestions, and use the attention mechanism to quantify the relevance between the multimodal data and the prediction results, and output a root cause analysis report of the fault; In step S54, output any one or more of the health status level, confidence interval, warning information, and maintenance suggestions to provide a basis for maintenance decisions.

[0097] In one embodiment, through the preprocessed multimodal data and the trained multimodal data fusion deep learning model, predict the health status of the railway infrastructure, and output multiple pieces of information such as the health status level, the confidence interval of the prediction results, warning information, and maintenance suggestions, to provide a basis for the safety maintenance and decision-making of railway facilities.

[0098] The multi-modal data (including vibration signals, image data, train operation parameters, etc.) of each monitoring point on the railway is collected through the real-time monitoring module. After preprocessing processes such as noise filtering, outlier removal, and missing data interpolation, each data stream is aligned according to a unified timestamp and features in the time domain, frequency domain, and spatial topology are extracted through independent feature extraction sub-networks. The processed standardized data is used as the model input to ensure high quality and consistency of data from different sources. The preprocessed data is input into a trained multi-modal data fusion deep learning model. The multi-modal data fusion deep learning model consists of a multi-task deep neural network self-organizing mapping module for fusion, a graph neural network module, and a multi-task long short-term memory network (MTL-LSTM) embedded with a Bayesian uncertainty quantification module. After synthesizing multi-modal features, the multi-modal data fusion deep learning model not only outputs the health status prediction results, but also quantifies the uncertainty of the model during the prediction process through the Bayesian method, and then obtains the confidence interval of the prediction results. The confidence interval provides a quantitative basis for subsequent risk assessment and decision-making, reflecting the credibility of the model prediction.

[0099] Based on historical data statistical analysis and expert experience, multiple health thresholds are preset to classify the status of railway infrastructure into several levels, such as "excellent", "good", "average", "poor", and "dangerous". The preset thresholds not only reflect the health index ranges of different devices under normal working conditions, but also provide a boundary basis for risk warning. The health status prediction results output by the deep learning model are compared with the preset thresholds. When the prediction results fall within the normal range, it is determined to be in a normal state, corresponding to outputting levels such as "excellent" or "good". If the prediction results exceed the preset normal range, it indicates potential risks, and it will be correspondingly determined to be in a "poor" or "dangerous" state, and the subsequent warning mechanism will be triggered, realizing automated health status level determination.

[0100] When the prediction result exceeds the normal threshold, it automatically enters the exception handling process. Further analysis is carried out on the over-standard data, and the severity of the exception is confirmed by combining historical data and the expert knowledge base, and then a warning message is generated. The warning message not only includes the exception prompt of the current health status level, but also identifies the potential failure risk areas and their development trends. For different abnormal situations, according to the failure mode library and maintenance experience, targeted maintenance suggestions are automatically generated. The content of the suggestions may involve measures such as reducing the train operation speed, strengthening local inspections, adjusting operation parameters, or immediately arranging on-site maintenance, so as to take intervention measures in time before the failure spreads. To improve the accuracy of the warning and the scientific nature of the decision-making, an attention mechanism is introduced to quantify the correlation between multi-modal data and the prediction result. By analyzing the contribution degree of each data modality to the final prediction result, a detailed root cause analysis report of the failure can be output. The root cause analysis report of the failure makes a quantitative description of the source, key features and possible failure modes of the abnormal data.

[0101] According to different application scenarios and user requirements, the health status level, confidence interval, warning information and maintenance suggestions are displayed in graphical form on the monitoring large screen or mobile terminal, which is convenient for on-site personnel to grasp the equipment status in real time. The relevant data is transmitted to the central monitoring platform or cloud server through the network for further analysis and decision-making by experts. The root cause analysis report of the failure is automatically generated according to the preset cycle, and relevant responsible persons are notified by means of emails, text messages, etc. to ensure timely response. The output information is not limited to a single piece of data, but integrates and outputs the health status level, confidence interval, warning information and maintenance suggestions to form a multi-level and all-round decision support system. Maintenance personnel can quickly formulate maintenance plans according to the output information and conduct on-site inspections and repairs targeted according to the root cause report of the failure, so as to effectively reduce the accident risk and maintenance cost.

[0102] Using the multi-modal data fusion deep learning model, not only can the key information in different data sources such as vibration, images and operation parameters be captured, but also through joint training and Bayesian uncertainty quantification, the prediction result has high accuracy and credibility. The preset health threshold and automatic determination mechanism realize an automated closed-loop from data input to decision output. The attention mechanism is used to quantitatively analyze the influence of each data modality on the prediction result, highlight the key abnormal information, and then generate a detailed root cause analysis report of the failure. This mechanism not only improves the response speed to abnormal situations, but also provides a data basis for subsequent technical improvements. By outputting any one or more of the health status level, confidence interval, warning information and maintenance suggestions, a multi-dimensional information interaction is formed to help maintenance personnel more comprehensively understand the equipment status, so as to make scientific and reasonable maintenance decisions. This multi-level and all-round information output greatly improves the overall risk prevention and control ability and decision-making efficiency of the system.

[0103] Taking a certain railway section as an example, the monitoring system collects preprocessed multi-modal data during routine monitoring. After being processed by a deep learning model, the predicted health status result is 0.65 (the value range is from 0 to 1, where 0 - 0.5 represents the normal state, 0.5 - 0.7 represents the general state, and above 0.7 represents the abnormal state), and the confidence interval is ±0.1. After comparing this result with the preset health threshold, it is determined that this section is in a "poor" state, and the system automatically triggers an alarm. The warning information generated by the system is "It is recommended to immediately conduct key inspections on this section and reduce the train passing speed." At the same time, the attention mechanism is used to compare the correlation between the data of each sensor and the prediction result, and a root cause analysis report of the fault is output. The report points out that the abnormality in this section may be caused by local foundation settlement, and it is recommended to further verify it in combination with historical maintenance records. Based on this, the maintenance personnel promptly dispatched a repair team to conduct on-site inspections and reinforcement treatments on this section, avoiding the further expansion of the accident.

[0104] In this embodiment, by inputting the real-time collected and preprocessed multi-modal data into a trained multi-modal data fusion deep learning model, the predicted health status result and the confidence interval of the railway infrastructure are obtained; the preset health threshold is used to compare the prediction result to automatically determine the health status level; when the prediction result exceeds the normal range, the warning mechanism is triggered to generate warning information and maintenance suggestions, and a root cause analysis report of the fault is output by means of the attention mechanism; one or more of the health status level, confidence interval, warning information, and maintenance suggestions are output in an intuitive and flexible manner, providing a comprehensive and scientific basis for maintenance decision-making. This method has the advantages of accurate prediction, timely response, precise fault location, and wide application range, and can significantly improve the monitoring level and safety maintenance efficiency of railways and other infrastructure, providing a solid technical foundation for subsequent technology upgrades and system integration.

[0105] In one embodiment, as Figure 6 shown, the following steps S61 - S64 are further included: In step S61, based on any one or more of the output health status level, confidence interval, warning information, and maintenance suggestions, targeted inspections or maintenance are carried out on the railway infrastructure, and the actual status data after inspection or maintenance is collected as feedback data; In step S62, the feedback data is compared with the corresponding prediction result to evaluate the accuracy of the model prediction; In step S63, using the feedback data, the multi-modal data fusion deep learning model is updated online adaptively by means of a closed-loop feedback mechanism using reinforcement learning or meta-learning algorithms; In step S64, the health threshold, sensor sampling strategy, and maintenance plan are dynamically adjusted according to the results of the updated multi-modal data fusion deep learning model, thereby further optimizing the prediction of the health status of railway infrastructure.

[0106] In one embodiment, an online adaptive update of the railway infrastructure health status prediction system is performed based on a feedback closed-loop mechanism. After a targeted inspection or maintenance of railway facilities, the actual status data is collected as feedback data and compared with the prediction results to evaluate the prediction accuracy of the model, and an online adaptive update of the model is achieved using reinforcement learning or meta-learning algorithms, thereby dynamically adjusting the health threshold, sensor sampling strategy, and maintenance plan.

[0107] According to the health status level, confidence interval, early warning information, and maintenance suggestions output by the system, maintenance personnel perform targeted inspections or repairs on abnormal areas or railway facilities with higher risks. During the inspection or maintenance process, actual status data, such as vibration amplitude, temperature change, structural stress, and image monitoring data, is collected through sensors and detection equipment installed on-site and stored in the system database as feedback data. The feedback data not only reflects the true status after the current maintenance but also serves as an important basis for subsequent evaluation of the model's prediction accuracy. The collected feedback data is compared with the prediction results output by the multi-modal data fusion deep learning model within the corresponding time period. Error analysis methods, such as mean square error and mean absolute error, are used to calculate the deviation between the prediction and the actual status to judge the prediction accuracy of the model. At the same time, combined with the confidence interval information, abnormal data is reconfirmed to ensure that the system can timely detect prediction results with large deviations during high-risk early warnings, thereby providing data support for subsequent model updates.

[0108] As a key link in the closed-loop system, after data comparison and error evaluation, the feedback data is fed back to the model training module. The feedback data is not only used to verify the accuracy of the current model but also serves as new training samples to continuously enrich the data set. Using the online learning mechanism, the model parameters can be adjusted in real-time after obtaining new feedback data to ensure that the model always adapts to the changes in the actual operating state.

[0109] To achieve the adaptive update of the model, this embodiment introduces reinforcement learning or meta-learning algorithms. Reinforcement learning: Set the state space as multi-dimensional information such as the current health state, prediction error, and feedback data, and the action space includes adjusting model parameters, modifying network structures, and updating weights. Through a reward mechanism (such as improved prediction accuracy and reduced error), continuously optimize the strategy to achieve the rapid adaptation of the model in a dynamic environment. Meta-learning: Construct a meta-learning framework for the model, enabling the model to quickly adjust and transfer learning when receiving a small amount of new feedback data, and enhancing the generalization ability of the model in different operating environments. Through a closed-loop feedback mechanism, the model automatically checks the difference between the prediction result and the actual state during each update process, and then realizes online adaptive adjustment through the reinforcement learning reward signal or meta-learning strategy, improving the overall prediction accuracy and robustness.

[0110] Based on the comparison result between the feedback data and the prediction result, dynamically optimize the preset health threshold. If there is a continuous prediction deviation or a systematic error between the model prediction result and the actual state, use the updated model result to reset the threshold for each health level to ensure that the health state level division is more reasonable and conforms to the current operating conditions of railway facilities. After the model is updated, dynamically adjust the sampling frequency and sampling method of each sensor according to the confidence interval information of the new prediction result and the change of the feedback data. For example, when there is a large uncertainty in certain areas, the sensor sampling frequency can be increased to obtain richer data, and vice versa to save resources. The dynamic adjustment of this strategy ensures the real-time and efficient data collection, and at the same time provides more refined data support for model update. Combine the updated health state prediction result and feedback data to automatically generate a new maintenance plan and early warning scheme. The maintenance suggestions not only include regular maintenance measures, but also combine the actual failure cause analysis report to propose targeted preventive measures, forming a complete full-life cycle maintenance management system, thereby further improving the safety and reliability of railway infrastructure.

[0111] For example, the output result of the health state prediction model of a certain railway section shows that the section is in a "poor" state, the predicted value is 0.78, the confidence interval is ±0.08, and the early warning information recommends a local inspection. The maintenance personnel conduct a detailed inspection of the section according to the recommendation and collect the actual on-site state data. After comparing the feedback data, it is found that the actual state prediction error is large (the actual value is 0.65). Analyze the cause of the deviation through the reinforcement learning algorithm, quickly adjust the model parameters, and at the same time reset the health threshold of the section. The updated model has a closer prediction result to the actual state in subsequent monitoring, and the error is significantly reduced. In addition, the sensor sampling frequency is automatically optimized to make the subsequent data collection more accurate, and the maintenance plan is adjusted according to the root cause report of the failure, providing detailed guidance for on-site maintenance.

[0112] In this embodiment, the actual status data after inspection or maintenance is collected and compared with the model prediction results. Under the closed-loop feedback mechanism, the online adaptive update of the model is realized by using reinforcement learning or meta-learning algorithms. By dynamically adjusting the health threshold, sensor sampling strategy, and maintenance plan, not only the accuracy of railway infrastructure health status prediction is effectively improved, but also a scientific basis is provided for maintenance decision-making, significantly enhancing the intelligence and adaptive ability of the system.

[0113] In one embodiment, Figure 8 is a block diagram of a railway infrastructure health status prediction device shown according to an exemplary embodiment. As Figure 8 shown, the railway infrastructure health status prediction device includes an acquisition module 81, a preprocessing module 82, a training module 83, a prediction module 84, and an adaptive module 85.

[0114] The acquisition module 81 is used to collect multi-modal sensor data reflecting the status of railway infrastructure. The multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and adjusts the sampling frequency of each sensor according to real-time risk assessment using an adaptive dynamic sampling strategy; The preprocessing module 82 is used to preprocess the multi-modal sensor data. The preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation filling, feature extraction, and normalization. The feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; The training module 83 is used to construct and train a multi-modal data fusion deep learning model. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the features of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which is embedded with a Bayesian uncertainty quantification module to obtain the health status prediction result and the confidence interval; The prediction module 84 is used to input the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model, obtain the health status prediction result of the railway infrastructure, and compare the health status prediction result with a preset health threshold to generate a warning message; The adaptive module 85 is used to collect maintenance feedback data, and online adaptively update the multi-modal data fusion deep learning model by using reinforcement learning or meta-learning methods through a closed-loop feedback mechanism. At the same time, an interpretable artificial intelligence module is constructed by using an attention mechanism to perform root cause analysis of faults on the prediction results, and output any one or more of a health status level, a confidence interval, a warning message, and a maintenance suggestion.

[0115] The acquisition module 81, the preprocessing module 82, the training module 83, the prediction module 84, and the adaptive module 85 included in the block diagram of the railway infrastructure health status prediction device are controlled to execute the railway infrastructure health status prediction method described in any one of the above embodiments.

[0116] As Figure 9 shown, the present invention provides an electronic device 900, which includes: a communication interface, a processor 901, and a memory 902; Among them, the memory 902 is used to store program instructions. When the program instructions are executed by the processor 901 communicatively connected to the memory 902 through the communication interface, multi-modal sensor data reflecting the state of the railway infrastructure is collected. The multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and the sampling frequencies of each sensor are adjusted according to a real-time risk assessment by using an adaptive dynamic sampling strategy; the multi-modal sensor data is preprocessed, and the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation and filling, feature extraction, and normalization. The feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; a multi-modal data fusion deep learning model is constructed and trained. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which embeds a Bayesian uncertainty quantification module to obtain a health status prediction result and a confidence interval; the preprocessed data collected in real time is input into the trained multi-modal data fusion deep learning model to obtain a health status prediction result of the railway infrastructure, and the health status prediction result is compared with a preset health threshold to generate a warning message; maintenance feedback data is collected, and the multi-modal data fusion deep learning model is online adaptively updated by using reinforcement learning or meta-learning methods through a closed-loop feedback mechanism. At the same time, an interpretable artificial intelligence module is constructed by using an attention mechanism to perform root cause analysis of faults on the prediction results, and output any one or more of a health status level, a confidence interval, a warning message, and a maintenance suggestion.

[0117] The present invention provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, multi-modal sensor data reflecting the state of railway infrastructure is collected. The multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and an adaptive dynamic sampling strategy is adopted to adjust the sampling frequency of each sensor according to real-time risk assessment; the multi-modal sensor data is preprocessed, and the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation and filling, feature extraction, and normalization. The feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; a multi-modal data fusion deep learning model is constructed and trained. The multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing map (MTDNN-SOM) module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network (MTL-LSTM) module, which is embedded with a Bayesian uncertainty quantification module to obtain a health state prediction result and a confidence interval; the preprocessed data collected in real time is input into the trained multi-modal data fusion deep learning model to obtain a health state prediction result of the railway infrastructure, and the health state prediction result is compared with a preset health threshold to generate a warning message; maintenance feedback data is collected, and the multi-modal data fusion deep learning model is updated online adaptively by using a reinforcement learning or meta-learning method through a closed-loop feedback mechanism. At the same time, an interpretable artificial intelligence module is constructed by using an attention mechanism to perform a root cause analysis of the prediction result, and any one or more of a health state level, a confidence interval, a warning message, and a maintenance suggestion are output.

[0118] It should be understood that the specific features, operations, and details described above regarding the method of the present invention can also be similarly applied to the device and system of the present invention, or vice versa. In addition, each step of the method of the present invention described above can be executed by the corresponding component or unit of the device or system of the present invention.

[0119] It should be understood that each module / unit of the device of the present invention can be implemented in whole or in part by software, hardware, firmware, or a combination thereof. Each module / unit can be embedded in the processor of a computer device in the form of hardware or firmware or independent of the processor, or stored in the memory of the computer device in the form of software for the processor to call to execute the operations of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0120] In one embodiment, a computer device is provided, which includes a memory and a processor. Computer instructions executable by the processor are stored on the memory. When the computer instructions are executed by the processor, the processor is instructed to execute the steps of the method of the embodiments of the present invention. The computer device can generally be a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc., connected through a system bus. The processor of the computer device can be used to provide necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the method of the present invention are executed.

[0121] The present invention can be implemented as a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method of the embodiments of the present invention are caused to be executed. In one embodiment, the computer program is distributed among a plurality of network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed by one or more computer devices or processors in a distributed manner. A single method step / operation, or two or more method steps / operations, can be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations can be executed by one or more computer devices or processors, and one or more other method steps / operations can be executed by one or more other computer devices or processors. One or more computer devices or processors can execute a single method step / operation, or execute two or more method steps / operations.

[0122] Those of ordinary skill in the art can understand that the method steps of the present invention can be instructed by a computer program to complete relevant hardware such as computer devices or processors. The computer program can be stored in a non-transitory computer-readable storage medium, and when the computer program is executed, the steps of the present invention are caused to be executed. Depending on the circumstances, any reference herein to a memory, storage, database, or other medium may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc. By collecting multi-modal sensor data such as track vibration, images, and train operation, and adopting preprocessing technologies such as adaptive dynamic sampling, noise filtering, time series alignment, and feature extraction, data standardization is achieved; multi-modal features are fused using a multi-task deep neural network self-organizing map and a graph neural network, supplemented by a multi-task long short-term memory network and a Bayesian uncertainty quantification module to accurately predict the railway health status; and closed-loop feedback, real-time warning, and root cause analysis of faults are used to assist decision-making, significantly reducing the accident risk.

[0123] The above-described technical features can be combined arbitrarily. Although not all possible combinations of these technical features have been described, any combination of these technical features should be considered to be covered by this specification, as long as such a combination does not exist in contradiction.

[0124] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the health status of railway infrastructure, characterized in that, Including: Collecting multi-modal sensor data reflecting the state of railway infrastructure, where the multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and adjusting the sampling frequency of each sensor using an adaptive dynamic sampling strategy based on real-time risk assessment; Preprocessing the multi-modal sensor data, where the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation for missing data, feature extraction, and normalization, and the feature extraction includes any one or more of time-domain features, frequency-domain features, and spatial topology features; Constructing and training a multi-modal data fusion deep learning model, where the multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing mapping module and a graph neural network module, which is used to extract and fuse the feature of each sensor data and model the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network module, which is embedded with a Bayesian uncertainty quantification module to obtain the health state prediction result and the confidence interval; Inputting the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model to obtain the health state prediction result of the railway infrastructure, and comparing the health state prediction result with a preset health threshold to generate a warning message; Collecting maintenance feedback data, and online adaptively updating the multi-modal data fusion deep learning model using reinforcement learning or meta-learning methods through a closed-loop feedback mechanism. At the same time, using an attention mechanism to construct an interpretable artificial intelligence module to perform root cause analysis of the prediction result and output any one or more of the health state level, confidence interval, warning message, and maintenance suggestion.

2. The method for predicting the health state of railway infrastructure according to claim 1, characterized in that, The collecting multi-modal sensor data reflecting the state of railway infrastructure includes: Collecting track vibration signal data through vibration sensors installed on the railway track structure; Collecting image data of railway infrastructure through monitoring cameras; Obtaining train operation parameter data from the train operation control system; Dynamically adjusting the sampling frequency of each sensor according to the risk index calculated by the real-time risk assessment model, and synchronizing each data according to a unified time reference.

3. The method for predicting the health state of railway infrastructure according to claim 1, wherein The preprocessing the multi-modal sensor data includes: Performing noise filtering, outlier removal, and interpolation processing of missing data on the multi-modal sensor data; Aligning data from different sources based on timestamps and extracting features including time-domain features, frequency-domain features, and spatial topology features; Normalizing the extracted feature parameters to form standardized data for subsequent model training.

4. The method for predicting the health status of railway infrastructure according to claim 1, characterized in that, The constructing and training a multi-modal data fusion deep learning model includes: Constructing a multi-modal data fusion deep learning model that integrates a multi-task deep neural network self-organizing mapping module and a graph neural network module, where each data modality is processed by an independent feature extraction sub-network, and the spatial topology association between railway monitoring points is established through a graph neural network; Embed a Bayesian uncertainty quantification module in the multi-task long short-term memory network, and use historical multi-modal data and corresponding health status labels to train the model; Evaluate the prediction accuracy of the model on the validation dataset, and adjust the model parameters or structure according to the evaluation results until the model converges to improve the prediction accuracy and model adaptability.

5. The method for predicting the health state of railway infrastructure according to claim 4, characterized in that, The output of any one or more of the health status level, confidence interval, warning information, and maintenance suggestions includes: Input the multi-modal data collected and preprocessed in real time into the trained multi-modal data fusion deep learning model to obtain the health status prediction result and confidence interval of the railway infrastructure; Compare the prediction result with the preset health threshold to determine the health status level of the railway infrastructure; When the prediction result exceeds the normal range, generate warning information and maintenance suggestions, and use the attention mechanism to quantify the correlation between the multi-modal data and the prediction result, and output a fault root cause analysis report; Output any one or more of the health status level, confidence interval, warning information, and maintenance suggestions to provide a basis for maintenance decisions.

6. The method for predicting the health status of railway infrastructure according to claim 5, characterized in that, It also includes: Based on any one or more of the output health status level, confidence interval, warning information, and maintenance suggestions, conduct targeted inspections or maintenance on the railway infrastructure, and collect the actual status data after inspection or maintenance as feedback data; Compare the feedback data with the corresponding prediction results to evaluate the accuracy of the model prediction; Use the feedback data to perform online adaptive update on the multi-modal data fusion deep learning model through a closed-loop feedback mechanism using reinforcement learning or meta-learning algorithms; Dynamically adjust the health threshold, sensor sampling strategy, and maintenance plan according to the results of the updated multi-modal data fusion deep learning model, so as to further optimize the health status prediction of the railway infrastructure.

7. A device for predicting the health status of railway infrastructure, characterized in that, It includes: A collection module for collecting multi-modal sensor data reflecting the status of railway infrastructure, where the multi-modal sensor data includes any one or more of vibration signals, image data, and train operation parameters, and adjusts the sampling frequency of each sensor using an adaptive dynamic sampling strategy based on real-time risk assessment; A preprocessing module for preprocessing the multi-modal sensor data, where the preprocessing includes any one or more of noise filtering, outlier removal, time alignment, interpolation filling, feature extraction, and normalization, and the feature extraction includes any one or more of time domain features, frequency domain features, and spatial topology features; A training module for constructing and training a multi-modal data fusion deep learning model, where the multi-modal data fusion deep learning includes a first module and a second module. The first module is a combination of a multi-task deep neural network self-organizing mapping module and a graph neural network module for extracting and fusing the features of each sensor data and modeling the spatial topology association between railway monitoring points. The second module is a multi-task long short-term memory network module with a Bayesian uncertainty quantification module embedded to obtain the health status prediction result and confidence interval; A prediction module, configured to input the preprocessed data collected in real time into the trained multi-modal data fusion deep learning model, obtain the prediction result of the health status of the railway infrastructure, and compare the health status prediction result with a preset health threshold to generate a warning message; An adaptive module, configured to collect maintenance feedback data, and perform online adaptive update on the multi-modal data fusion deep learning model by using a reinforcement learning or meta-learning method through a closed-loop feedback mechanism. Meanwhile, an interpretable artificial intelligence module is constructed by using an attention mechanism to perform root cause analysis of the prediction result, and output any one or more of the health status level, confidence interval, warning message, and maintenance suggestion.

8. The railway infrastructure health status prediction device according to claim 7, wherein the device is deployed on a railway infrastructure health status prediction platform the railway infrastructure health status prediction platform includes any one or more of a platform large screen, a platform home page, dynamic monitoring, warning analysis, fault diagnosis, health assessment, fault prediction, maintenance decision-making, comprehensive analysis, equipment management, basic information, configuration management, and system management.

9. An electronic device, characterized in that, including: a communication interface, a processor, and a memory; wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor communicatively connected to the memory through the communication interface, the electronic device implements the railway infrastructure health status prediction method according to any one of claims 1 to 6.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by a computer, the computer implements the railway infrastructure health status prediction method according to any one of claims 1 to 6.

Images