High-speed rail environment monitoring method based on multi-mode sensor

Abstract

The invention relates to the technical field of Internet of Things, in particular to a high-speed rail environment monitoring method based on a multi-mode sensor, which comprises the following steps: acquiring measurement data of the multi-mode sensor for temperature, vibration and air pressure in real time, and constructing a state observation vector at the current moment; generating an initial parameter matrix of multi-modal dynamic Kalman filtering by using a generative adversarial network; correcting the state observation vector at the current moment based on improved multi-modal dynamic Kalman filtering to obtain a target state vector; and calculating an environment score according to the state value of each modal sensor in the target state vector at the current moment, evaluating the high-speed rail environment at the current moment, and taking corresponding response measures. According to the method, multi-modal sensor data can be integrated in a complex scene of high-speed running of the high-speed rail, quantified environment scores are output in real time, the running risk level is visually reflected, a reliable basis is provided for train dispatching, fault pre-judgment and emergency decision making, and the running safety and the operation and maintenance efficiency of the high-speed rail are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things, in particular to a high-speed rail environment monitoring method based on a multi-modal sensor. BACKGROUND

[0002] With the continuous improvement of high-speed rail running speed (part of the line speed exceeds 350 km / h), higher requirements are put forward for the real-time, accuracy and reliability of the environment monitoring system. Single sensor is limited by physical principle, has defects such as large blind area, single information dimension and weak anti-interference ability, and is difficult to meet the multi-scene monitoring demand in complex environment. For example, in special sections such as tunnels and mountainous areas, single sensor is easily affected by factors such as light mutation, electromagnetic interference and shielding, resulting in missed detection and high false alarm rate, which cannot provide comprehensive and accurate environment information for train operation in time, seriously threatening the safety of high-speed rail operation.

[0003] In order to overcome the shortcomings of single sensor, multi-modal sensor technology emerges as the times require. This technology integrates multiple types of sensors to realize multi-dimensional data acquisition of high-speed rail running environment, and to a certain extent, improves the comprehensiveness of environment monitoring.

[0004] However, in the complex scene of high-speed running, high vibration and multiple noises of high-speed rail, the data fusion of multi-modal sensor still has the following obvious defects:

[0005] (1) Data redundancy and noise interference: such as patent CN2021203648524 proposes to reduce interference by physically isolating signal lines (independent cable transmission of temperature and vibration signals), but does not solve the redundancy problem when multiple sensor data is fused. Temperature and vibration data have strong correlation in time series, and independent processing leads to information overlap, and high-frequency vibration noise will still affect the accuracy of temperature data through sensor coupling. Although the literature "Multi-source data fusion method based on radial basis function generative adversarial network" (Hu Dengfeng et al., Acta Aeronautica et Astronautica 2025) fuses data by using generative adversarial network, it does not design a noise suppression mechanism for dynamic environment, resulting in that the fusion result is easily disturbed by sensor instantaneous abnormal value;

[0006] (2) Insufficient real-time performance: traditional fusion algorithms such as weighted average and neural network require complex iterative calculation, for example, the literature "Distributed data fusion management system under Internet of Things platform" (Zhang Hui et al., 2024) uses serial ETL tools for data cleaning and conversion, introduces at least 20 ms delay, which is difficult to meet the millisecond-level early warning demand of high-speed rail, and patent CN2020108314746 (high-speed rail power socket monitoring system) relies on server-side centralized calculation, the data transmission and processing link is too long, which further aggravates the delay.

[0007] (3) Multi-modal data coordination is poor: The literature "Distributed data fusion management system under Internet of Things platform" integrates heterogeneous data through data format standardization, but lacks a dynamic weight distribution mechanism. When the vibration sensor fails due to impact, the system cannot automatically increase the dynamic fusion weight of temperature data, resulting in a large error in environmental score.

[0008] In summary, the existing multi-modal sensor for high-speed rail environment monitoring still has problems such as data redundancy, insufficient data processing real-time performance, and poor multi-modal data coordination in data fusion, which easily leads to environmental state misjudgment and cannot provide timely and accurate safety warning for train operation. SUMMARY

[0009] To this end, the technical problem to be solved by the present application is to overcome the problem of inaccurate environmental monitoring caused by sensor data redundancy, insufficient processing real-time performance, and poor coordination in the high-speed running environment of high-speed rail in the prior art.

[0010] To solve the above technical problems, the present application provides a high-speed rail environment monitoring method based on multi-modal sensors, comprising:

[0011] Real-time acquisition of measurement data of multi-modal sensors for temperature, vibration and air pressure, and construction of a state observation vector at the current time;

[0012] Based on the improved multi-modal dynamic Kalman filter, the state observation vector at the current time is corrected to obtain a target state vector, including:

[0013] According to the state transition matrix and the state prediction vector at the last time, the state prediction vector at the current time is obtained;

[0014] According to the state transition matrix and the noise covariance matrix and the posterior error covariance matrix at the last time, the prior error covariance matrix at the current time is updated;

[0015] According to the observation residual matrix at the current time, the noise covariance matrix at the last time and the prior error covariance matrix at the current time, the noise covariance matrix at the current time is obtained; the Kalman gain matrix at the current time is calculated according to the prior error covariance matrix at the current time and the noise covariance matrix;

[0016] According to the state prediction vector at the current time, the state observation vector, the observation residual matrix, the prior error covariance matrix and the initial noise covariance matrix, the health state value of each modal sensor at the current time is calculated by Mahalanobis distance; the dynamic fusion weight of each modal sensor at the current time is calculated according to the health state value at the current time and the observation variance;

[0017] The state prediction vector at the current time is corrected according to the Kalman gain matrix at the current time, the observation residual matrix and the dynamic fusion weight of each modal sensor, and a target state vector at the current time is obtained.

[0018] An environment score is calculated according to the state value of each modal sensor in the target state vector at the current time, and the high-speed rail environment at the current time is evaluated.

[0019] Preferably, when the probe offset Δd detected by the displacement sensor in the multi-modal sensor is greater than 0.2 mm, the state observation vector X at the current time is compensated by the probe offset Δd, and the compensated state observation vector is X corrected = X + 0.8·Δd.

[0020] Preferably, the state transition matrix and the initial noise covariance matrix of the multi-modal dynamic Kalman filter are generated by using a generative adversarial network, comprising:

[0021] The original measurement values of each modal of each sample in the multi-modal sensor historical data set are generated by using a generative adversarial network to generate vibration noise;

[0022] The state transition matrix and the initial noise covariance matrix are calculated according to the original measurement values of each modal of each sample in the multi-modal sensor historical data set and the corresponding vibration noise.

[0023] Preferably, the state prediction vector at the current time is obtained according to the state transition matrix and the state prediction vector at the previous time, and the formula is:

[0024]

[0025] wherein, is the state prediction vector at the k time, is the state prediction vector at the k-1 time, Φ is the state transition matrix, and Γ k is the control input matrix at the k time, u k is the vibration control amount at the k time.

[0026] Preferably, the prior error covariance matrix at the current time is updated according to the state transition matrix and the noise covariance matrix and the posterior error covariance matrix at the previous time, and the formula is:

[0027]

[0028] wherein, is the prior error covariance matrix at the k time, Φ is the state transition matrix, is the posterior error covariance matrix at the k-1 time, R k-1 is the noise covariance matrix at the k-1 time.

[0029] Preferably, the noise covariance matrix at the current time is obtained based on the observation residual matrix at the current time, the noise covariance matrix at the previous time, and the prior error covariance matrix at the current time, using the following formula:

[0030]

[0031] Among them, R k and R k-1 Let be the noise covariance matrices at time k and time k-1, respectively, and α be the forgetting factor. Let X be the observation residual matrix at time k. k Let k be the state observation vector at time k. Let H be the state prediction vector at time k, and let H be the identity matrix. Let be the prior error covariance matrix at time k;

[0032] The Kalman gain matrix at the current time is calculated based on the prior error covariance matrix and the noise covariance matrix at the current time, using the following formula:

[0033]

[0034] in, Let be the Kalman gain matrix at time k.

[0035] Preferably, the health state value of each modal sensor is calculated using Mahalanobis distance based on the current state prediction vector, state observation vector, prior error covariance matrix, and initial noise covariance matrix, as follows:

[0036]

[0037] Where, d k,j Let X be the health status value of the j-th modal sensor at time k. k,j H is the j-th value of the state observation vector at time k. j Let j be the vector of the j-th row of the identity matrix. Let k be the state prediction vector at time k, ∈ k,j Let j be the row vector of the observation residual matrix at time k. Let R be the prior error covariance matrix at time k. opt,j Let j be the vector of the initial noise covariance matrix;

[0038] The dynamic fusion weights are calculated based on the current health status values ​​and observation variances of each modal sensor, using the following formula:

[0039]

[0040] Among them, w k,jis a dynamic fusion weight of the jth modal sensor at the kth moment, is an observation variance of the jth modal sensor at the kth moment.

[0041] Preferably, the state prediction vector at the current moment is corrected according to the Kalman gain matrix at the current moment, the observation error vector and the dynamic fusion weight of each modal sensor to obtain the target state vector at the current moment, and the formula is:

[0042]

[0043] wherein, is the target state vector at the kth moment, is the state prediction vector at the kth moment, w k,j is a dynamic fusion weight of the jth modal sensor at the kth moment, is the jth row vector of the Kalman gain matrix at the kth moment, ∈ k,j is the jth row vector of the observation residual matrix at the kth moment.

[0044] Preferably, the environmental score is calculated according to the state value of each modal sensor in the target state vector at the current moment, and the formula is:

[0045]

[0046] wherein, Score k is the environmental score at the kth moment, λ j is the score weight of the jth modal sensor, is the jth value of the target state vector at the kth moment, X ref is a standard state vector, ΔX max is the maximum allowable deviation.

[0047] Preferably, the FPGA is used to correct the state observation vector at the current moment based on the improved multi-modal dynamic Kalman filter to obtain the target state vector.

[0048] The above technical scheme of the present application has the following beneficial effects compared with the prior art:

[0049] The high-speed rail environment monitoring method based on a multi-modal sensor provided by the application improves the traditional Kalman filtering method, automatically identifies the spatiotemporal correlation of multi-modal sensor data such as temperature, vibration and air pressure, reduces the accumulation of redundant information, significantly improves the efficiency of data fusion, realizes real-time response and correction of multi-modal sensor data, and solves the problem of insufficient high-frequency noise suppression caused by the fixed noise covariance matrix of the traditional Kalman filtering method by updating the noise covariance matrix in real time through residual driving, effectively suppressing the interference of noise in the correction process.

[0050] Further, the application combines the generation ability of the generative adversarial network with the dynamic characteristics of the Kalman filter, generates high-quality noise simulation data by using the generative adversarial network to optimize the state transition matrix and the initial noise covariance matrix, so that the initial parameter matrix for Kalman filtering is more consistent with the real noise statistical characteristics, significantly reduces the interference of high-frequency vibration noise on temperature data, suppresses the pollution of multi-modal noise on the fusion result, and further improves the fusion accuracy of multi-modal data. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to make the content of the application more easily understood, the application will be further described in detail below according to specific embodiments of the application and in conjunction with the drawings, in which:

[0052] Figure 1 is a flowchart of the high-speed rail environment monitoring method based on a multi-modal sensor provided by the application;

[0053] Figure 2 is a flowchart of the generative adversarial network;

[0054] Figure 3 is a comparison diagram of the 100Hz vibration noise generated by the GAN in the example of the application and the noise template of the measured frequency spectrum of the Beijing-Shanghai high-speed rail;

[0055] Figure 4 is a comparison diagram of the noise suppression performance of the temperature noise spectrum of the MDKF and the traditional Kalman filter at 350km / h in the example of the application;

[0056] Figure 5is the delay distribution measured figure after the fusion algorithm of the GAN and the multimodal dynamic Kalman filter applied by the embodiment of the application is adopted, wherein Figure 5 (a) in the figure is a delay distribution probability density comparison chart, Figure 5 (b) in the figure is a delay distribution box plot comparison chart;

[0057] Figure 6 is a hardware cooperative flow chart of the high-speed rail environment monitoring method based on a multimodal sensor. DETAILED DESCRIPTION

[0058] The application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the application and implement it, but the embodiments are not limiting to the application.

[0059] Referring to Figure 1 , the application provides a high-speed rail environment monitoring method based on a multimodal sensor, comprising:

[0060] S1: Real-time acquisition of measurement data of the multimodal sensor for temperature, vibration and air pressure, and construction of a state observation vector at the current time.

[0061] S2: Generation of a state transition matrix and an initial noise covariance matrix of the improved multimodal dynamic Kalman filter (MDKF) of the application by using a generative adversarial network, comprising:

[0062] Generation of vibration noise for the original measurement value of each modality of each sample in the multimodal sensor historical data set by using the generative adversarial network;

[0063] Calculation of the state transition matrix and the initial noise covariance matrix according to the original measurement value of each modality of each sample in the multimodal sensor historical data set and the corresponding vibration noise.

[0064] Specifically, to break through the limitation of the traditional Kalman filter on the initial parameters, the embodiment proposes a pre-training optimization layer based on a generative adversarial network (GAN), establishes a physical model of high-speed rail vibration noise by a deep learning method, and optimizes the initial parameter matrix of the multimodal dynamic Kalman filter. The initial parameter matrix includes the state transition matrix and the initial noise covariance matrix.

[0065] The generator-discriminator adversarial framework is trained with the historical data set of the Beijing-Shanghai high-speed rail as an input source (containing vibration data with a characteristic frequency of 100 Hz). Referring to Figure 2As shown, the generator adopts a one-dimensional convolutional neural network (1D-CNN) structure to learn the time-domain characteristics of vibration noise caused by track irregularities and output vibration noise templates; the discriminator adopts a bidirectional long short-term memory network (Bi-LSTM) to identify real noise and synthesized noise through time sequence feature analysis. Both are continuously optimized through minimax game, and the formula can be expressed as:

[0066]

[0067] Wherein, G represents the generator, D represents the discriminator, G(z) represents the vibration noise output by the generator, x represents the sample, p data represents the distribution of real data, z represents a random noise vector, p z represents the prior distribution of noise.

[0068] After the generative adversarial network is trained and converged, the trained generative adversarial network is used to generate vibration noise G j,i (z j ) from the original measurement value y i of each sample of each mode in the multi-modal sensor historical data set.

[0069] Figure 3 Figure is a comparison diagram of the 100Hz vibration noise generated by the GAN in the embodiment of the present application and the noise template of the measured spectrum of the Beijing-Shanghai high-speed rail. As can be seen from the comparison of the 100Hz vibration noise generated by the GAN and the measured spectrum, the GAN captures the time-frequency domain coupling characteristics of the noise through adversarial training, so that the generated initial matrix is more consistent with the statistical characteristics of the real noise, thereby improving the filtering accuracy of the MDKF.

[0070] The generative adversarial network generates training data with stronger robustness in the extreme electromagnetic interference scene, ensures the continuous and stable operation of the monitoring system in the high-speed rail long-term high-frequency vibration and multi-noise environment, and greatly reduces the failure rate.

[0071] The initial noise covariance matrix is calculated according to the original measurement value y j,i of each sample of each mode in the multi-modal sensor historical data set and the corresponding vibration noise G j (z i ).

[0072] For a sensor with m modes, the initial noise covariance matrix R opt is an m x m symmetric matrix. The multi-modal sensor adopted in the embodiment is a three-modal sensor of temperature, vibration and air pressure, that is, m = 3, which is represented as:

[0073]

[0074] where T represents the temperature modality, V represents the vibration modality, and P represents the pressure modality.

[0075] The calculation formula of each element in the initial noise covariance matrix is represented as:

[0076]

[0077] where R opt (j, k) is the value of the jth row and kth element of the initial noise covariance matrix, N is the number of samples, y j,i is the original measurement value of the jth modality of the ith sample, G j (z i ) is the corresponding vibration noise of y j,i , y j',i is the original measurement value of the j'th modality of the ith sample, G j' (z i ) is the corresponding vibration noise of y j',i ; j = 1 represents temperature, j = 2 represents vibration, and j = 3 represents pressure.

[0078] For example, the noise covariance R TT of the temperature modality and the covariance R TV between the noise of the temperature and vibration modalities are represented as:

[0079]

[0080] According to the mean square error of the original measurement value y j,i of each modality of each sample in the historical data set of the multi-modality sensor and the corresponding vibration noise G j (z i ), the state transition matrix is calculated, including:

[0081] The state transition matrix Φ is used to describe the time evolution relationship between modalities, and its formula is represented as:

[0082]

[0083] The state transition relationship of each modality itself is represented by the autoregressive coefficient. Taking the vibration modality as an example, the calculation formula of φ VV is:

[0084]

[0085] The state transition relationship between modalities is represented by the cross-correlation coefficient. Taking the state transition relationship between the vibration modality and the temperature modality as an example, the calculation formula of φ TV is:

[0086]

[0087] wherein,

[0088] The application combines the generation ability of the generative adversarial network with the dynamic characteristics of the Kalman filter, generates high-quality noise simulation data by using the generative adversarial network to optimize the state transition matrix and the initial noise covariance matrix, so that the initial parameter matrix for the Kalman filter is more consistent with the real noise statistical characteristics, significantly reduces the interference of high-frequency vibration noise on the temperature data, suppresses the pollution of multi-modal noise on the fusion result, shortens the convergence time of subsequent multi-modal dynamic Kalman filter by 62%, solves the filter divergence problem caused by artificial setting of initial values in the traditional method, and further improves the fusion precision of multi-modal data.

[0089] S3: correcting the state observation vector at the current time based on the improved multi-modal dynamic Kalman filter to obtain a target state vector, comprising:

[0090] S31: obtaining a state prediction vector at the current time according to the state transition matrix and the state prediction vector at the previous time, and the formula is:

[0091]

[0092] wherein, is the state prediction vector at the k time, is the state prediction vector at the k-1 time, Φ is the state transition matrix, Γ k is the control input matrix at the k time, u k is the vibration control amount at the k time, which is calculated in real time by a three-axis acceleration sensor.

[0093] S32: updating the prior error covariance matrix at the current time according to the state transition matrix, the noise covariance matrix at the previous time and the posterior error covariance matrix at the previous time, and the formula is:

[0094]

[0095] wherein, is the prior error covariance matrix at the k time, which is used to represent the state prediction uncertainty; Φ is the state transition matrix, is the posterior error covariance matrix at the k-1 time, R k-1 is the noise covariance matrix at the k-1 time.

[0096] S33: obtaining the noise covariance matrix at the current time according to the observation residual matrix at the current time, the noise covariance matrix at the previous time and the prior error covariance matrix at the current time, and the formula is:

[0097]

[0098] wherein R k and R k-1 are noise covariance matrices at k time and k-1 time respectively, and α is a forgetting factor for ensuring fast adaptation to burst noise, and the embodiment takes a value of 0.95; is an observation residual matrix at k time, and X k is a state observation vector at k time, is a state prediction vector at k time, and H is a unit matrix, is a prior error covariance matrix at k time.

[0099] The improved multi-modal dynamic Kalman filter overcomes the defect of fixed noise covariance of the traditional Kalman filter by updating the noise covariance matrix at each time step.

[0100] The Kalman gain matrix at the current time is calculated according to the prior error covariance matrix and the noise covariance matrix at the current time, and the formula is:

[0101]

[0102] wherein, is the Kalman gain matrix at k time.

[0103] S34: According to the state prediction vector at the current time, the state observation vector, the observation residual matrix, the prior error covariance matrix and the initial noise covariance matrix, the health state value of each modal sensor at the current time is calculated by the Mahalanobis distance, and the formula is:

[0104]

[0105] wherein d k,j is the health state value of the jth modal sensor at k time, X k,j is the jth value of the state observation vector at k time, H j is the jth row vector of the unit matrix, is the state prediction vector at k time, and ∈ k,j is the jth row vector of the observation residual matrix at k time, is the prior error covariance matrix at k time, and R 0,j is the jth row vector of the initial noise covariance matrix;

[0106] The dynamic fusion weight corresponding to each modal sensor is calculated according to the health state value at the current time and the observation variance, and the formula is:

[0107]

[0108] wherein w k,jis a dynamic fusion weight of the jth modal sensor at the kth moment, is an observation variance of the jth modal sensor at the kth moment.

[0109] S35: correcting the state prediction vector at the current moment according to the Kalman gain matrix at the current moment, the observation residual matrix and the dynamic fusion weight of each modal sensor, to obtain a target state vector at the current moment, and the formula is:

[0110]

[0111] wherein, is the target state vector at the kth moment, is the state prediction vector at the kth moment, w k,j is a dynamic fusion weight of the jth modal sensor at the kth moment, is the jth row vector of the Kalman gain matrix at the kth moment, and k,j is the jth row vector of the observation residual matrix at the kth moment.

[0112] The improved multi-modal dynamic Kalman filtering algorithm introduces a double mechanism of noise covariance adaptive update and fusion weight dynamic adjustment, and realizes the fusion processing of the three-dimensional state vectors of temperature, vibration and air pressure.

[0113] Figure 4 is a noise suppression performance comparison chart of temperature noise spectrum of MDKF and traditional Kalman filtering at 350km / h in the examples of the application, and from the chart, it can be seen that the application (red solid line) presents a significant depression at the full frequency band, especially at the 100Hz / 300Hz characteristic peak, and the measured temperature noise is reduced from ±1.2℃ to ±0.3℃, compared with the traditional Kalman filtering (blue dotted line).

[0114] Preferably, the embodiment adopts FPGA (Field-Programmable Gate Array) to correct the state observation vector at the current moment based on the improved multi-modal dynamic Kalman filtering to obtain the target state vector. Wherein, each sensor channel is allocated an independent MDKF calculation core (16 channels in parallel).

[0115] The embodiment optimizes the calculation process through the FPGA hardware acceleration architecture, compresses the fusion delay to within 3ms, and supports multi-sensor parallel processing in combination with the distributed computing framework, reduces the data transmission link length, and completely solves the delay accumulation problem caused by centralized computing. This improvement enables the system to realize real-time data fusion and abnormal early warning in the high-speed running environment of high-speed rail, and greatly improves the response speed of train operation safety.

[0116] Figure 5is a measured graph of delay distribution after the fusion algorithm of the embodiment of the application applies GAN and multi-modal dynamic Kalman filtering using FPGA, wherein Figure 5 (a) in Figure is a delay distribution probability density comparison chart, Figure 5 (b) in Figure is a delay distribution box plot comparison chart. As can be seen from the figure, the traditional scheme has large delay fluctuations, and the fusion algorithm of the embodiment effectively reduces the calculation delay.

[0117] Preferably, to solve the problem of data distortion in the extreme environment of high-speed rail, the embodiment designs a hardware compensation mechanism at the physical layer, which suppresses the influence of vibration and temperature drift on the sensor through mechanical structure innovation, and the specific embodiment is as follows:

[0118] The first level protection strategy is used for vibration energy absorption and thermal management. The embodiment uses 6061 aluminum alloy heat sink and temperature feedback circuit to form a sealed heat dissipation module, which dynamically adjusts the heat dissipation power of the multi-modal sensor according to the real-time temperature:

[0119]

[0120] wherein, P cool is the heat dissipation power, T is the real-time temperature of the sensor, when the temperature exceeds the critical value of 40℃, the heat dissipation power is increased to 7.5W, to ensure the thermal stability of the sensor in the extreme environment of-40℃ to 85℃.

[0121] The second level protection strategy is used for displacement-algorithm collaborative compensation. When the displacement sensor in the multi-modal sensor detects that the probe offset Δd>0.2mm (caused by high-frequency vibration or mechanical fatigue), the current state observation vector X is compensated by using the probe offset Δd, and the compensated state observation vector is X corrected = X + 0.8·Δd.

[0122] Based on the problem that the traditional rigid fixed structure is easy to cause heat dissipation failure and sensor displacement under high-frequency vibration of high-speed rail, the embodiment uses a damping spring suspension design instead of rigid connection, which reduces the probe displacement through elastic buffering, and avoids the heat sink from falling off due to vibration. The sealed heat dissipation module combines with the algorithm to dynamically monitor the temperature data, intelligently adjusts the heat dissipation power, solves the contradiction between impact resistance and heat dissipation, prolongs the hardware life by more than 2 times, and significantly reduces the maintenance cost.

[0123] S4: According to the state value of each modal sensor in the target state vector at the current time, the environment score is calculated, the high-speed rail environment at the current time is evaluated, and corresponding response measures are taken.

[0124] According to the state value of each modal sensor in the target state vector at the current time, the environment score is calculated, and the formula is:

[0125]

[0126] wherein Score k is the environment score at time k, λ j is the score weight of the jth modal sensor, is the jth value of the target state vector at time k, X ref is the standard state vector designed according to the high-speed rail safety standard, ΔX max is the maximum allowable deviation.

[0127] The score weight of the embodiment is λ = [0.4, 0.3, 0.3].

[0128] A three-level safety response mechanism is triggered according to the score result, as shown in Table 1.

[0129] Table 1: Three-level safety response mechanism

[0130] Score interval Corresponding measure Execution delay Physical meaning 85-100 Normal monitoring - Each parameter fluctuation within the safety threshold 70-84 Acoustic and light early warning < 500 ms Single parameter continuous abnormality <70 Trigger emergency braking <200 ms Multiple parameters deteriorate in coordination

[0131] Figure 6 is a hardware cooperative flowchart of a high-speed rail environment monitoring method based on a multi-modal sensor according to the present application.

[0132] In summary, the high-speed rail environment monitoring method based on a multi-modal sensor according to the present application improves the traditional Kalman filtering method, automatically identifies the spatio-temporal correlation of multi-modal sensor data such as temperature, vibration, and air pressure, reduces the accumulation of redundant information, significantly improves the efficiency of data fusion, realizes real-time response and correction of multi-modal sensor data, and solves the problem of insufficient high-frequency noise suppression caused by the fixed noise covariance matrix of the traditional Kalman filtering method by updating the noise covariance matrix in real time driven by the residual error. In the correction process, the interference of noise is effectively suppressed. At the same time, the dynamic fusion weight of each modal sensor is calculated in real time based on the Mahalanobis distance, so that when a certain modal sensor fails due to impact, the fusion weight of other modal sensor data can be automatically increased, effectively reducing the deviation of data fusion, improving the synergy of multi-modal sensor data fusion, and significantly improving the stability of subsequent calculation of environment score. Therefore, the present application can quickly integrate multi-modal sensor data in the complex scene of high-speed rail operation, output quantitative environment score in real time, intuitively reflect the running risk level, provide reliable basis for train scheduling, fault prediction, and emergency decision-making, and significantly enhance the safety and operation efficiency of high-speed rail operation.

[0133] Further, the application combines the generation capability of the generative adversarial network with the dynamic characteristics of the Kalman filter, generates high-quality noise simulation data by using the generative adversarial network to optimize the state transition matrix and the initial noise covariance matrix, so that the initial parameter matrix for the Kalman filter is more in line with the real noise statistical characteristics, the interference of high-frequency vibration noise on the temperature data is significantly reduced, the pollution of the multi-modal noise on the fusion result is inhibited, and the fusion precision of the multi-modal data is further improved.

[0134] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0135] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0136] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 functions specified in the flowchart

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 functions specified in the flowchart

[0138] Obviously, the above embodiments are merely example for clearly illustrating, and are not limitation to the embodiments. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments are not required to be enumerated, and the obvious changes or variations derived from the above are still within the protection scope of the present application.

Claims

1. A method for high-speed rail environmental monitoring based on multimodal sensors, characterized in that, include: Real-time acquisition of temperature, vibration, and air pressure measurement data from multimodal sensors to construct the current state observation vector; The target state vector is obtained by correcting the current state observation vector using an improved multimodal dynamic Kalman filter, including: The current state prediction vector is obtained from the state transition matrix and the state prediction vector of the previous time step. Based on the state transition matrix, the noise covariance matrix and the posterior error covariance matrix of the previous time step, the prior error covariance matrix of the current time step is updated. The noise covariance matrix at the current time is obtained from the observation residual matrix at the current time, the noise covariance matrix at the previous time, and the prior error covariance matrix at the current time; the Kalman gain matrix at the current time is calculated from the prior error covariance matrix and the noise covariance matrix at the current time. Based on the current state prediction vector, state observation vector, observation residual matrix, prior error covariance matrix, and initial noise covariance matrix, calculate the current health status value of each modal sensor using Mahalanobis distance; calculate the corresponding dynamic fusion weight based on the current health status value and observation variance of each modal sensor. The current state prediction vector is corrected based on the Kalman gain matrix, observation residual matrix and dynamic fusion weights of each modal sensor at the current time to obtain the target state vector at the current time. The environmental score is calculated based on the state values ​​of each modal sensor in the target state vector at the current moment, and the high-speed rail environment at the current moment is evaluated.

2. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, When the probe offset Δd detected by the displacement sensor in the multimodal sensor is greater than 0.2 mm, the current state observation vector X is compensated using the probe offset Δd, resulting in the compensated state observation vector X. corrected =X + 0.8·Δd.

3. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, Generative adversarial networks are used to generate the state transition matrix and initial noise covariance matrix for multimodal dynamic Kalman filtering, including: Generative adversarial networks are used to generate vibration noise from the raw measurements of each mode of each sample in the historical dataset of a multimodal sensor. The state transition matrix and initial noise covariance matrix are calculated based on the original measurement values ​​of each mode of each sample in the historical dataset of the multimodal sensor and their corresponding vibration noise.

4. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, The current state prediction vector is obtained from the state transition matrix and the state prediction vector from the previous time step, using the following formula: in, Let k be the state prediction vector at time k. Let Φ be the state prediction vector at time k-1, Φ be the state transition matrix, and Γ be the state prediction vector at time k-1. k Let u be the control input matrix at time k. k Let be the vibration control quantity at time k.

5. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1 is characterized in that, Based on the state transition matrix, the noise covariance matrix of the previous time step, and the posterior error covariance matrix, the prior error covariance matrix of the current time step is updated as follows: in, Let Φ be the prior error covariance matrix at time k, and Φ be the state transition matrix. Let R be the posterior error covariance matrix at time k-1. k-1 Let be the noise covariance matrix at time k-1.

6. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, The noise covariance matrix at the current time is obtained from the observation residual matrix at the current time, the noise covariance matrix at the previous time, and the prior error covariance matrix at the current time. The formula is as follows: Among them, R k and R k-1 Let be the noise covariance matrices at time k and time k-1, respectively, and α be the forgetting factor. Let X be the observation residual matrix at time k. k Let k be the state observation vector at time k. Let H be the state prediction vector at time k, and let H be the identity matrix. Let be the prior error covariance matrix at time k; The Kalman gain matrix at the current time is calculated based on the prior error covariance matrix and the noise covariance matrix at the current time, using the following formula: in, Let be the Kalman gain matrix at time k.

7. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, Based on the current state prediction vector, state observation vector, prior error covariance matrix, and initial noise covariance matrix, the health state value of each modal sensor is calculated using Mahalanobis distance, as follows: Where, d k,j Let X be the health status value of the j-th modal sensor at time k. k,j H is the j-th value of the state observation vector at time k. j Let j be the vector of the j-th row of the identity matrix. Let k be the state prediction vector at time k, ∈ k,j Let j be the row vector of the observation residual matrix at time k. Let R be the prior error covariance matrix at time k. opt,j Let j be the vector of the initial noise covariance matrix; The dynamic fusion weights are calculated based on the current health status values ​​and observation variances of each modal sensor, using the following formula: Among them, w k,j Let J be the dynamic fusion weight of the j-th modal sensor at time k. Let be the observation variance of the j-th modal sensor at time k.

8. The high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, The current state prediction vector is corrected based on the Kalman gain matrix, observation error vector, and dynamic fusion weights of each modal sensor at the current moment to obtain the target state vector at the current moment, as shown in the formula: in, Let be the target state vector at time k. Let w be the state prediction vector at time k. k,j Let j be the dynamic fusion weight of the j-th modal sensor at time k. Let be the j-th row vector of the Kalman gain matrix at time k, ∈ k,j Let be the j-th row vector of the observation residual matrix at time k.

9. A high-speed rail environment monitoring method based on multimodal sensors according to claim 1, characterized in that, The environmental score is calculated based on the state values ​​of each modal sensor in the target state vector at the current moment, using the following formula: Among them, Score k Let λ be the environmental score at time k. j Let j be the scoring weight of the j-th modal sensor. Let X be the j-th value of the target state vector at time k. ref Let ΔX be the standard state vector. max This represents the maximum permissible deviation.

10. A high-speed rail environment monitoring method based on a multimodal sensor according to claim 1, characterized in that, An improved multimodal dynamic Kalman filter based on FPGA is used to correct the current state observation vector to obtain the target state vector.

Images