Railway signal monitoring data generation method and system

Abstract

The invention provides a railway signal monitoring data generation method and system, and relates to the technical field of railways, and the method comprises the steps: carrying out the time-space correlation fusion processing of obtained carrier frequency characteristics, response coding information, the receiving time of transponder signal data, train operation speed data and train positioning data, and generating a monitoring data record under a unified coordinate system; adopting a rule-based data compression algorithm to adaptively select a compression rule and a compression rate to perform data compression on the monitoring data record; packaging the compressed monitoring data record into a monitoring data file in a standard format; and the data management server decompresses and decodes the monitoring data file, cleans, classifies and indexes the data in the monitoring data file by adopting a MapReduce parallel computing framework, stores the structured monitoring data into a distributed database, and performs fragmentation and redundant backup management on the database.

Description

Technical Field

[0001] The present invention relates to railway technology, and in particular to a method and system for generating railway signal monitoring data. Background Art

[0002] In order to ensure the safety and efficiency of train operation, it is necessary to monitor railway signal equipment in real time to promptly discover and deal with hidden dangers of equipment failure. Traditional railway signal monitoring mainly relies on manual inspections and regular maintenance, which has problems such as low monitoring efficiency, delayed fault detection, and insufficient data utilization, making it difficult to meet the safety and operation needs of modern railways.

[0003] In recent years, new generation information technologies such as the Internet of Things and big data have been widely used in the railway field, providing new technical means and solutions for railway signal monitoring. Various sensors and data acquisition terminals are used to monitor the status and parameters of signal equipment online, and the monitoring data is transmitted to the data center in real time through wired or wireless communication networks. The monitoring data is then processed and mined through the big data analysis platform to achieve fault diagnosis, health assessment and predictive maintenance of signal equipment, greatly improving the intelligence level and operation and maintenance efficiency of railway signal monitoring.

[0004] However, railway signal monitoring data has the characteristics of many types, large scale and strong timeliness, which puts higher requirements on the generation, transmission, storage and processing of data. As a result, the existing railway signal monitoring system still has deficiencies in data quality, real-time performance, reliability and other aspects, and cannot fully realize the value of massive monitoring data. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for generating railway signal monitoring data, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provide a method for generating railway signal monitoring data, including:

[0008] The acquired track circuit carrier frequency signal data is preprocessed, and the time-frequency feature matrix of the track circuit carrier frequency signal data is extracted by short-time Fourier transform, and then the time-frequency feature matrix is ​​extracted by using a convolutional neural network to obtain the carrier frequency characteristics of the track circuit carrier frequency signal data; the transponder signal data is decoded, and the response coding information corresponding to the transponder signal data is extracted by a matched filtering method, and the receiving time of the transponder signal data is determined by a correlation peak search algorithm; the carrier frequency characteristics, response coding information, the receiving time of the transponder signal data, the train running speed data and the train positioning data are subjected to time-space correlation fusion processing, and the monitoring data records under a unified coordinate system are generated by data timestamp alignment and spatial position mapping;

[0009] Performing data verification and anomaly detection on the monitoring data record, if there is no problem with the data verification and anomaly detection, using a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data record; encapsulating the compressed monitoring data record into a monitoring data file in a standard format, the monitoring data file consists of a file header, a data body and a file footer, the file header contains metadata such as data source, generation time and format version, the data body adopts ProtocolBuffer encoding format, and the file footer adopts CRC32 algorithm to generate a checksum;

[0010] The data management server decompresses and decodes the monitoring data files, and uses the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, stores the structured monitoring data in the distributed database, and performs sharding and redundant backup management on the database.

[0011] In an optional embodiment,

[0012] Decoding the transponder signal data, extracting the response coding information corresponding to the transponder signal data by a matched filtering method, and determining the receiving time of the transponder signal data by a correlation peak search algorithm include:

[0013] Performing matched filtering on the transponder signal data, and extracting the response coding information in the transponder signal data by setting a filter that matches the transponder signal data;

[0014] Performing a correlation peak search on the response coding information, determining the correlation peak position of the response coding information by calculating the autocorrelation function of the response coding information, and determining the time point corresponding to the correlation peak position as the receiving time of the transponder signal data;

[0015] Generate decoded transponder information according to the response coding information and the receiving time, wherein the decoded transponder information includes a transponder identifier and a receiving time;

[0016] The steps of the matched filtering process include:

[0017] Constructing a filter matching the transponder signal data according to known parameters of the transponder signal data;

[0018] Filtering the transponder signal data using the filter to obtain a filtered transponder signal sequence;

[0019] Sampling and judging the filtered transponder signal sequence, judging the amplitude of each sampling point according to a preset response coding waveform, and obtaining response coding information;

[0020] The step of performing a correlation peak search on the response coding information comprises:

[0021] Performing an autocorrelation operation on the response coding information to obtain an autocorrelation function sequence;

[0022] Performing peak search on the autocorrelation function sequence to determine the peak position and peak size;

[0023] The point where the peak position is the largest and the peak value exceeds the preset peak value threshold is determined as the correlation peak, and the time of the sampling point corresponding to the correlation peak is determined as the receiving time;

[0024] The receiving time of the transponder signal data is determined according to the receiving time and the sampling period.

[0025] In an optional embodiment,

[0026] The carrier frequency characteristics, the response coding information, the receiving time of the transponder signal data, the train running speed data and the train positioning data are subjected to time-space correlation fusion processing, and the monitoring data records under the unified coordinate system are generated by aligning the data timestamps and mapping the spatial positions, including:

[0027] Synchronize the clock sources of multiple monitoring devices distributed in different geographical locations. The clock source synchronization is achieved by deploying a master clock and a slave clock of the IEEE 1588 time protocol in the monitoring devices, obtaining the time deviation of the slave clock relative to the master clock through communication between the slave clock and the master clock, and adjusting the slave clock according to the time deviation to achieve synchronization of the clock sources of each monitoring device;

[0028] Establish the mapping relationship between the railway geographic information system line coordinate system and the WGS-84 coordinate system, obtain the spatial coordinate sequence of the center line of the railway geographic information system line coordinate system, use the least squares method to fit the transformation model parameters from the railway geographic information system line coordinate system to the WGS-84 coordinate system, and convert the installation position of the monitoring equipment from the railway geographic information system line coordinate system to the WGS-84 coordinate system to obtain the geographic spatial position of the monitoring equipment;

[0029] The train positioning data in the WGS-84 coordinate system output by the train positioning equipment is matched with the railway line through the hidden Markov model matching algorithm to determine the mileage position of the train positioning data in the railway line.

[0030] In an optional embodiment,

[0031] The method further comprises:

[0032] A quadtree is used to divide and index the two-dimensional geographic space of the monitoring data records. On each leaf node of the quadtree, a B+ tree index is used to organize the monitoring data records in the geographic space grid of the leaf node, and the key value of the B+ tree index is the timestamp of the monitoring data record.

[0033] For each piece of train positioning data, based on the spatiotemporal composite index of quadtree and B+ tree, track circuit monitoring data, balise monitoring data and train speed monitoring data within a preset time range before and after the timestamp of the train positioning data and within a preset distance range before and after the geographic location are retrieved;

[0034] The retrieved monitoring data within the same time and space range are clustered according to timestamps, and each clustered data set is subjected to association fusion processing. The association fusion processing normalizes the track circuit monitoring data according to carrier frequency type and monitoring parameters, and encodes the balise monitoring data according to information type, and encapsulates the normalized track circuit monitoring data, the encoded balise monitoring data, the train speed data and the train positioning data into a unified structured monitoring data record, wherein the monitoring data record contains a unique identifier, timestamp, spatial position, speed, track circuit status and balise information.

[0035] In an optional embodiment,

[0036] Adopting a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data records; packaging the compressed monitoring data records into a monitoring data file in a standard format includes:

[0037] Adopting a rule-based data compression algorithm, according to the data type and numerical distribution characteristics of the monitoring data record, adaptively selecting a compression rule matching the monitoring data record, setting a plurality of predefined compression ratios, adaptively compressing the monitoring data record, and obtaining a compressed monitoring data record;

[0038] Encapsulating the compressed monitoring data records into a monitoring data file in a standard format, wherein the monitoring data file consists of three parts: a file header, a data body, and a file footer;

[0039] The file header adopts JSON format and contains metadata fields such as data source, generation time, and format version;

[0040] Convert the compressed monitoring data record into ProtocolBuffer encoding format as the data body of the monitoring data file;

[0041] The CRC32 algorithm is used to calculate the file header and the data body to generate a check code, and the check code is used as the file tail of the monitoring data file.

[0042] In an optional embodiment,

[0043] Rule-based data compression algorithms include:

[0044] Divide the monitoring data records into blocks of fixed size, and compress each data block independently;

[0045] Initially selecting a compression rate from predefined compression rates according to data characteristics of the data block;

[0046] Compressing the data block using the selected compression rate, and counting the compressed data size and compression time;

[0047] Calculating a performance score of the compression rate, the performance score comprehensively considering the compression rate, the compression speed and the compression effect;

[0048] Compare the performance scores of multiple compression rates, select the compression rate with the best performance score as the compression rate of the next data block, and randomly explore other compression rates;

[0049] The steps of data decompression and decapsulation include:

[0050] Read the file header of the monitoring data file and parse the file header field;

[0051] Read the data body of the monitoring data file, and decode the data encoded by ProtocolBuffer into monitoring data records;

[0052] Read the file tail of the monitoring data file, extract the CRC32 check code, recalculate the check code of the file header and the data body, compare whether the two check codes are consistent, and verify the integrity of the file;

[0053] Decompress and restore the monitoring data record according to the compression algorithm and compression rule of the monitoring data record.

[0054] According to a second aspect of the embodiments of the present invention,

[0055] Provide railway signal monitoring data generation system, including:

[0056] The first unit is used to pre-process the acquired track circuit carrier frequency signal data, extract the time-frequency feature matrix of the track circuit carrier frequency signal data by short-time Fourier transform, and then use a convolutional neural network to extract features of the time-frequency feature matrix to obtain the carrier frequency features of the track circuit carrier frequency signal data; decode the transponder signal data, extract the response coding information corresponding to the transponder signal data by a matched filtering method, and determine the receiving time of the transponder signal data by a correlation peak search algorithm; perform time-space correlation fusion processing on the carrier frequency features, response coding information, receiving time of the transponder signal data, train speed data and train positioning data, and generate monitoring data records in a unified coordinate system by data timestamp alignment and spatial position mapping;

[0057] The second unit is used to perform data verification and anomaly detection on the monitoring data record. If there is no problem with the data verification and anomaly detection, a rule-based data compression algorithm is used to adaptively select compression rules and compression rates to compress the monitoring data record; the compressed monitoring data record is encapsulated into a monitoring data file in a standard format, and the monitoring data file consists of a file header, a data body, and a file footer. The file header contains metadata such as data source, generation time, and format version. The data body adopts the ProtocolBuffer encoding format, and the file footer adopts the CRC32 algorithm to generate a checksum.

[0058] The third unit is used for the data management server to decompress and decode the monitoring data files, and use the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, store the structured monitoring data in the distributed database, and perform sharding and redundant backup management on the database.

[0059] According to a third aspect of the embodiments of the present invention,

[0060] An electronic device is provided, comprising:

[0061] processor;

[0062] a memory for storing processor-executable instructions;

[0063] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0064] A fourth aspect of the embodiments of the present invention is:

[0065] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0066] This application adopts a method combining short-time Fourier transform and convolutional neural network, making full use of the time-frequency domain characteristics of the carrier signal, adaptively extracting the key features of the carrier signal, overcoming the problem of insufficient feature extraction capabilities of traditional methods, and providing high-quality feature data for subsequent state monitoring and fault diagnosis. Through matching filtering and correlation peak search algorithms, the coding information and receiving time of the transponder signal are accurately extracted, effectively suppressing noise and interference, improving the accuracy and real-time performance of transponder signal decoding, and providing reliable data support for train positioning and speed measurement.

[0067] Through data timestamp alignment and spatial position mapping, the monitoring data from multiple sources such as track circuits, balises, and train positioning are fused in a unified coordinate system to form a monitoring data record with consistent time and space and clear semantics, which is convenient for subsequent data analysis and mining, and improves the comprehensive utilization efficiency of monitoring data. Through data verification and anomaly detection, errors, omissions, anomalies and other quality problems in monitoring data are timely discovered and filtered, improving the accuracy and reliability of data; the use of adaptive data compression algorithms reduces the storage and transmission overhead of data while ensuring data quality; the use of standardized data encapsulation formats enhances the interpretability and interoperability of data, while protecting the integrity and security of data through verification codes.

[0068] By adopting the MapReduce parallel computing framework and distributed database, distributed storage and processing of monitoring data are realized, overcoming the performance bottleneck of traditional centralized data management; through data cleaning, classification and indexing, data organization and retrieval methods are optimized, and the efficiency of data query and analysis is improved; through data sharding and redundant backup, data load balancing and high availability are guaranteed, reducing the system's operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of a flow chart of a method for generating railway signal monitoring data according to an embodiment of the present invention;

[0070] Figure 2 The figure is a schematic diagram of the structure of a railway signal monitoring data generating system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0073] Figure 1 FIG. 1 is a flow chart of a method for generating railway signal monitoring data according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0074] S101. Preprocess the acquired track circuit carrier frequency signal data, extract the time-frequency feature matrix of the track circuit carrier frequency signal data by short-time Fourier transform, and then use convolutional neural network to extract features of the time-frequency feature matrix to obtain the carrier frequency features of the track circuit carrier frequency signal data; decode the transponder signal data, extract the response coding information corresponding to the transponder signal data by matched filtering, and determine the receiving time of the transponder signal data by correlation peak search algorithm; perform time-space correlation fusion processing on the carrier frequency features, response coding information, receiving time of the transponder signal data, train running speed data and train positioning data, and generate monitoring data records in a unified coordinate system by data timestamp alignment and spatial position mapping;

[0075] S102. Perform data verification and anomaly detection on the monitoring data record. If there is no problem with the data verification and anomaly detection, use a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data record; encapsulate the compressed monitoring data record into a monitoring data file in a standard format, wherein the monitoring data file consists of a file header, a data body, and a file footer. The file header contains metadata such as data source, generation time, and format version. The data body adopts the ProtocolBuffer encoding format, and the file footer adopts the CRC32 algorithm to generate a checksum.

[0076] S103. The data management server decompresses and decodes the monitoring data files, and uses the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, stores the structured monitoring data in the distributed database, and performs sharding and redundant backup management on the database.

[0077] This application adopts a method combining short-time Fourier transform and convolutional neural network, making full use of the time-frequency domain characteristics of the carrier signal, adaptively extracting the key features of the carrier signal, overcoming the problem of insufficient feature extraction capabilities of traditional methods, and providing high-quality feature data for subsequent state monitoring and fault diagnosis. Through matching filtering and correlation peak search algorithms, the coding information and receiving time of the transponder signal are accurately extracted, effectively suppressing noise and interference, improving the accuracy and real-time performance of transponder signal decoding, and providing reliable data support for train positioning and speed measurement.

[0078] Through data timestamp alignment and spatial position mapping, the monitoring data from multiple sources such as track circuits, balises, and train positioning are fused in a unified coordinate system to form a monitoring data record with consistent time and space and clear semantics, which is convenient for subsequent data analysis and mining, and improves the comprehensive utilization efficiency of monitoring data. Through data verification and anomaly detection, errors, omissions, anomalies and other quality problems in monitoring data are timely discovered and filtered, improving the accuracy and reliability of data; the use of adaptive data compression algorithms reduces the storage and transmission overhead of data while ensuring data quality; the use of standardized data encapsulation formats enhances the interpretability and interoperability of data, while protecting the integrity and security of data through verification codes.

[0079] By adopting the MapReduce parallel computing framework and distributed database, distributed storage and processing of monitoring data are realized, overcoming the performance bottleneck of traditional centralized data management; through data cleaning, classification and indexing, data organization and retrieval methods are optimized, and the efficiency of data query and analysis is improved; through data sharding and redundant backup, data load balancing and high availability are guaranteed, reducing the system's operation and maintenance costs.

[0080] In an optional embodiment,

[0081] Decoding the transponder signal data, extracting the response coding information corresponding to the transponder signal data by a matched filtering method, and determining the receiving time of the transponder signal data by a correlation peak search algorithm include:

[0082] Performing matched filtering on the transponder signal data, and extracting the response coding information in the transponder signal data by setting a filter that matches the transponder signal data;

[0083] Performing a correlation peak search on the response coding information, determining the correlation peak position of the response coding information by calculating the autocorrelation function of the response coding information, and determining the time point corresponding to the correlation peak position as the receiving time of the transponder signal data;

[0084] Generate decoded transponder information according to the response coding information and the receiving time, wherein the decoded transponder information includes a transponder identifier and a receiving time;

[0085] The steps of the matched filtering process include:

[0086] Constructing a filter matching the transponder signal data according to known parameters of the transponder signal data;

[0087] Filtering the transponder signal data using the filter to obtain a filtered transponder signal sequence;

[0088] Sampling and judging the filtered transponder signal sequence, judging the amplitude of each sampling point according to a preset response coding waveform, and obtaining response coding information;

[0089] The step of performing a correlation peak search on the response coding information comprises:

[0090] Performing an autocorrelation operation on the response coding information to obtain an autocorrelation function sequence;

[0091] Performing peak search on the autocorrelation function sequence to determine the peak position and peak size;

[0092] The point where the peak position is the largest and the peak value exceeds the preset peak value threshold is determined as the correlation peak, and the time of the sampling point corresponding to the correlation peak is determined as the receiving time;

[0093] The receiving time of the transponder signal data is determined according to the receiving time and the sampling period.

[0094] For example, in the railway signal monitoring system, the transponder is a key device for train positioning and speed measurement. By decoding the transponder signal data, the transponder's coding information and reception time can be obtained, providing important data support for subsequent train positioning and speed estimation.

[0095] Since the transponder signal is susceptible to noise and interference during transmission, resulting in a decrease in signal quality, the transponder signal data needs to be preprocessed before decoding. The main steps of preprocessing are as follows:

[0096] Signal filtering. A bandpass filter is used to filter the transponder signal to remove high-frequency noise and low-frequency interference components in the signal. The center frequency of the bandpass filter is set to the carrier frequency of the transponder signal, and the passband range is determined according to the bandwidth of the transponder signal. This solution uses a sixth-order Butterworth finite impulse response (FIR) filter, which has a flat frequency response and linear phase characteristics, and can effectively suppress noise.

[0097] Signal smoothing. In order to eliminate the burrs and spike noise in the signal, the moving average method is used to smooth the filtered signal. The moving average method performs local averaging of the signal through a sliding window. The window size is adjusted according to the smoothness of the signal, usually 2 to 5 times the carrier period.

[0098] Signal normalization. Due to the different installation locations and environmental conditions of different transponders, the received signal amplitudes vary greatly. In order to facilitate subsequent decoding processing, the signal amplitude needs to be normalized. The normalization method is to scale the signal amplitude range to the [0,1] interval.

[0099] The transponder uses Manchester encoding to encode the device ID, and the encoding information of each transponder is different. Manchester encoding uses the midpoint of the bit period as the jump point. The high level in the first half of the bit period indicates the code "1", and the high level in the second half of the bit period indicates the code "0".

[0100] The receiving time of the balise signal is an important parameter for train positioning and speed estimation. The accuracy of the receiving time directly affects the accuracy of positioning and speed estimation. This scheme uses a correlation peak search algorithm to determine the receiving time of the balise signal. The specific steps are as follows:

[0101] Calculate the cross-correlation function. The pre-processed transponder signal x norm The cross-correlation function R is obtained by performing cross-correlation operation with the standard coded signal s(t) of the transponder. xs (t):

[0102]

[0103] The cross-correlation function represents the correlation between the transponder signal and the standard coded signal at different time offsets. At the starting position of the transponder signal, the correlation is the largest, corresponding to the global maximum point of the cross-correlation function.

[0104] Search for the cross-correlation peak point. Perform a peak search on the cross-correlation function and find the global maximum point as the cross-correlation peak. In order to improve the accuracy of the peak search, the quadratic function interpolation method is used to perform local fitting on the cross-correlation function, and the position of the cross-correlation peak is accurately located by solving the extreme point of the quadratic function.

[0105] Calculate the signal reception time. According to the position of the cross-correlation peak tpea k , combined with the start time t of the standard coded signal of the transponder start , calculate the receiving time t of the transponder signal rx :

[0106] t rx =t peak -t start;

[0107] Among them, t rx The time base is the local clock of the transponder signal receiving device.

[0108] In order to ensure the reliability of the decoding results, it is necessary to verify and handle errors on the extracted transponder encoding information and receiving time. The main steps of verification are as follows:

[0109] Parity check. The coded information of the transponder usually uses parity check bits to verify the correctness of the coding. According to the parity check rules specified in the transponder protocol, the parity check bits of the coded information are calculated and compared with the parity check bits obtained by decoding. If the check bits are inconsistent, it means that there are errors in the coded information and error processing is required.

[0110] For coded information that fails verification, the following strategies are used for error handling:

[0111] (1) Retransmission strategy. If the transponder supports the retransmission mechanism, a retransmission request is sent to the transponder, asking it to resend the encoded information. The number of retransmissions is adaptively adjusted based on the transponder's retransmission limit and communication quality, usually 1 to 3 times.

[0112] (2) Error correction strategy. If the number of retransmissions reaches the upper limit and still fails to pass the verification, an error correction algorithm is used to correct the coded information. Commonly used error correction algorithms include Hamming code, convolutional code, and Reed-Solomon code. Select a suitable error correction algorithm according to the transponder protocol and design the corresponding codec.

[0113] (3) Discarding strategy: If the coded information after error correction still fails to pass the verification, the decoding is considered to have failed and the decoding result is discarded and not used as valid positioning and velocity measurement data.

[0114] Receiving time verification. For the successfully decoded transponder signal, it is also necessary to verify the rationality of its receiving time. The main indicators of the verification include:

[0115] (1) Time continuity. Check the difference between the reception time of the current signal and the previous reception time to determine whether it is within a reasonable range. If the time difference exceeds the preset threshold (such as greater than 2 times the transponder trigger interval), the time is considered discontinuous and marked as abnormal data.

[0116] (2) Time consistency. Compare the receiving time of the transponder signal with the time measurement value of other positioning equipment (such as GNSS) to determine whether it is within the allowable error range. If the time deviation exceeds the preset threshold (such as greater than 1 second), it is considered inconsistent and marked as abnormal data.

[0117] Through the above decoding, checking and error handling steps, the coded information and receiving time of the transponder can be reliably extracted from the transponder signal data, providing high-quality data input for subsequent train positioning and speed estimation. This scheme uses signal processing technologies such as matched filtering and correlation peak search to overcome the noise interference and distortion problems of the transponder signal and achieve accurate extraction of transponder information. Compared with the traditional decoding method based on threshold judgment, this scheme has stronger adaptability and robustness, and can meet the transponder signal decoding needs in complex environments.

[0118] In an optional embodiment,

[0119] The carrier frequency characteristics, the response coding information, the receiving time of the transponder signal data, the train running speed data and the train positioning data are subjected to time-space correlation fusion processing, and the monitoring data records under the unified coordinate system are generated by aligning the data timestamps and mapping the spatial positions, including:

[0120] Synchronize the clock sources of multiple monitoring devices distributed in different geographical locations. The clock source synchronization is achieved by deploying a master clock and a slave clock of the IEEE 1588 time protocol in the monitoring devices, obtaining the time deviation of the slave clock relative to the master clock through communication between the slave clock and the master clock, and adjusting the slave clock according to the time deviation to achieve synchronization of the clock sources of each monitoring device;

[0121] Establish the mapping relationship between the railway geographic information system line coordinate system and the WGS-84 coordinate system, obtain the spatial coordinate sequence of the center line of the railway geographic information system line coordinate system, use the least squares method to fit the transformation model parameters from the railway geographic information system line coordinate system to the WGS-84 coordinate system, and convert the installation position of the monitoring equipment from the railway geographic information system line coordinate system to the WGS-84 coordinate system to obtain the geographic spatial position of the monitoring equipment;

[0122] The train positioning data in the WGS-84 coordinate system output by the train positioning equipment is matched with the railway line through the hidden Markov model matching algorithm to determine the mileage position of the train positioning data in the railway line.

[0123] For example, since the monitoring devices in the railway signal monitoring system are distributed in different geographical locations and use independent clock sources, the monitoring data collected by different devices are inconsistent in timestamps. In order to achieve the spatiotemporal correlation of monitoring data, it is first necessary to synchronize the clocks of each monitoring device. This solution uses the IEEE 1588 Precision Time Protocol (PTP) to achieve time synchronization between monitoring devices. The specific steps are as follows:

[0124] The PTP master clock is deployed in the monitoring system as the time reference of the entire system. The PTP master clock receives timing signals from GPS / Beidou satellites and synchronizes with UTC (Coordinated Universal Time).

[0125] Each monitoring device acts as a PTP slave clock and communicates with the PTP master clock through the network. The PTP master clock sends Sync synchronization messages and Follow_Up follow-up messages to the slave clock. The slave clock calculates the time deviation of the slave clock relative to the master clock based on the message reception timestamp and the master clock sending timestamp included.

[0126] The slave clock adjusts its own clock according to the calculated time deviation to achieve synchronization with the master clock. The synchronization accuracy can reach the microsecond level, meeting the needs of monitoring data association and fusion.

[0127] In order to further improve the reliability of time synchronization, PTP time synchronization messages are periodically added during the data collection process of the monitoring equipment. Once the time deviation is found to exceed the threshold, the clock is adjusted in time to ensure the consistency of the monitoring data timestamp.

[0128] Most railway signal monitoring equipment uses the line coordinate system (LKS) to represent the installation location of the equipment and the spatial attributes of the monitoring data. The train positioning data uses the geographic coordinate system (WGS-84). In order to achieve the correlation and fusion of monitoring data in a unified coordinate system, it is necessary to convert between coordinate systems and map spatial positions. The specific steps are as follows:

[0129] The mapping relationship between the line coordinate system and the WGS-84 coordinate system is established. The spatial coordinate sequence of the line centerline is obtained through the railway GIS system, and then the least squares method is used to fit the transformation model from the line coordinate system to the WGS-84 coordinate system, including translation, rotation and scale transformation parameters.

[0130] For the monitoring equipment of track circuits and balises, the line coordinates of the equipment are converted to the WGS-84 coordinate system using the coordinate transformation model according to the installation mileage location of the equipment to obtain the geographic space location of the equipment.

[0131] For train positioning data, since the positioning device directly outputs WGS-84 coordinates, there is no need to perform coordinate transformation. However, the train positioning data needs to be matched with the railway line to determine the mileage position of the train on the line. A map matching algorithm based on the hidden Markov model (HMM) is used to comprehensively consider the train's position, speed, trajectory and other information to achieve accurate matching of train positioning data with the railway line.

[0132] After completing the coordinate conversion and map matching, the monitoring data of the track circuit, the balise, and the train positioning data are uniformly expressed in the WGS-84 coordinate system, laying the foundation for subsequent spatiotemporal association. At the same time, in order to facilitate the visualization and analysis of the data, the WGS-84 coordinates can be converted again to a plane coordinate system, such as the Gauss-Krüger projection or the UTM projection.

[0133] After achieving time synchronization and spatial mapping of monitoring data, it is necessary to further associate and fuse multi-source monitoring data to generate consistent and semantically rich monitoring data records. This scheme designs a data association and fusion method based on spatiotemporal indexing. The specific steps are as follows:

[0134] Design a spatiotemporal index structure for monitoring data. Use a quadtree to divide the two-dimensional space, and each leaf node represents a spatial grid. On each leaf node, use a B+ tree index to organize the monitoring data in the grid, and the key value of the B+ tree is the timestamp of the monitoring data. Through this spatiotemporal composite index, monitoring data within a specified time period and a specified spatial range can be efficiently retrieved.

[0135] For each train positioning data, track circuit monitoring data and balise monitoring data within a certain range before and after its timestamp (e.g. 5 seconds before and after) and a certain range of spatial position (e.g. 500 meters before and after) are retrieved. Through spatiotemporal indexing, traversal of all data can be avoided, thus improving retrieval efficiency.

[0136] The retrieved monitoring data within the same time and space range are clustered according to timestamps, and data with similar timestamps are aggregated into the same set.

[0137] For each cluster set, data association and fusion are performed. First, the track circuit monitoring data in the set is normalized according to the carrier frequency type, monitoring parameters, etc., and the balise monitoring data is encoded according to the information type. Then, the normalized track circuit monitoring data, the encoded balise monitoring data, the train speed data, and the train positioning data are encapsulated into a monitoring data record to form a structured data format. The data record contains key fields such as unique identification ID, timestamp, spatial position, speed, track circuit status, and balise information.

[0138] Perform data verification on the generated monitoring data records to check the integrity, consistency and logical rationality of the data. For data that does not meet the requirements, mark them as abnormal or discard them.

[0139] Finally, the verified monitoring data records are written into the monitoring database for subsequent data analysis and application. Before writing into the database, the data records can be compressed and encrypted to reduce data storage space and improve data security.

[0140] Through the above steps of time synchronization, space mapping, data fusion, etc., the consistent representation and associated integration of railway signal monitoring data in time and space dimensions are realized, forming a monitoring data record with complete semantics under a unified coordinate system, laying a data foundation for subsequent data analysis and mining. This scheme fully considers the spatiotemporal characteristics of railway signal monitoring data, designs an efficient spatiotemporal index structure and data fusion algorithm, improves the real-time and scalability of massive monitoring data processing, and is of great significance to improving railway operation safety, transportation efficiency, and operation and maintenance management.

[0141] In an optional embodiment,

[0142] The method further comprises:

[0143] A quadtree is used to divide and index the two-dimensional geographic space of the monitoring data records. On each leaf node of the quadtree, a B+ tree index is used to organize the monitoring data records in the geographic space grid of the leaf node, and the key value of the B+ tree index is the timestamp of the monitoring data record.

[0144] For each piece of train positioning data, based on the spatiotemporal composite index of quadtree and B+ tree, track circuit monitoring data, balise monitoring data and train speed monitoring data within a preset time range before and after the timestamp of the train positioning data and within a preset distance range before and after the geographic location are retrieved;

[0145] The retrieved monitoring data within the same time and space range are clustered according to timestamps, and each clustered data set is subjected to association fusion processing. The association fusion processing normalizes the track circuit monitoring data according to carrier frequency type and monitoring parameters, and encodes the balise monitoring data according to information type, and encapsulates the normalized track circuit monitoring data, the encoded balise monitoring data, the train speed data and the train positioning data into a unified structured monitoring data record, wherein the monitoring data record contains a unique identifier, timestamp, spatial position, speed, track circuit status and balise information.

[0146] For example, in the railway signal monitoring system, a large amount of monitoring data comes from various types of monitoring equipment distributed in different geographical locations, including track circuits, transponders, train positioning devices, etc. In order to achieve efficient storage, fast retrieval and correlation analysis of monitoring data, it is necessary to design a reasonable data organization and index structure.

[0147] Railway signal monitoring data has obvious geospatial attributes, and the installation locations of monitoring equipment are distributed in different mileage sections of the railway line. In order to support the spatial range retrieval of monitoring data, it is necessary to establish an efficient geospatial index. Considering the spatial distribution characteristics of railway lines, this scheme uses a quadtree to partition and index the two-dimensional geographic space of monitoring data. A quadtree is a hierarchical space partitioning tree, where each node represents a square geographic area, and each non-leaf node is evenly divided into four sub-areas, which are recursively divided until the spatial granularity of the leaf node reaches the set threshold.

[0148] The steps to construct a quadtree index are as follows:

[0149] (1) Determine the geographic spatial scope of the monitoring data, that is, the minimum enclosing rectangular area covering the locations of all monitoring devices.

[0150] (2) The rectangular area is used as the root node of the quadtree, and the root node is recursively divided into four equal parts until the spatial side length of the leaf node is less than or equal to the set granularity threshold (such as 100 meters). Each division is divided into four sub-areas: northeast (NE), southeast (SE), southwest (SW), and northwest (NW) according to the coordinates of the center point of the node.

[0151] (3) On a leaf node, maintain a pointer to the monitoring data within the geographic space of the node.

[0152] The query process of the quadtree index is as follows:

[0153] (1) Given a geographic spatial query range (such as a rectangular area or a circular area), starting from the root node, recursively determine whether the query range intersects with the spatial area of ​​the current node.

[0154] (2) If they do intersect, continue to recursively determine whether the query range intersects with the spatial areas of the four child nodes of the current node.

[0155] (3) If the current node is a leaf node, all monitoring data within the geographic space of the node are obtained through the pointer on the node.

[0156] (4) The monitoring data of all intersecting leaf nodes are merged to obtain the query result.

[0157] Railway signal monitoring data is a kind of time series data. Each piece of monitoring data has a timestamp attribute, which indicates the data collection time. In order to support the time range retrieval of monitoring data, it is necessary to build a B+ tree index with timestamp as the key value on each leaf node of the quadtree. B+ tree is a multi-way balanced search tree with the advantages of high query efficiency, high insertion and deletion efficiency, and high storage utilization. It is widely used in database indexes and file systems.

[0158] The steps to construct a B+ tree index are as follows:

[0159] (1) On each leaf node of the quadtree, a B+ tree index is constructed based on the timestamp of the monitoring data within the geographic space of the node.

[0160] (2) Each node of a B+ tree contains multiple key values ​​(timestamps) and pointers to child nodes. The size of the node is fixed, usually an integer multiple of the disk block.

[0161] (3) The leaf nodes of the B+ tree contain all key values, and are arranged in order of key value size. Each key value corresponds to a pointer to a data block, which stores all monitoring data records with that timestamp.

[0162] (4) Non-leaf nodes of the B+ tree do not store data, but only store key values ​​and pointers to child nodes to speed up the search of key values.

[0163] The query process of B+ tree index is as follows:

[0164] (1) Given a timestamp query range [t_start, t_end], starting from the root node of the B+ tree, recursively perform a binary search in the key values ​​of the nodes.

[0165] (2) If the key value found is equal to t_start, then starting from the data block pointer corresponding to the key value, the subsequent data blocks are scanned sequentially until the key value is greater than t_end.

[0166] (3) If the key value found is greater than t_start, continue to recursively search in the child nodes of the node where the key value is located.

[0167] (4) If the current node is a leaf node, all data blocks in the node whose key values ​​are in the range [t_start, t_end] are sequentially scanned.

[0168] (5) All monitoring data records in the data blocks that meet the timestamp range conditions are merged to obtain the query result.

[0169] Using the above-mentioned quadtree geospatial index and B+ tree timestamp index, the spatiotemporal correlation retrieval of monitoring data can be realized. The specific steps are as follows:

[0170] (1) Given a train location data, extract its timestamp t_train and geospatial coordinates (x_train, y_train).

[0171] (2) On the quadtree, based on the geographic spatial location of (x_train, y_train), retrieve the leaf node N_train where it is located, as well as the set of leaf nodes {N_neighbor} adjacent to N_train. The search range is a circular area with (x_train, y_train) as the center and a preset distance threshold (such as 50 meters) as the radius.

[0172] (3) On the B+ tree corresponding to N_train and {N_neighbor}, retrieve all monitoring data records with timestamps in the range of [t_train-T_interval, t_train+T_interval] according to the timestamp of t_train, where T_interval is the preset time threshold (e.g., 5 seconds).

[0173] (4) All the retrieved monitoring data records are grouped according to data type to obtain the track circuit monitoring data set {R_tc}, the balise monitoring data set {R_balise} and the train speed monitoring data set {R_speed} that are temporally and spatially associated with the train positioning data.

[0174] For each train positioning data, the monitoring data set {R_tc}, {R_balise} and {R_speed} obtained through spatiotemporal correlation retrieval needs to be further fused to generate a structured monitoring data record. The main steps of fusion are as follows:

[0175] (1) Data normalization. Since different types of monitoring data have different data formats and value ranges, they need to be normalized to convert them into a unified representation.

[0176] For track circuit monitoring data, extract the carrier frequency type and monitoring parameters (such as level amplitude, phase, etc.), and map the monitoring value to the interval [0,1] according to the carrier frequency type and parameter type. For transponder monitoring data, extract the transponder code and receiving time, encode the code according to the information type (such as position, speed limit, etc.), and generate a fixed-length binary code string.

[0177] (2) Data clustering. In {R_tc}, {R_balise}, and {R_speed}, there may be multiple monitoring data records with similar timestamps, which need to be clustered to merge them into one record.

[0178] The density-based DBSCAN clustering algorithm is used to cluster records whose timestamp difference is less than a preset threshold (such as 0.1 seconds) into one category. For each category of records, the median of the timestamp is taken as the timestamp of the category, and the average of the normalized monitoring values ​​is taken as the monitoring value of the category.

[0179] (3) Data encapsulation: The clustered track circuit monitoring data, balise monitoring data, train speed data and train positioning data are encapsulated into a structured monitoring data record.

[0180] The monitoring data record contains the following fields: unique identifier, timestamp, spatial position, train speed, track circuit status, balise code, etc. The unique identifier is generated by the timestamp and train number of the train positioning data, and is used to identify the uniqueness of the record. The timestamp is the median of the timestamps after all monitoring data are clustered. The spatial position is the coordinate value of the train positioning data. The train speed is the average value after the speed monitoring data is clustered. The track circuit status is composed of the normalized monitoring values ​​after the track circuit monitoring data is clustered, and the monitoring values ​​of different carrier frequency types are separated by separators. The balise code is composed of the code string after the balise monitoring data is clustered.

[0181] The fused monitoring data records need to be persistently stored for subsequent query analysis and data mining. This solution uses the distributed NoSQL database MongoDB to store monitoring data records. MongoDB supports flexible data models, high scalability, high availability and other features, and is suitable for the storage and management of massive unstructured data. Monitoring data records are stored in MongoDB in the form of JSON format documents, and each record corresponds to a document. Create a collection named "rail_signal_data" to store all monitoring data records. In order to optimize the query efficiency of geographic spatial locations, a 2dsphere spatial index is established on the "rail_signal_data" collection, with the index field being "location" and the index type being GeoJSON. At the same time, a B-tree index is established on the "timestamp" field to speed up range queries of timestamps.

[0182] For the monitoring data records stored in MongoDB, the following commonly used query interfaces are designed:

[0183] (1) Timestamp range query: Based on the input start timestamp and end timestamp, query all monitoring data records within the range.

[0184] (2) Geographical space range query: Based on the input geographic space range (such as circular area, rectangular area), query all monitoring data records within the range.

[0185] (3) Condition combination query. Based on the input of multiple query conditions (such as timestamp range, geographic space range, track circuit status, train speed range, etc.), query the monitoring data records that meet all the conditions.

[0186] In order to ensure the reliability and security of monitoring data, it is necessary to back up and archive the data regularly. The backup strategy is as follows:

[0187] (1) Perform a full backup of the MongoDB database every morning. The backup files are stored on an independent backup server and the backup files for the last 7 days are retained.

[0188] (2) Perform an incremental backup of the MongoDB database every hour. The backup files are stored on an independent backup server, and the backup files for the last 24 hours are retained.

[0189] The archiving strategy is as follows:

[0190] (1) Every month, the monitoring data records from one month ago are exported from the MongoDB database, converted into CSV format files, and stored on a separate archive server.

[0191] (2) On the archiving server, the monitoring data files in CSV format are compressed and encrypted, and the corresponding metadata files are generated to record the time range, data volume and other information of the data.

[0192] (3) Classify and catalog archived data files to facilitate subsequent query and analysis.

[0193] In an optional embodiment,

[0194] Adopting a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data records; packaging the compressed monitoring data records into a monitoring data file in a standard format includes:

[0195] Adopting a rule-based data compression algorithm, according to the data type and numerical distribution characteristics of the monitoring data record, adaptively selecting a compression rule matching the monitoring data record, setting a plurality of predefined compression ratios, adaptively compressing the monitoring data record, and obtaining a compressed monitoring data record;

[0196] Encapsulating the compressed monitoring data records into a monitoring data file in a standard format, wherein the monitoring data file consists of three parts: a file header, a data body, and a file footer;

[0197] The file header adopts JSON format and contains metadata fields such as data source, generation time, and format version;

[0198] Convert the compressed monitoring data record into ProtocolBuffer encoding format as the data body of the monitoring data file;

[0199] The CRC32 algorithm is used to calculate the file header and the data body to generate a check code, and the check code is used as the file tail of the monitoring data file.

[0200] In an optional embodiment,

[0201] Rule-based data compression algorithms include:

[0202] Divide the monitoring data records into blocks of fixed size, and compress each data block independently;

[0203] Initially selecting a compression rate from predefined compression rates according to data characteristics of the data block;

[0204] Compressing the data block using the selected compression rate, and counting the compressed data size and compression time;

[0205] Calculating a performance score of the compression rate, the performance score comprehensively considering the compression rate, the compression speed and the compression effect;

[0206] Compare the performance scores of multiple compression rates, select the compression rate with the best performance score as the compression rate of the next data block, and randomly explore other compression rates;

[0207] The steps of data decompression and decapsulation include:

[0208] Read the file header of the monitoring data file and parse the file header field;

[0209] Read the data body of the monitoring data file, and decode the data encoded by ProtocolBuffer into monitoring data records;

[0210] Read the file tail of the monitoring data file, extract the CRC32 check code, recalculate the check code of the file header and the data body, compare whether the two check codes are consistent, and verify the integrity of the file;

[0211] Decompress and restore the monitoring data record according to the compression algorithm and compression rule of the monitoring data record.

[0212] For example,

[0213] In the railway signal monitoring system, massive monitoring data records need to be stored and transmitted for a long time. In order to reduce data storage space and transmission bandwidth and improve data processing efficiency, the monitoring data needs to be compressed.

[0214] Monitoring data records contain various types of data, such as timestamps, geographic location coordinates, track circuit status, transponder codes, etc. Different types of data have different numerical distribution characteristics and redundancy characteristics, and compression rules need to be designed specifically. This scheme adopts a rule-based data compression algorithm, designs multiple compression rules according to data type and numerical distribution, and dynamically selects the optimal compression rule and compression rate through an adaptive selection algorithm.

[0215] The design principles of compression rules are as follows:

[0216] (1) Timestamp compression rules. The timestamps of monitoring data records are monotonically increasing, and the difference between the timestamps of adjacent records is usually small. Delta coding can be used to take the difference between the timestamp of each record and the timestamp of the previous record as the coding value, and then entropy coding (such as Huffman coding, arithmetic coding, etc.) is performed on the difference.

[0217] (2) Geographic location compression rules. The geographic location coordinates of monitoring data records usually change little. Predictive coding can be used to predict the coordinate value of the current record based on the coordinate values ​​of the previous records. The difference between the predicted value and the actual value is used as the encoding value, and the difference is then quantized and entropy encoded.

[0218] (3) Track circuit state compression rules. Track circuit states are composed of multiple discrete state values, and the frequencies of occurrence of different states vary greatly. Dictionary coding can be used to build a dictionary based on the frequency of occurrence of state values, and use the index of the state value in the dictionary as the encoding value.

[0219] (4) Transponder code compression rules. Transponder codes are composed of fixed-length binary code strings, and the bit positions at different locations have different change frequencies. Run-length coding can be used to represent consecutive identical bits with their values ​​and lengths, and then entropy coding is performed on the run length.

[0220] Different compression rules have different compression effects on different types and distributions of data. A fixed compression rate may result in poor compression effect or too long compression time. In order to balance the compression rate and compression effect, this scheme adopts an adaptive compression rate selection algorithm to dynamically adjust the compression rate according to the characteristics of the data and the compression effect. The steps of the adaptive compression rate selection algorithm are as follows:

[0221] (1) Data segmentation: The monitoring data records to be compressed are segmented into blocks of fixed size (e.g., 1MB), and each data block is compressed and decompressed independently.

[0222] (2) Compression rate initialization: Set a set of predefined compression rates (such as 1x, 2x, 4x, 8x), and randomly select a compression rate for each data block at the beginning.

[0223] (3) Data compression: For each data block, compress it using the currently selected compression rate and compression rule, and record the compressed data size and compression time.

[0224] (4) Compression rate evaluation. Calculate the performance score of the current compression rate based on the compressed data size and compression time. The performance score takes into account three factors: compression rate, compression speed, and compression effect, and is calculated using a weighted average method.

[0225] (5) Compression ratio adjustment. Based on the performance score of the current compression ratio, compare it with the performance scores of other compression ratios and select the compression ratio with the highest performance score as the compression ratio of the next data block. At the same time, randomly select other compression ratios with a certain probability (such as 10%) to explore a better compression ratio.

[0226] (6) Iterative compression: Repeat steps (3)-(5) until all data blocks are compressed.

[0227] Through the adaptive compression rate selection algorithm, the compression rate can be continuously optimized during the data compression process, while ensuring the compression effect, the compression speed can be improved and the compression time can be reduced.

[0228] The compressed monitoring data records are encapsulated into a monitoring data file in a standard format to facilitate data transmission, storage and analysis. The monitoring data file format designed in this scheme is as follows:

[0229] (1) File header. The file header contains metadata about the data file, such as data source, generation time, format version, etc., which is used to describe the properties and structure of the data file. The file header uses JSON format for easy parsing and expansion.

[0230] (2) Data body. The data body consists of compressed monitoring data records, using the ProtocolBuffer encoding format, corresponding to the structured definition of the monitoring data records. ProtocolBuffer is an efficient data serialization format that supports cross-platform and cross-language, and has high compression and parsing efficiency.

[0231] (3) File tail. The file tail contains the checksum of the data file, which is used to verify the integrity and correctness of the data. The checksum is generated using the CRC32 algorithm, which calculates the file header and data body to obtain a 32-bit checksum value, which is appended to the end of the file.

[0232] The file header uses JSON format and contains the following fields:

[0233] (1) data_source: data source, which refers to the data collection equipment or system, such as "railway signal monitoring system".

[0234] (2)generate_time: file generation time, in ISO8601 format, such as "2023-05-20T12:30:00Z".

[0235] (3) format_version: format version number, indicating the version of the monitoring data file format, such as "1.0".

[0236] (4) compress_algorithm: Compression algorithm, indicating the compression algorithm used by the data body, such as "rule_based_adaptive".

[0237] (5)encrypt_algorithm: encryption algorithm, indicating the encryption algorithm used for the data body, such as "AES-256-CBC", or "none" if not encrypted.

[0238] (6) encrypt_key: encryption key, empty if the data body is not encrypted.

[0239] The advantages of using ProtocolBuffer encoding are as follows:

[0240] (1) High data compression efficiency. ProtocolBuffer uses an efficient encoding method that can compress structured data into a very small space.

[0241] (2) Fast parsing speed. ProtocolBuffer-encoded data can be quickly parsed into structured objects without the need for complex text parsing.

[0242] (3) Cross-platform and cross-language. Protocol Buffer supports multiple programming languages ​​and can transmit and parse data between different platforms and systems.

[0243] The file tail consists of a 32-bit CRC32 checksum, which is calculated on the file header and data body to verify the integrity and correctness of the data file. The calculation steps of the CRC32 checksum are as follows:

[0244] (1) Combine the file header JSON string and the data body ProtocolBuffer encoded byte array into a complete byte array.

[0245] (2) Perform CRC32 calculation on the merged byte array to obtain a 32-bit check value.

[0246] (3) Write the checksum to the end of the file in big-endian order.

[0247] When parsing the monitoring data file, it is necessary to recalculate the CRC32 checksum of the file header and data body and compare it with the checksum value at the end of the file. If they are inconsistent, it means that an error occurred during the transmission or storage of the file.

[0248] The compression process of monitoring data records is as follows:

[0249] (1) Data segmentation: The monitoring data records are segmented into blocks of fixed size, and each data block is compressed independently.

[0250] (2) Compression rate selection: For each data block, the optimal compression rate is selected through an adaptive compression rate selection algorithm.

[0251] (3) Compression rule application: Based on the selected compression ratio and data type, apply the corresponding compression rule to each record in the data block to generate compressed data.

[0252] (4) ProtocolBuffer encoding: Convert the compressed data into ProtocolBuffer format and generate a data body byte array.

[0253] The packaging process of the monitoring data file is as follows:

[0254] (1) Generate a file header. Generate a file header string in JSON format based on the attributes of the monitoring data and the compression algorithm.

[0255] (2) Assemble the data body. Write the ProtocolBuffer encoded byte array of the compressed monitoring data record into the data body in sequence.

[0256] (3) Calculate the checksum. Combine the file header string and the data body byte array and calculate the CRC32 checksum.

[0257] (4) Write to the end of the file. Write the CRC32 checksum to the end of the file in big-endian byte order.

[0258] (5) Save the file. Write the file header, data body and file footer into the output file in sequence to generate a complete monitoring data file.

[0259] The decompression and decapsulation process of the monitoring data file is as follows:

[0260] (1) Read the file header. Read the file header string in JSON format from the beginning of the file and parse the file header fields.

[0261] (2) Read the data body. Starting from the end of the file header, read a byte array of the specified length as the data body encoded by ProtocolBuffer.

[0262] (3) Verify file integrity. Read the CRC32 checksum at the end of the file and compare it with the recalculated checksums of the file header and data body. If they are consistent, the file is intact.

[0263] (4) Decode the data body. Decode the data body encoded by ProtocolBuffer into an array of MonitorRecord objects.

[0264] (5) Decompress the data record. For each MonitorRecord object, decompress it and restore it to the original monitoring data record according to the compression algorithm and compression rules.

[0265] When compressing and decompressing monitoring data records, parallel processing can be used to fully utilize the computing power of multi-core CPUs and improve data processing efficiency. The specific optimization measures are as follows:

[0266] (1) Data segmentation: The monitoring data records are segmented into blocks of fixed size. Each data block can be compressed and decompressed independently without affecting each other.

[0267] (2) Thread pool: Create a thread pool of fixed size, where each thread is responsible for processing the compression or decompression task of a data block.

[0268] (3) Task submission: Submit the data block compression or decompression task to the thread pool, which will allocate idle threads to perform the task.

[0269] (4) Result merging: After all tasks are completed, the compressed or decompressed data blocks are merged into a complete monitoring data record.

[0270] By parallelizing compression and decompression, the multi-core characteristics of the CPU can be fully utilized to significantly improve the speed and throughput of data processing.

[0271] When compressing and decompressing a large number of monitoring data records, memory allocation and release are required frequently, which may lead to memory fragmentation and increased GC overhead. In order to reduce memory allocation and GC pressure, the following optimization measures can be taken: Object pool. Create a MonitorRecord object pool, pre-allocate a certain number of objects, reuse these objects during compression and decompression, and avoid frequent creation and destruction of objects. Byte array pool. Create a byte array pool, pre-allocate byte arrays of a certain size, reuse these byte arrays during compression and decompression, and avoid frequent allocation and release of memory.

[0272] In an optional embodiment,

[0273] The data management server decompresses and decodes the monitoring data files, and uses the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, stores the structured monitoring data in the distributed database, and performs sharding and redundant backup management on the database.

[0274] In the railway signal monitoring system, the massive monitoring data collected by the monitoring equipment needs to be efficiently stored, managed and analyzed to support the system's real-time monitoring, fault diagnosis and predictive maintenance functions.

[0275] After the data management server receives the compressed monitoring data file, it first needs to decompress the file to restore the original monitoring data record. The decompression process is carried out in the following steps:

[0276] Read the file header. Read the file header string in JSON format from the starting position of the monitoring data file, parse the file header field, and obtain metadata such as data source, generation time, format version, and compression algorithm.

[0277] Read the data body. Starting from the end of the file header, read the byte array of the specified length as the compressed monitoring data record.

[0278] Verify file integrity. Read the CRC32 checksum at the end of the file, recalculate the CRC32 checksum of the file header and data body, and compare the two checksums to see if they are consistent, to verify whether the file is intact during transmission and storage.

[0279] Data decompression: Decompress the monitoring data records in the data body according to the compression algorithm specified in the file header and restore the original data format.

[0280] The decompressed monitoring data records are encoded in ProtocolBuffer format and need to be decoded and converted into structured data objects. The decoding process is carried out in the following steps:

[0281] ProtocolBuffer decoding: Use the decoder generated by the ProtocolBuffer compiler to decode the monitoring data record in byte array format into a MonitorRecord object.

[0282] Data conversion. Extract each field in the MonitorRecord object and convert it to the data type corresponding to the database table structure, such as converting the timestamp to the date and time type, and converting the longitude and latitude to the geographic coordinate type.

[0283] The decompressed and decoded monitoring data records are stored in the memory in a structured form for subsequent data processing and analysis.

[0284] The raw data collected by the monitoring equipment may have quality problems such as noise, outliers, and duplicate records, and data cleaning is required to ensure the accuracy and reliability of the data. The main steps of data cleaning are as follows:

[0285] Deduplication. Deduplication is performed on the data according to the unique identifier of the monitoring data record (such as timestamp, device number, record number, etc.), and duplicate records are deleted. Outlier detection. Statistical analysis is performed on each field of the monitoring data, and the mean, variance, median and other indicators of the numerical distribution are calculated. According to the preset threshold, outliers are identified and marked. Common outliers include measurement values ​​that exceed the range, constant values ​​at multiple consecutive time points, and data points with sudden changes. Noise filtering. For continuously sampled monitoring data (such as vibration, sound, etc.), digital signal processing algorithms such as Kalman filtering and wavelet transform are used to smooth and denoise the data to eliminate high-frequency noise and random disturbances. Missing value processing. For missing or invalid monitoring data, interpolation, fitting, machine learning and other methods are used to estimate and fill in according to the semantics and context of the data to ensure the integrity and continuity of the data.

[0286] The data cleaning process can use the MapReduce parallel computing framework to distribute the cleaning tasks to multiple nodes for concurrent execution, thereby improving the efficiency and speed of data processing.

[0287] Railway signal monitoring data includes many types, such as track circuit status, transponder code, signal light position, switch position, etc. Different types of data have different data structures, time granularity and business semantics, and data classification is required for subsequent storage, query and analysis. The main steps of data classification are as follows: Data type identification. According to the field name and data type of the monitoring data record, a rule engine or machine learning algorithm is used to automatically identify the data type, such as state quantity, event, alarm, log, etc.

[0288] Business rule matching. According to predefined business rules and domain knowledge, the monitoring data is semantically understood and classified, such as by monitoring object (turnout, signal, track circuit, etc.), monitoring indicator (voltage, current, displacement, etc.), fault type (circuit break, poor contact, bit error, etc.). Metadata labeling. Metadata tags are added to the classified monitoring data to record contextual information such as the source, timestamp, spatial location, and equipment model of the data, so as to facilitate data retrieval and association analysis.

[0289] The data classification process can also use the MapReduce parallel computing framework to concurrently execute data type identification and business rule matching tasks on multiple nodes to improve classification efficiency.

[0290] In order to achieve fast retrieval and query of monitoring data, it is necessary to index the data to speed up data access. Common indexing technologies include:

[0291] B+ tree index. B+ tree is a balanced multi-way search tree that supports fast key value query and range query. B+ tree index can be established on the timestamp, device number, location and other fields of monitoring data to achieve millisecond-level data retrieval.

[0292] Inverted index. The inverted index extracts keywords from the data, establishes a mapping relationship between keywords and data records, and supports full-text search based on keywords. An inverted index can be established on the text fields of the monitoring data (such as fault descriptions, work order content, etc.) to achieve flexible text queries.

[0293] Geospatial index. Geospatial index is used to optimize the query and analysis of spatial data, such as finding monitoring data within a certain geographic area, calculating the distance between two monitoring points, etc. Commonly used geospatial indexes include R-tree, quadtree, grid index, etc.

[0294] The creation and update of indexes can be synchronized with data storage, or asynchronously, and new and updated data can be indexed regularly. Index files can be stored in distributed file systems (such as HDFS) or object storage (such as S3) to facilitate data sharing and backup.

[0295] The storage of monitoring data needs to consider factors such as large data volume, frequent access, and high availability. Using a distributed database is a common solution. Distributed databases store data in multiple nodes and achieve load balancing and fault tolerance through data sharding and replication. Common distributed databases include:

[0296] Distributed relational databases. Such as MySQL Cluster, PostgreSQL-XL, etc., support SQL query and transaction processing, suitable for structured monitoring data storage. Distributed NoSQL databases. Such as Cassandra, HBase, etc., use column storage or key-value storage, support high-concurrency reading and writing of massive data, suitable for semi-structured and unstructured monitoring data storage. Time series databases. Such as InfluxDB, OpenTSDB, etc., are specially optimized for the storage and query of time series data, support efficient data compression and aggregation calculation, and are suitable for storing monitoring data with high sampling frequency.

[0297] When storing data, it is necessary to design appropriate data models and table structures according to the characteristics of the data and the query mode, and optimize the data organization and indexing methods. For example, for frequently queried monitoring indicators, a separate summary table can be established to pre-calculate statistical values; for large-field monitoring data (such as images, audio, etc.), object storage can be used to store metadata in the database to achieve separate storage of data.

[0298] Distributed databases need data sharding and backup to achieve data load balancing and high availability. Data sharding refers to horizontally splitting a large table into multiple small tables, which are stored on different nodes. Each node only stores a part of the data. Common sharding strategies include: Range sharding. According to the data's timestamp, device number and other fields, the data is divided into multiple continuous ranges, and each range corresponds to a shard. Hash sharding. Use hash functions (such as consistent hashing) to map data to different shards, and each shard is responsible for data within a hash value range.

[0299] Data sharding can be statically configured using predefined rules or dynamically adjusted based on data volume and access patterns. The metadata information of the shards needs to be synchronized to all nodes to ensure data consistency and availability. Data backup refers to copying data to multiple nodes to prevent data loss caused by single point failures. Common backup strategies include:

[0300] Master-slave replication. Set one node as the master node, responsible for writing and updating data; other nodes act as slave nodes, asynchronously replicate the master node's data, and provide read-only services. Multi-master replication. Set multiple nodes as master nodes, each of which is responsible for writing and updating a portion of the data; master nodes use distributed consistency protocols (such as Paxos, Raft, etc.) to achieve data synchronization and conflict resolution. Remote disaster recovery. Replicate data to multiple data centers that are geographically far apart to prevent data loss caused by regional failures (such as earthquakes, floods, etc.).

[0301] Data backup can be done by selecting synchronous or asynchronous replication, and setting different replication factors and consistency levels based on business needs and cost budget. Backup data needs to be checked and repaired regularly to ensure data integrity and consistency.

[0302] The monitoring data processing flow includes the following steps:

[0303] Data reception. The data management server receives monitoring data files from monitoring devices or data transmission servers through network interfaces or message queues. Data decompression and decoding. Decompress monitoring data files and decode ProtocolBuffer to restore the original monitoring data records. Data cleaning and classification. Using the MapReduce parallel computing framework, perform cleaning operations such as deduplication, anomaly detection, and noise filtering on monitoring data, and classify and label the data according to business rules.

[0304] Data indexing and storage. Create an index for the cleaned monitoring data and store it in a distributed database, while performing data sharding and backup. Use big data analysis platforms (such as Hadoop, Spark, etc.) to perform batch and stream processing on monitoring data to achieve data aggregation, statistics, mining and other analysis tasks; use visualization tools (such as Grafana, Tableau, etc.) to display and interact with analysis results.

[0305] The various steps in the data processing flow can be executed serially or in a pipelined parallel manner to improve the efficiency and throughput of data processing.

[0306] Figure 2 FIG. 1 is a schematic diagram of the structure of a railway signal monitoring data generating system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0307] The first unit is used to pre-process the acquired track circuit carrier frequency signal data, extract the time-frequency feature matrix of the track circuit carrier frequency signal data by short-time Fourier transform, and then use a convolutional neural network to extract features of the time-frequency feature matrix to obtain the carrier frequency features of the track circuit carrier frequency signal data; decode the transponder signal data, extract the response coding information corresponding to the transponder signal data by a matched filtering method, and determine the receiving time of the transponder signal data by a correlation peak search algorithm; perform time-space correlation fusion processing on the carrier frequency features, response coding information, receiving time of the transponder signal data, train speed data and train positioning data, and generate monitoring data records in a unified coordinate system by data timestamp alignment and spatial position mapping;

[0308] The second unit is used to perform data verification and anomaly detection on the monitoring data record. If there is no problem with the data verification and anomaly detection, a rule-based data compression algorithm is used to adaptively select compression rules and compression rates to compress the monitoring data record; the compressed monitoring data record is encapsulated into a monitoring data file in a standard format, and the monitoring data file consists of a file header, a data body, and a file footer. The file header contains metadata such as data source, generation time, and format version. The data body adopts the ProtocolBuffer encoding format, and the file footer adopts the CRC32 algorithm to generate a checksum.

[0309] The third unit is used for the data management server to decompress and decode the monitoring data files, and use the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, store the structured monitoring data in the distributed database, and perform sharding and redundant backup management on the database.

[0310] According to a third aspect of the embodiments of the present invention,

[0311] An electronic device is provided, comprising:

[0312] processor;

[0313] a memory for storing processor-executable instructions;

[0314] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0315] A fourth aspect of the embodiments of the present invention is:

[0316] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0317] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0318] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating railway signal monitoring data, characterized in that: include: Preprocess the acquired track circuit carrier frequency signal data, extract the time-frequency feature matrix of the track circuit carrier frequency signal data by short-time Fourier transform, and then use convolutional neural network to extract features of the time-frequency feature matrix to obtain the carrier frequency features of the track circuit carrier frequency signal data; Decode the transponder signal data, extract the response coding information corresponding to the transponder signal data by a matched filtering method, and determine the receiving time of the transponder signal data by a correlation peak search algorithm; perform spatiotemporal correlation fusion processing on the carrier frequency characteristics, response coding information, receiving time of the transponder signal data, train speed data, and train positioning data, and generate monitoring data records in a unified coordinate system by aligning data timestamps and mapping spatial positions; Performing data verification and anomaly detection on the monitoring data record, if there is no problem with the data verification and anomaly detection, using a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data record; encapsulating the compressed monitoring data record into a monitoring data file in a standard format, the monitoring data file consists of a file header, a data body and a file footer, the file header contains metadata such as data source, generation time and format version, the data body adopts ProtocolBuffer encoding format, and the file footer adopts CRC32 algorithm to generate a checksum; The data management server decompresses and decodes the monitoring data files, and uses the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, stores the structured monitoring data in the distributed database, and performs sharding and redundant backup management on the database.

2. The method according to claim 1, characterized in that Decoding the transponder signal data, extracting the response coding information corresponding to the transponder signal data by a matched filtering method, and determining the receiving time of the transponder signal data by a correlation peak search algorithm include: Performing matched filtering on the transponder signal data, and extracting the response coding information in the transponder signal data by setting a filter that matches the transponder signal data; Performing a correlation peak search on the response coding information, determining the correlation peak position of the response coding information by calculating the autocorrelation function of the response coding information, and determining the time point corresponding to the correlation peak position as the receiving time of the transponder signal data; Generate decoded transponder information according to the response coding information and the receiving time, wherein the decoded transponder information includes a transponder identifier and a receiving time; The steps of the matched filtering process include: Constructing a filter matching the transponder signal data according to known parameters of the transponder signal data; Filtering the transponder signal data using the filter to obtain a filtered transponder signal sequence; Sampling and judging the filtered transponder signal sequence, judging the amplitude of each sampling point according to a preset response coding waveform, and obtaining response coding information; The step of performing a correlation peak search on the response coding information comprises: Performing an autocorrelation operation on the response coding information to obtain an autocorrelation function sequence; Performing peak search on the autocorrelation function sequence to determine the peak position and peak size; The point where the peak position is the largest and the peak value exceeds the preset peak value threshold is determined as the correlation peak, and the time of the sampling point corresponding to the correlation peak is determined as the receiving time; The receiving time of the transponder signal data is determined according to the receiving time and the sampling period.

3. The method according to claim 1, characterized in that The carrier frequency characteristics, the response coding information, the receiving time of the transponder signal data, the train running speed data and the train positioning data are subjected to time-space correlation fusion processing, and the monitoring data records under the unified coordinate system are generated by aligning the data timestamps and mapping the spatial positions, including: Synchronize the clock sources of multiple monitoring devices distributed in different geographical locations. The clock source synchronization is achieved by deploying a master clock and a slave clock of the IEEE 1588 time protocol in the monitoring devices, obtaining the time deviation of the slave clock relative to the master clock through communication between the slave clock and the master clock, and adjusting the slave clock according to the time deviation to achieve synchronization of the clock sources of each monitoring device; Establish the mapping relationship between the railway geographic information system line coordinate system and the WGS-84 coordinate system, obtain the spatial coordinate sequence of the center line of the railway geographic information system line coordinate system, use the least squares method to fit the transformation model parameters from the railway geographic information system line coordinate system to the WGS-84 coordinate system, and convert the installation position of the monitoring equipment from the railway geographic information system line coordinate system to the WGS-84 coordinate system to obtain the geographic spatial position of the monitoring equipment; The train positioning data in the WGS-84 coordinate system output by the train positioning equipment is matched with the railway line through the hidden Markov model matching algorithm to determine the mileage position of the train positioning data in the railway line.

4. The method according to claim 3, characterized in that The method further comprises: A quadtree is used to divide and index the two-dimensional geographic space of the monitoring data records. On each leaf node of the quadtree, a B+ tree index is used to organize the monitoring data records in the geographic space grid of the leaf node, and the key value of the B+ tree index is the timestamp of the monitoring data record. For each piece of train positioning data, based on the spatiotemporal composite index of quadtree and B+ tree, track circuit monitoring data, balise monitoring data and train speed monitoring data within a preset time range before and after the timestamp of the train positioning data and within a preset distance range before and after the geographic location are retrieved; The retrieved monitoring data within the same time and space range are clustered according to timestamps, and each clustered data set is subjected to association fusion processing. The association fusion processing normalizes the track circuit monitoring data according to carrier frequency type and monitoring parameters, and encodes the balise monitoring data according to information type, and encapsulates the normalized track circuit monitoring data, the encoded balise monitoring data, the train speed data and the train positioning data into a unified structured monitoring data record, wherein the monitoring data record contains a unique identifier, timestamp, spatial position, speed, track circuit status and balise information.

5. The method according to claim 1, characterized in that Adopting a rule-based data compression algorithm to adaptively select compression rules and compression ratios to compress the monitoring data records; packaging the compressed monitoring data records into a monitoring data file in a standard format includes: Adopting a rule-based data compression algorithm, according to the data type and numerical distribution characteristics of the monitoring data record, adaptively selecting a compression rule matching the monitoring data record, setting a plurality of predefined compression ratios, adaptively compressing the monitoring data record, and obtaining a compressed monitoring data record; Encapsulating the compressed monitoring data records into a monitoring data file in a standard format, wherein the monitoring data file consists of three parts: a file header, a data body, and a file footer; The file header adopts JSON format and contains metadata fields such as data source, generation time, and format version; Convert the compressed monitoring data record into ProtocolBuffer encoding format as the data body of the monitoring data file; The CRC32 algorithm is used to calculate the file header and the data body to generate a check code, and the check code is used as the file tail of the monitoring data file.

6. The method according to claim 5, characterized in that Rule-based data compression algorithms include: Divide the monitoring data records into blocks of fixed size, and compress each data block independently; Initially selecting a compression rate from predefined compression rates according to data characteristics of the data block; Compressing the data block using the selected compression rate, and counting the compressed data size and compression time; Calculating a performance score of the compression rate, the performance score comprehensively considering the compression rate, the compression speed and the compression effect; Compare the performance scores of multiple compression rates, select the compression rate with the best performance score as the compression rate of the next data block, and randomly explore other compression rates; The steps of data decompression and decapsulation include: Read the file header of the monitoring data file and parse the file header field; Read the data body of the monitoring data file, and decode the data encoded by ProtocolBuffer into monitoring data records; Read the file tail of the monitoring data file, extract the CRC32 check code, recalculate the check code of the file header and the data body, compare whether the two check codes are consistent, and verify the integrity of the file; Decompress and restore the monitoring data record according to the compression algorithm and compression rule of the monitoring data record.

7. A railway signal monitoring data generation system, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to pre-process the acquired track circuit carrier frequency signal data, extract the time-frequency feature matrix of the track circuit carrier frequency signal data by short-time Fourier transform, and then use convolutional neural network to extract features of the time-frequency feature matrix to obtain the carrier frequency features of the track circuit carrier frequency signal data; Decode the transponder signal data, extract the response coding information corresponding to the transponder signal data by a matched filtering method, and determine the receiving time of the transponder signal data by a correlation peak search algorithm; perform spatiotemporal correlation fusion processing on the carrier frequency characteristics, response coding information, receiving time of the transponder signal data, train speed data, and train positioning data, and generate monitoring data records in a unified coordinate system by aligning data timestamps and mapping spatial positions; The second unit is used to perform data verification and anomaly detection on the monitoring data record. If there is no problem with the data verification and anomaly detection, a rule-based data compression algorithm is used to adaptively select compression rules and compression rates to compress the monitoring data record; the compressed monitoring data record is encapsulated into a monitoring data file in a standard format, and the monitoring data file consists of a file header, a data body, and a file footer. The file header contains metadata such as data source, generation time, and format version. The data body adopts the ProtocolBuffer encoding format, and the file footer adopts the CRC32 algorithm to generate a checksum. The third unit is used for the data management server to decompress and decode the monitoring data files, and use the MapReduce parallel computing framework to clean, classify and index the data in the monitoring data files, store the structured monitoring data in the distributed database, and perform sharding and redundant backup management on the database.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images