Intelligent railway signal monitoring system based on multi-source data fusion

Abstract

The invention relates to the technical field of railway signal monitoring, in particular to a railway signal intelligent monitoring system based on multi-source data fusion, which comprises a multi-source sensing front end, a high-precision time reference unit, a heterogeneous data space-time registration engine, a multi-feature fusion analysis module, an intelligent diagnosis decision unit and a comprehensive monitoring interaction platform. According to the invention, all-directional synchronous monitoring of railway signal equipment, track states and obstacles is realized, the monitoring dimension and integrity are improved, multi-modal data such as electricity, light and waves are unified to a consistent space-time framework through space-time accurate registration, and the problem of data isomerism is solved; through feature level fusion and cross verification, potential contradictions or association abnormities among different logics such as signal lamp states, switch machine actions, track occupation and the like can be found, and the fault recognition accuracy, especially the recognition accuracy of composite faults or hidden faults, is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of railway signal monitoring, and particularly relates to a railway signal intelligent monitoring system based on multi-source data fusion. BACKGROUND

[0002] In the field of railway transportation, the reliability and safety of the signal system are crucial. Existing railway signal monitoring technologies usually rely on the collection and analysis of single or limited types of data, such as independently monitoring the electrical performance parameters of signal equipment, monitoring the track area through video surveillance, or detecting obstacles using specific sensors. The single dimension of this monitoring makes it difficult to comprehensively perceive the state of signal equipment, track occupancy, and external environmental risks, and cannot form multi-dimensional cross verification. The recognition ability of complex faults or hidden risks is limited. In the face of extreme environments such as heavy rain, thick fog, or electromagnetic pulse interference, the perception accuracy of existing systems will decrease significantly, and false positives or false negatives are likely to occur, making it difficult to meet the stringent requirements of railway operation for high real-time and high accuracy of the monitoring system.

[0003] Further, since different monitoring devices usually use independent time references and sampling clocks, when collecting high-speed changing electrical signal waveforms, video image streams, and radar point clouds, etc. multi-source asynchronous data, significant clock deviation and drift will occur. This inconsistency in time sequence makes it difficult to accurately align the data from different sensors on the time axis, leading to errors in event causal relationship judgment during subsequent joint analysis, affecting the timeliness and accuracy of state diagnosis.

[0004] At the data analysis level, existing methods mostly focus on independent feature extraction and threshold judgment of single modal data (such as waveforms or images), and cannot effectively build the internal logic between signal display, turnout action, and track occupancy, thereby limiting the system's ability to perform logical consistency verification and complex fault reasoning. SUMMARY

[0005] The application aims to provide a railway signal intelligent monitoring system based on multi-source data fusion, comprising: a multi-source perception front end for synchronously collecting electrical performance parameters of railway signal equipment, video image streams of track idle states, and radar point cloud data of obstacle intrusion; a high-precision time reference unit for providing a unified time calibration pulse for the multi-source perception front end by using Beidou timing signals, and controlling sampling clock deviation of the whole system to be in the nanosecond level by using master-slave clock synchronization protocols, so as to eliminate clock drift errors of different devices in the high-speed sampling process; a heterogeneous data space-time registration engine physically connected with the multi-source perception front end and the high-precision time reference unit, for performing three-dimensional space nonlinear mapping alignment of pixel space coordinates in the video image, polar coordinates of the radar point cloud data, and physical mileage coordinates of the railway signal equipment by establishing a high-precision track line physical geometric model, and performing secondary spline interpolation alignment processing on data streams of different refresh frequencies based on the time calibration pulse; a multi-feature fusion analysis module connected with the heterogeneous data space-time registration engine, for performing dimension reduction processing on electrical performance waveforms, visual semantic features, and spatial depth features by using a deep feature extraction layer, obtaining a fusion feature vector by establishing a multi-modal correlation matrix and performing vector projection in a high-dimensional feature space, and realizing cross-checking of signal light states, switch action curves, and track occupation logic at a feature level; an intelligent diagnosis decision unit receiving the fusion feature vector output by the multi-feature fusion analysis module, combining a preset railway signal logic rule library and a fault mode sample library, identifying signal display errors, switch action timeouts, and sensor failure abnormalities by using a probability reasoning mechanism based on confidence weight, and having a mechanism for automatically adjusting sensing weight according to environmental visibility; and a comprehensive monitoring interaction platform for performing three-dimensional digital twin visualization output and multi-level fault early warning on the recognition result of the intelligent diagnosis decision unit, and performing corresponding safety protocol locking operations on a train operation control system according to the warning level.

[0006] By using the above technical solution, all-round synchronous monitoring of railway signal equipment (electrical characteristics), track states (vision), and obstacles (radar) is realized, and monitoring dimension and integrity are improved; by accurate space-time registration, multi-modal data such as electricity, light, and waves are unified into a consistent space-time framework, and the data heterogeneity problem is solved; by feature-level fusion and cross-checking, potential contradictions or correlation abnormalities between signal light states, switch actions, and track occupation can be found, and the fault recognition accuracy, especially the recognition accuracy of complex faults or implicit faults, is improved.

[0007] Optionally, the heterogeneous data space-time registration engine comprises a dynamic weight distribution submodule, which adjusts the contribution of the electrical signal, the visual signal and the radar signal in the spatial fusion process in real time according to the data quality evaluation value of each sensor at the current time, and automatically increases the weight coefficient of the radar point cloud data in the obstacle distance discrimination when it is detected that the contrast of the visual signal is lower than a preset threshold, so as to ensure the stability of the space-time alignment.

[0008] By adopting the above technical solutions, the robustness of the system under non-ideal perception conditions is improved. For example, when the quality of visual data is reduced due to low light, rain and fog weather, the system can automatically reduce the weight of the visual data and increase the weight of the radar data, so as to ensure the continuity of key functions such as obstacle detection, realize adaptive optimization of the data fusion process, and improve the overall quality and reliability of the space-time registration result. The dynamic weight distribution is closely combined with the multi-source perception front end and the space-time registration engine, and the spatial mapping alignment process is dynamically optimized according to the real-time changes of the data quality of different sensors, so that the fusion process is intelligently weighted, thereby enhancing the adaptability of the whole system to the dynamic environment.

[0009] Optionally, the heterogeneous data space-time registration engine further comprises a nonlinear time compensator, which monitors the transmission delay in real time, calculates the delay deviation of each data source to the server, and reconstructs the data sequence in the cache queue by using a quadratic spline interpolation algorithm, so as to ensure that the data packets participating in the fusion processing are completely overlapped on the logical time axis.

[0010] By adopting the above technical solutions, the timing error introduced by the transmission link is eliminated, and each data packet participating in the fusion is strictly aligned in the logical time, thereby further improving the accuracy of multi-source data fusion. Not only the interpolation problem of different frequency data is solved, but also the additional delay caused by network transmission uncertainty is actively identified and compensated. The dynamic weight distribution submodule and the nonlinear time compensator work together to ensure the high precision and high reliability of the output data of the space-time registration engine in the time and space dimensions.

[0011] Optionally, the multi-source perception front end integrates an environment perception component, which provides environment correction parameters for the subsequent fusion algorithm by monitoring the atmospheric visibility and precipitation, and is used for parameter gain adjustment of the visual feature extraction model in rainy and snowy weather.

[0012] By adopting the above technical solutions, the system can detect environmental changes in advance and proactively adjust the processing algorithm parameters, such as filter parameters to enhance image contrast and suppress rain and snow noise. This enables the system to maintain effective visual analysis capabilities even in severe weather, improving the overall monitoring system's adaptability to environmental changes and its all-weather operation capability. It also adds a new perception dimension of environmental parameters to multi-source data and feeds this information back to the deep feature extraction layer used to process visual data, achieving closed-loop optimization of perception and processing based on environmental conditions.

[0013] Optionally, the multi-feature fusion analysis module establishes a fusion logic based on confidence assessment to perform correlation analysis on the abnormal fluctuation characteristics of the switch machine current curve and the physical position offset characteristics of the turnout gap detected by visual monitoring. When the mutual information value of the two types of features exceeds the preset risk threshold, it is determined to be a hidden danger of the equipment and an alarm is triggered.

[0014] By adopting the above technical solution and analyzing the correlation (i.e., mutual information value) between electrical and visual signals at the feature level, early equipment hazards characterized by multiple weak anomalies that are difficult to detect by a single sensor can be discovered. This reduces the false alarm rate caused by occasional interference or errors of a single sensor, improves the accuracy and reliability of alarms, and provides a more accurate basis for preventive maintenance. By conducting in-depth correlation analysis between electrical performance waveforms and visual semantic features and using quantitative methods such as confidence assessment and mutual information value, more accurate fusion diagnosis than simple logic verification is achieved.

[0015] Optionally, the intelligent diagnostic decision-making unit has an extreme environment compensation mechanism. Under strong electromagnetic interference or extreme lighting conditions, the extreme environment compensation mechanism calls the steady-state feature template in the historical operating cycle to perform residual correction on the degraded features collected in real time, and extracts the signal pulse edges masked by environmental noise, thereby maintaining a constant recognition accuracy.

[0016] By adopting the above technical solutions, the system's signal recognition and fault diagnosis capabilities under extreme electromagnetic and lighting conditions are significantly improved, avoiding system malfunctions or misjudgments caused by environmental noise. By introducing historical steady-state feature templates as reference benchmarks, an effective noise suppression and feature recovery method is provided, ensuring the robustness of core diagnostic functions. The use of the sample library is expanded, and it can be used as a reference template for clean signals in extreme environments for the repair of real-time signals, thereby ensuring that the input features of the probabilistic inference mechanism are effective even under harsh conditions.

[0017] Optionally, the intelligent diagnostic decision unit further includes an adaptive threshold adjustment component, which dynamically updates the limit standards in the fault judgment logic based on changes in day and night light intensity and seasonal temperature fluctuations, thereby reducing false alarms caused by periodic environmental changes.

[0018] By adopting the above technical solutions, the diagnostic threshold can be dynamically and precisely set, enabling the fault judgment criteria to adapt to the periodic changes in the environment. This effectively reduces regular false alarms caused by periodic changes in environmental background noise, while avoiding missed alarms caused by overly lenient threshold settings in good environmental conditions. This further improves the accuracy and reliability of the system's operation 24 / 7 throughout the year. The extreme environment compensation mechanism handles sudden and severe extreme environmental interference, while the adaptive threshold adjustment component handles gradual and regular changes in the environmental background. The two work together on the intelligent diagnostic decision unit, giving its diagnostic logic a multi-level environmental adaptability to periodic changes.

[0019] Optionally, the system further includes distributed edge computing nodes, which are deployed in signal rooms along the railway line to perform on-site preprocessing and feature compression on the raw high-frequency data collected by the multi-source sensing front end, and only transmit the processed key feature vectors to the central server to reduce the bandwidth pressure on the backbone transmission network.

[0020] By adopting the above technical solutions, the bandwidth pressure and data transmission costs of the backbone communication network are greatly reduced, and the end-to-end latency of data transmission is reduced. Because feature extraction is completed locally, the amount of data uploaded is greatly reduced, the computing load of the central server is distributed, and the ability and scalability of the entire system to process massive amounts of data are improved. An edge computing node is added, which forwards some computing tasks (such as feature extraction), directly optimizing the performance and efficiency of data transmission and central processing in the system.

[0021] Optionally, the central server is deployed with a global knowledge graph, which integrates the operational data uploaded by all edge computing nodes across the entire line, analyzes the logical coupling relationship between signal devices in different sections, and realizes regional linkage monitoring and preventive maintenance.

[0022] By adopting the above technical solutions, the correlation between devices can be modeled through a global knowledge graph, enabling the inference of cascading effects of faults or anomalies. This supports regional preventative maintenance decisions. For example, when early signs of degradation are detected in a critical device, other logically related devices can be checked in advance, or the operating strategies of the relevant areas can be adjusted, improving the system-level security early warning capabilities and enabling the discovery of systemic risks that can only be triggered by the combined effects of multiple device states. The global knowledge graph works closely with the edge computing node architecture. Edge computing nodes are responsible for local feature extraction and preliminary processing, while the global knowledge graph of the central server uses the key feature vectors uploaded by all nodes to perform system-level correlation analysis and knowledge mining at a higher level, greatly improving the overall intelligence level of the system.

[0023] Optionally, the integrated monitoring and interaction platform supports multi-dimensional playback, which can synchronously play back the electrical signal waveforms, video recordings and radar scan trajectories before and after the fault in a unified three-dimensional view, providing complete data link support for manual review of the cause of the fault.

[0024] By adopting the above technical solutions, an extremely intuitive and efficient tool is provided for fault reproduction and manual verification. It synchronously reproduces scattered data of different formats in the same spatiotemporal coordinate system, making the entire process of fault occurrence clear at a glance. This greatly shortens the time for fault analysis, location and responsibility determination, and improves operation and maintenance efficiency. The intuitive multi-dimensional data playback itself is also an effective means of technical training and case teaching. It makes full use of the multi-source fusion data that has been spatiotemporally aligned and presents it in the most intuitive way of time synchronization playback, perfectly realizing the value closed loop from data fusion to visualization application. Attached Figure Description

[0025] Figure 1 This is a block diagram of the intelligent railway signal monitoring system based on multi-source data fusion proposed in this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0027] like Figure 1As shown, this application provides a railway signal intelligent monitoring system based on multi-source data fusion, including: a multi-source sensing front-end, used to synchronously collect electrical performance parameters of railway signaling equipment, video image streams of track idling status, and radar point cloud data of obstacle intrusion; a high-precision time reference unit, which uses BeiDou timing signals to provide a unified time calibration pulse for the multi-source sensing front-end, and controls the sampling clock deviation of the entire system to the nanosecond level through a master-slave clock synchronization protocol to eliminate clock drift errors of different devices during high-speed sampling; a heterogeneous data spatiotemporal registration engine, which physically connects the multi-source sensing front-end and the high-precision time reference unit, and establishes a high-precision track line physical geometric model to perform three-dimensional nonlinear mapping and alignment of pixel spatial coordinates in video images, polar coordinates of radar point cloud data, and physical mileage coordinates of railway signaling equipment, and performs secondary spline interpolation alignment processing on data streams with different refresh frequencies based on the time calibration pulse; and multi-feature fusion analysis. The module, whose input is connected to the heterogeneous data spatiotemporal registration engine, uses a deep feature extraction layer to reduce the dimensionality of electrical performance waveforms, visual semantic features, and spatial depth features. By establishing a multimodal correlation matrix and performing vector projection in a high-dimensional feature space, a fused feature vector is obtained, realizing cross-verification of signal light status, switch machine action curves, and track occupancy logic at the feature level. The intelligent diagnostic decision unit receives the fused feature vector output by the multi-feature fusion analysis module, combines it with a preset railway signal logic rule library and fault mode sample library, and uses a probabilistic reasoning mechanism based on confidence weights to identify signal display errors, switch machine action timeouts, and sensor failure anomalies. It also has a mechanism to automatically adjust the sensing weights according to environmental visibility. The integrated monitoring and interaction platform is used to output three-dimensional digital twin visualization of the identification results of the intelligent diagnostic decision unit and provide multi-level fault warnings. Based on the warning level, it executes the corresponding safety protocol interlocking operation to the train operation control system.

[0028] Specifically, regarding the detailed construction and hardware deployment scheme of the multi-source sensing front-end in this application embodiment, the system first installs voltage and current transformers with high sampling frequencies at signal rooms and key equipment nodes along the railway line. These voltage and current transformers can be selected from through-core electromagnetic induction sensors with a measurement accuracy of 0.5%. Non-contact current sensing is achieved by passing the signal power line through a toroidal ferrite core with high permeability. The integrated high-performance analog-to-digital converter chip can digitally convert current pulses at a frequency of 10,000 times per second, linearly mapping 0 to 20 amps of AC current into a 0 to 5 volt digital voltage signal, ensuring that the system can capture minute ripples and distortions in the switch machine's operating current waveform in real time. Simultaneously, a high-definition industrial camera is deployed high up on the signal column or tower of the equipment room. This camera can use a one-inch large-size stacked photosensitive element with a physical resolution of 3840 x 2160 pixels, capable of continuously capturing dynamic images of the track area at a high frame rate of 60 frames per second, with a photosensitivity as low as 0.001 lux. This ensures that the status and color of signal lights and the physical position changes of turnout gaps can still be clearly identified under low light conditions such as at night or in tunnels. With a zoom lens with autofocus, the system can magnify the details of key components within a range of 100 to 300 meters ahead according to the preset monitoring focal length, thereby acquiring a video image stream containing rich semantic information. In addition, in response to the unique obstacle intrusion and spatial distance perception requirements of the railway environment, the multi-source sensing front end can also integrate a millimeter-wave radar operating in the 77 GHz band. This millimeter-wave radar has an effective detection range of over 200 meters and excellent medium penetration performance, capable of penetrating extreme weather conditions such as rain, snow, fog, and haze. In practical applications, its angular resolution reaches within 1 degree, and its horizontal detection angle width covers a range of ±60 degrees. It can output the distance, speed, and orientation information of obstacles ahead of the track in real time and generate dense point cloud data distribution. Through the reflection characteristics of high-frequency electromagnetic waves, it accurately delineates the occupancy of track space, thereby making up for the shortcoming of reduced effective line of sight of visual monitoring in adverse weather conditions.

[0029] It is understood that the embodiments of this application can use a timing receiver supporting both BeiDou-2 and BeiDou-3 dual-mode systems as the global clock source. This timing receiver continuously receives navigation messages transmitted by BeiDou navigation satellites in orbit through a high-gain antenna installed in an open area. Using the received precise satellite orbit parameters and clock deviation correction values, it calculates the local Coordinated Universal Time in real time through the pseudorange differential positioning principle. Its time accuracy can be stably maintained within 10 nanoseconds, providing an absolutely unified and highly reliable nanosecond-level reference time standard for the entire monitoring system. In order to distribute this reference time standard signal losslessly to all sensing nodes in the entire system, the time standard unit is designed with a precise time calibration pulse distribution logic. The timing receiver outputs a standard second pulse signal every second. This second pulse signal is sent to the clock distribution center as a master clock trigger command and is synchronously distributed to voltage and current sensors distributed in different geographical locations through a dedicated fiber optic channel. The current transformer sampling unit, the high-definition camera control board, and the radar processor—each branch node—immediately resets its local crystal oscillator count upon receiving the rising edge of the second pulse signal, thus achieving global synchronization at the physical link level. To address the dynamic clock drift generated during data transmission and processing, the system also establishes a compensation mechanism based on a precision time protocol. By monitoring the time difference between the feedback pulses from each front-end device and the main reference pulse in real time, the system can calculate the frequency deviation rate of each device due to environmental temperature fluctuations or component aging. It then uses a proportional-integral-derivative control algorithm to generate a compensation factor, adjusting the timestamp field in the data encapsulation packet in real time. This ensures that the sampling time deviation of data from different sources is controlled within 50 microseconds when it reaches the back-end server, effectively eliminating time-axis misalignment of the data stream caused by inconsistent sampling frequencies or transmission delay fluctuations.

[0030] It is understandable that, regarding the geometric transformation and alignment algorithm of the heterogeneous data spatiotemporal registration engine in this application embodiment, the system first constructs a high-precision three-dimensional coordinate system of the track space as a unified spatial reference for all data. This three-dimensional coordinate system of the track space takes the origin of the starting and ending points of the track centerline of the current monitoring section as the origin, the track extension direction as the vertical axis, the direction perpendicular to the track centerline as the horizontal axis, and the direction perpendicular to the track plane as the elevation axis. By utilizing the three-dimensional modeling data of the track obtained by a high-precision laser scanner, the system pre-calibrates the physical mileage coordinates of key equipment such as signal lights, switch machines, and insulator joints in this three-dimensional coordinate system of the track space. For two-dimensional video images captured by high-definition industrial cameras, the heterogeneous data spatiotemporal registration engine uses the homography matrix transformation from video pixel coordinates to physical coordinates to achieve spatial alignment. By setting standard calibration points on the track site, the transformation relationship between the camera imaging plane and the track plane is calculated. When the camera captures the pixel position of a signal light or obstacle, the registration engine performs inverse perspective transformation through the homography matrix, mapping the two-dimensional pixel points to the corresponding coordinate points in the three-dimensional coordinate system of the track space, thereby... It eliminates perspective scaling and geometric distortion caused by shooting angle; for the polar coordinate radar point cloud data output by millimeter-wave radar, the heterogeneous data spatiotemporal registration engine performs coordinate rotation and translation processes. Based on the radar's installation height, depression angle, and horizontal deflection angle on the signal tower, it establishes a transformation matrix from the radar's local polar coordinate system to the global orbital space three-dimensional coordinate system, projects the discrete point cloud onto the orbital center coordinate system, and uses real-time dynamic differential positioning information to correct geometric offsets caused by orbital curves, thereby ensuring that the target's spatial position sensed by the radar is consistent with the actual orbital position. To ensure complete overlap of features, the heterogeneous data spatiotemporal registration engine also employs a fine calibration mechanism based on projection overlap. This mechanism projects the 3D target bounding box detected by the radar onto the 2D image plane of the camera. By comparing the differences between the pixel color features within the radar target bounding box and the background model, it calibrates the minute drifts of the matrix parameters in real time. For example, when a signal arm sways slightly due to strong winds, the system can automatically adjust the spatial transformation parameters in reverse based on the offset of the image feature points to maintain pixel-level overlap between the radar point cloud and the video image.

[0031] Understandably, the multi-feature fusion analysis module receives a standardized information bundle output from the heterogeneous data spatiotemporal registration engine. By deeply mining the nonlinear correlation between electrical performance parameters, video spatial features, and radar depth features, it constructs a high-dimensional feature space that can characterize the operating status of railway signaling equipment. Specifically, for the electrical indicator of switch machine operating current, the system introduces a time-frequency feature extraction mechanism based on non-stationary signal processing technology. Through empirical mode decomposition algorithms, the original current time series data with a sampling frequency of 10 kHz is decomposed into a series of intrinsic mode function components with different feature scales. This transforms the weak current distortion originally superimposed in the time domain into a Hilbert spectrum with extremely high resolution in both the frequency and time domains, thereby extracting multi-dimensional feature vectors including instantaneous frequency, edge spectrum energy distribution ratio, and average feature frequency. When processing visual and spatial data containing track equipment topology, the multi-feature fusion analysis module constructs a deep spatiotemporal graph convolutional neural network model. This model integrates various sensor nodes in the railway signaling system. Abstracted as vertices in a graph structure, and the physical connections between devices defined as edges, a hierarchical architecture with alternating spatial and temporal convolution operations is used to aggregate and extract the state features of adjacent signaling devices. This simulates the logical coupling effect within the signaling system, where an abnormal state of a signal is often reflected in the action sequence of adjacent track circuits or switch machines through interlocking logic. To capture the dynamic evolution of railway signaling device states over time, the aforementioned spatiotemporal graph convolutional neural network model is configured with feature processing units based on a one-dimensional gated convolution structure in the temporal dimension. It uses a sliding time window to compress and abstract the feature stream, and selectively retains trend features with long-term influence through a gating mechanism similar to a long short-term memory network. Finally, it outputs a fused feature vector that integrates spatial topological attributes and temporal series features. This fused feature vector not only includes the color pixel ratio of the signal light at the current moment and the physical opening of the turnout gap, but also the slope of these parameters over multiple past sampling periods, realizing digital modeling of the entire process of signaling device actions.

[0032] Understandably, the input of the intelligent diagnostic decision-making unit receives the fused feature vector output by the multi-feature fusion analysis module. This vector is first aligned with a preset railway signal logic rule base. This rule base follows the logical constraints of the signal interlocking table, defining deterministic causal relationships between current, voltage, light display color, and the occupancy status of the track ahead. When the monitored real-time parameters satisfy a preset logical path, the system quickly identifies the possible operating state or fault category. To handle complex operating conditions with ambiguity and uncertainty, the intelligent diagnostic decision-making unit introduces a dynamic evaluation algorithm based on confidence weights. This algorithm assigns different decision weights to electrical signals, visual signals, and radar signals based on atmospheric visibility, precipitation, and electromagnetic interference intensity fed back by the current environmental sensing components. Under standard clear weather conditions, the weight of visual signals is set to a higher value to leverage the advantages of refined monitoring. However, in dense fog or heavy rain... Under certain weather conditions, the weight of visual signals is automatically reduced while the weight of radar signals is increased. Through this dynamically adjusted weighted summation, the system generates a final fault probability distribution, thus providing the probability of a fault occurring at the statistical level. When performing qualitative analysis on specific fault modes, the intelligent diagnostic decision unit uses a hierarchical filtering process of a logic tree to match the fused features with known fault mode samples. Taking switch machine fault diagnosis as an example, the system first performs a primary classification based on whether the current action time exceeds a preset threshold. If it exceeds the threshold, it enters a secondary evaluation path of mechanical resistance and electrical power loss, further analyzing the distribution characteristics of the Hilbert energy spectrum in different frequency bands. If the proportion of low-frequency energy is abnormal and accompanied by slow turnout rotation in the visual features, it is determined to be due to poor lubrication or foreign object jamming in the mechanical transmission system. This decision-making method based on multi-dimensional data cross-verification effectively solves the problem of insufficient reliability of a single sensor in harsh environments.

[0033] Understandably, the integrated monitoring and interaction platform supports multi-dimensional playback, which can synchronously play back electrical signal waveforms, video recordings, and radar scan trajectories before and after a fault in a unified three-dimensional view, providing complete data link support for manual verification of the cause of the fault.

[0034] It is understood that in this embodiment, the integrated monitoring and interaction platform, based on a three-dimensional geometric model of railway infrastructure generated by high-precision laser scanning, constructs a virtual digital twin scene using a high-performance graphics rendering engine. This virtual twin can synchronously display the physical state of signals, the opening angle of switches, and the spatial location of obstacles in real time. On the interactive interface, the system displays pixel-level overlays of electrical signal waveforms, video images, and radar point cloud trajectories acquired from different physical sources along a unified time axis. The rendering refresh rate is maintained at over 60 frames per second, ensuring smooth visual display and a sense of real-time performance. When a signal fault or intrusion alarm occurs, the platform supports multi-dimensional backtracking. The system automatically retrieves all synchronous data streams before and after the fault from the distributed storage matrix and recreates the fault process in slow motion within the three-dimensional scene. The voltage and current changes of electrical signals, the physical deformation in the visual representation, and the motion trajectory detected by radar are precisely mapped to the corresponding components of the virtual equipment. Through this intuitive data visualization, technicians can observe the microscopic movements of signal equipment from any perspective. By paying attention to details, the platform significantly shortens the analysis cycle and on-site repair time for railway signal accidents. Furthermore, the integrated monitoring and interaction platform has established a response mechanism covering three levels: alert, warning, and severe alarm. Based on the risk level output by the intelligent diagnostic decision unit, it automatically triggers corresponding safety defense interlocking commands. When a severe violation or interlocking failure is identified, it immediately sends an emergency interlocking request to the train operation control system via a dedicated safety communication interface, forcibly setting the signal display of the relevant section to a red-light prohibition state and blocking all routes leading to the faulty area, ensuring absolute safety for train operations. When a potential hazard is identified, the platform only displays a warning and generates a work order. However, when a severe violation event or dangerous violation of interlocking logic is detected, the platform immediately triggers the highest-level alarm and sends an emergency interlocking request to the train operation control system via a dedicated safety communication interface, forcibly setting the signal of the relevant section to a red-light prohibition state and blocking routes. Simultaneously, it sends emergency notifications to dispatchers and maintenance personnel via SMS and voice modules, ensuring that the system can respond extremely quickly to prevent the spread of safety risks in extreme emergencies.

[0035] Understandably, the heterogeneous data spatiotemporal registration engine includes a dynamic weight allocation submodule. Based on the data quality assessment value of each sensor at the current moment, the dynamic weight allocation submodule adjusts the contribution of electrical signals, visual signals and radar signals in the spatial fusion process in real time. When the contrast of the visual signal is detected to be lower than the preset threshold, the weight coefficient of the radar point cloud data in obstacle distance discrimination is automatically increased to ensure the stability of spatiotemporal alignment.

[0036] Specifically, the dynamic weight allocation submodule establishes a weight adjustment logic based on a signal quality evaluation function by acquiring real-time operational status evaluation indicators of various sensing elements in the multi-source sensing front-end. The video image stream generated by the high-definition industrial camera is continuously input to the image information entropy calculation unit. This unit statistically analyzes the uniformity of image brightness distribution and the sharpness of edge features. When the image contrast value drops below a preset threshold due to direct sunlight or dense fog, the system automatically defines the quality evaluation coefficient of that visual channel as a low-confidence state. Simultaneously, the millimeter-wave radar detection unit calculates the spatial depth confidence level for the current detection task based on the signal-to-noise ratio of the reflected echo and the stability of the clustered targets. If the radar beam's ability to penetrate rain and snow environments... If the attenuation rate remains within the normal range, the system determines that the radar signal has high reliability. The dynamic weight allocation submodule then calls the weight offset matrix stored in the read-only memory and adjusts the contribution percentage of electrical signals, visual features, and radar depth data in the final fusion decision space in real time through linear weighted summation. For example, in extreme weather conditions with extremely low visibility, the system will forcibly reduce the spatial position weight of the visual signal, while compensatorily increasing the weight of millimeter-wave radar with penetrating characteristics. Combined with the real-time electrical change characteristics collected by voltage and current transformers, the system performs multi-dimensional verification of the status of signal equipment in a three-dimensional coordinate system. This ensures that even in extremely degraded environmental conditions, the system can still accurately determine track occupancy and equipment status through high-weighted radar and electrical data.

[0037] Understandably, the heterogeneous data spatiotemporal registration engine also includes a nonlinear time compensator. By monitoring the transmission delay in real time, it calculates the delay deviation of each data source to the server, and uses a quadratic spline interpolation algorithm to reconstruct the data sequence in the cache queue, ensuring that the data packets participating in the fusion process completely overlap on the logical time axis.

[0038] Specifically, the nonlinear time compensator is deployed in the computation layer of the heterogeneous data spatiotemporal registration engine to sense dynamic latency caused by transmission network jitter or fluctuations in edge computing node processing load in real time. The system encapsulates an absolute reference timestamp, generated by a BeiDou time receiver with an accuracy better than 10 nanoseconds, at the header of each sensed data packet. The nonlinear time compensator accurately calculates the absolute transmission latency of each heterogeneous data stream by comparing the difference between the actual system time of the data packet arriving at the backend server and this absolute reference timestamp. Addressing the misalignment issue between the video stream output by industrial cameras and the point cloud stream output by millimeter-wave radar at sampling times, the nonlinear time compensator... The compensator allocates a deep elastic cache queue in memory and uses a quadratic spline interpolation algorithm to upsample the low-frequency radar data in the time dimension. Specifically, by calculating the time interval between adjacent point cloud frames and the second derivative of the target motion vector, a virtual depth position point is interpolated at each millisecond node on the logical time axis. This increases the logical sampling frequency of the radar point cloud at the algorithm level, thereby enabling the microsecond-level waveform sampling of electrical signals, image frames of video streams, and interpolated radar spatial points to be aligned with extremely low time deviation within the same millisecond-level time window. This eliminates spatial mapping misalignment caused by asynchronous sampling periods of sensor hardware.

[0039] Understandably, the multi-source sensing front end integrates an environmental sensing component. This component monitors atmospheric visibility and precipitation to provide environmental correction parameters for subsequent fusion algorithms, which are used to adjust the parameter gain of the visual feature extraction model in rainy or snowy weather.

[0040] Specifically, the environmental sensing component acquires meteorological and physical parameters of the monitoring area in real time through an atmospheric visibility meter and a precipitation sensor integrated on the signal post. When the precipitation monitoring value exceeds the preset standard or the visibility decreases to the warning range, the environmental sensing component immediately sends an environmental correction parameter command to the multi-source sensing front end. This command triggers the internal processing chip of the high-definition industrial camera to activate automatic gain control compensation and digital defogging algorithm. By increasing the charge accumulation time of the photosensitive element and adjusting the brightness gain coefficient, the physical features of the originally blurred signal light display area and turnout gap in the image are enhanced at the pixel level. In addition, the system corrects the refractive index of the radial velocity measurement value of the millimeter-wave radar according to the precipitation intensity and compensates for the deviation of the propagation rate of electromagnetic waves in humid media using a preset air density and water vapor content relationship curve. This ensures that the decrease in the sensing and recognition accuracy of the system is strictly limited to a very small range when encountering extreme convective weather or continuous precipitation interference, greatly improving the robustness of the railway signal monitoring system under all-weather operating conditions.

[0041] Understandably, the multi-feature fusion analysis module establishes a fusion logic based on confidence assessment to perform correlation analysis on the abnormal fluctuation characteristics of the switch machine current curve and the physical position offset characteristics of the turnout gap detected by vision. When the mutual information value of the two types of features exceeds the preset risk threshold, it is judged as a hidden danger of the equipment and an alarm is triggered.

[0042] Specifically, the multi-feature fusion analysis module performs in-depth analysis of the high-frequency electrical characteristics acquired by the current transformers installed in the relay assembly rack of the signal control room. It uses Discrete Fourier Transform to extract high-frequency harmonic components characterizing motor wear and low-frequency energy fluctuation characteristics characterizing mechanical jamming from the switch machine current curve. Simultaneously, it retrieves the sequence of physical position changes of the turnout gap identified by high-definition cameras within the same time period. Image processing algorithms are used to measure the gap width between the turnout switch rail and the stock rail, and this physical displacement process is mapped as a displacement-to-time characteristic curve. The multi-feature fusion analysis module then establishes a system based on mutual information theory. The correlation matrix calculates the nonlinear correlation between the instantaneous slope change of the current curve and the acceleration change of the displacement curve. If the current data shows stall characteristics exceeding the threshold and the visual features indicate that the turnout displacement is stagnant in the abnormal range, the system calculates the conditional entropy of the two types of features under the joint probability distribution to obtain the current fault confidence score. When the fault confidence score exceeds the warning limit, the system immediately determines that the switch machine has a mechanical jamming fault. Combining the complementary advantages of the two types of features, the system eliminates false alarms caused by current bias drift of a single current transformer or visual obstruction, thereby achieving extremely high confidence monitoring of the status of key train operation equipment.

[0043] Understandably, the intelligent diagnostic decision-making unit has an extreme environment compensation mechanism. Under strong electromagnetic interference or extreme lighting conditions, the extreme environment compensation mechanism calls the steady-state feature templates from the historical operating cycle to perform residual correction on the degraded features collected in real time, and extracts the signal pulse edges that are covered by environmental noise, thereby maintaining a constant recognition accuracy.

[0044] Specifically, the intelligent diagnostic decision unit automatically activates when it detects strong electromagnetic interference in the external environment. Utilizing a large number of steady-state feature templates stored within historical operating cycles as a benchmark, it performs multi-dimensional projection analysis on the real-time acquired degradation feature vectors. When high-voltage electromagnetic pulse interference from the railway contact network causes random noise points in the voltage transformer's sampling data, the residual correction logic first initiates a time window smoothing algorithm based on adaptive median filtering to remove abnormal jump points that significantly deviate from physical logic. Subsequently, it uses a spatiotemporal graph convolutional neural network to obtain spatial redundancy features from adjacent monitoring nodes, combined with the historical distribution of steady-state electrical parameters, to calculate the estimated residual term under the current noise background. By subtracting this residual term from the real-time observation value, the system can reconstruct the original signal pulse edges masked by electromagnetic noise. The cross-correlation coefficient between the reconstructed signal waveform and the standard template remains at an extremely high level, ensuring that the diagnostic decision unit can accurately capture the millisecond-level action sequence of signal equipment even in complex physical environments. This achieves the continuity of monitoring results in the spatial dimension and the stability in the temporal dimension.

[0045] Understandably, the intelligent diagnostic decision unit also includes an adaptive threshold adjustment component. This component dynamically updates the limit standards in the fault judgment logic based on changes in day and night light intensity and seasonal temperature fluctuations, thereby reducing false alarms caused by periodic environmental changes.

[0046] Specifically, the adaptive threshold adjustment component establishes a multivariate threshold correction model that includes geographical location, real-time illuminance, humidity, and temperature data. This model reconstructs the static limits in the fault judgment logic in real time. In high-temperature summer environments, it automatically and dynamically increases the envelope range of the normal current waveform according to a preset ratio based on the temperature value fed back by the sensor. During the day-night cycle, to address the huge illuminance range faced by the visual recognition system, the adaptive threshold adjustment component adjusts the binarization threshold in the image segmentation algorithm in real time based on the average gray value output by the photosensitive element, and simultaneously updates the brightness weighting parameters in the traffic light color discrimination matrix. This makes the system's fault judgment standard a flexible logical surface that dynamically shifts with the periodic rhythm of the environment, effectively reducing the false alarm rate caused by natural environmental changes.

[0047] Understandably, the system also includes distributed edge computing nodes, which are deployed in signal rooms along the railway line to perform on-site preprocessing and feature compression on the raw high-frequency data collected by the multi-source sensing front end. Only the processed key feature vectors are transmitted to the central server to reduce the bandwidth pressure on the backbone transmission network.

[0048] Specifically, the distributed edge computing nodes can be deployed inside the signal machine rooms along the railway. Its hardware consists of embedded processing modules with high-performance parallel computing capabilities, which are specifically used to process the large-bandwidth raw data streams generated by the front-end current transformers, cameras, and radars. After receiving the high-frequency sampled electrical signal data, the edge computing nodes first perform local primary filtering and normalization processing, and use the fast Fourier transform to convert the time-domain signal into a frequency-domain feature vector, only retaining the main feature components, and compressing the original data throughput to less than one percent of the initial size; for video data, the pixel difference algorithm with motion compensation is used to extract the dynamic target features within the track area, eliminate the irrelevant static scene information in the background, and use an efficient coding standard to convert the video stream into a structured feature vector containing semantic tags; through this distributed processing architecture, the original monitoring data that originally occupied a huge bandwidth is compressed into a compact feature vector and transmitted back to the central server through an optical fiber backbone network with a transmission rate of 1000 megabits per second. This not only significantly reduces the transmission delay and network congestion risk of long-distance transmission, but also enables the central server to concentrate computing power to process higher-level global logical analysis tasks, achieving efficient coordination between the sensing layer and the decision-making layer of the monitoring system. The central server also deploys a global knowledge graph. By integrating the operation data uploaded by each edge computing node along the whole line, it analyzes the logical coupling relationship between signal devices in different sections to achieve regional linkage monitoring and preventive maintenance. When a certain node reports an early warning, the global knowledge graph will automatically start an associated search, analyze all the route states of the device in the interlocking table, so as to identify potential systemic safety risks, and the response time is strictly controlled within an extremely short range.

[0049] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0050] In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the module or unit is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0051] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0052] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0053] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0054] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A railway signal intelligent monitoring system based on multi-source data fusion, characterized in that, include: Multi-source sensing front end, used to synchronously collect electrical performance parameters of railway signaling equipment, video image streams of track idling status, and radar point cloud data of obstacle intrusion; The high-precision time reference unit uses the BeiDou time signal to provide a unified time calibration pulse for the multi-source sensing front end. Through the master-slave clock synchronization protocol, the sampling clock deviation of the entire system is controlled at the nanosecond level to eliminate clock drift error of different devices in the high-speed sampling process. The heterogeneous data spatiotemporal registration engine physically connects the multi-source sensing front-end and the high-precision time reference unit. By establishing a high-precision physical geometric model of the track line, it performs nonlinear mapping and alignment of the pixel spatial coordinates in the video image, the polar coordinates of the radar point cloud data, and the physical mileage coordinates of the railway signal equipment in three-dimensional space. Based on the time calibration pulse, it performs secondary spline interpolation alignment processing on the data streams with different refresh frequencies. The multi-feature fusion analysis module is connected to the heterogeneous data spatiotemporal registration engine at its input end. It uses a deep feature extraction layer to perform dimensionality reduction processing on electrical performance waveforms, visual semantic features and spatial depth features. By establishing a multimodal correlation matrix and performing vector projection in a high-dimensional feature space, it obtains a fused feature vector. At the feature level, it realizes cross-verification of signal light status, switch machine action curve and track occupancy logic. The intelligent diagnostic decision unit receives the fused feature vector output by the multi-feature fusion analysis module, combines it with the preset railway signal logic rule library and fault mode sample library, and uses a probability reasoning mechanism based on confidence weight to identify signal display errors, switch machine action timeouts and sensor failures. It also has a mechanism to automatically adjust the sensing weights according to the environmental visibility. as well as The integrated monitoring and interaction platform is used to output three-dimensional digital twin visualization of the identification results of the intelligent diagnostic decision-making unit and provide multi-level fault warnings, and to execute corresponding safety protocol interlocking operations to the train operation control system according to the warning level.

2. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 1, characterized in that, The heterogeneous data spatiotemporal registration engine includes a dynamic weight allocation submodule. The dynamic weight allocation submodule adjusts the contribution of electrical signals, visual signals and radar signals in the spatial fusion process in real time according to the data quality evaluation value of each sensor at the current moment. When the contrast of the visual signal is detected to be lower than a preset threshold, the weight coefficient of radar point cloud data in obstacle distance discrimination is automatically increased to ensure the stability of spatiotemporal alignment.

3. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 2, characterized in that, The heterogeneous data spatiotemporal registration engine also includes a nonlinear time compensator, which monitors the transmission delay in real time, calculates the delay deviation of each data source to the server, and uses a quadratic spline interpolation algorithm to reconstruct the data sequence in the cache queue, ensuring that the data packets participating in the fusion process completely overlap on the logical time axis.

4. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 1, characterized in that, The multi-source sensing front end integrates an environmental sensing component, which monitors atmospheric visibility and precipitation to provide environmental correction parameters for subsequent fusion algorithms, and is used to adjust the parameter gain of the visual feature extraction model under rainy and snowy weather.

5. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 1, characterized in that, The multi-feature fusion analysis module establishes a fusion logic based on confidence assessment to perform correlation analysis on the abnormal fluctuation characteristics of the switch machine current curve and the physical position offset characteristics of the turnout gap detected by visual monitoring. When the mutual information value of the two types of features exceeds the preset risk threshold, it is determined to be a hidden danger of the equipment and an alarm is triggered.

6. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 1, characterized in that, The intelligent diagnostic decision-making unit has an extreme environment compensation mechanism. Under strong electromagnetic interference or extreme lighting conditions, the extreme environment compensation mechanism calls the steady-state feature templates from the historical operating cycle to perform residual correction on the degraded features collected in real time, and extracts the signal pulse edges that are covered by environmental noise, thereby maintaining a constant recognition accuracy.

7. A railway signal intelligent monitoring system based on multi-source data fusion according to claim 6, characterized in that, The intelligent diagnostic decision-making unit also includes an adaptive threshold adjustment component, which dynamically updates the limit standards in the fault judgment logic based on changes in day and night light intensity and seasonal temperature fluctuations, thereby reducing false alarms caused by periodic environmental changes.

8. The intelligent railway signal monitoring system based on multi-source data fusion according to claim 1, characterized in that, The system also includes distributed edge computing nodes, which are deployed in signal rooms along the railway line. These nodes are used to preprocess and compress the raw high-frequency data collected by the multi-source sensing front end on-site, and only transmit the processed key feature vectors to the central server to reduce the bandwidth pressure on the backbone transmission network.

9. A railway signal intelligent monitoring system based on multi-source data fusion according to claim 8, characterized in that, The central server is equipped with a global knowledge graph. By integrating the operational data uploaded by edge computing nodes across the entire line, it analyzes the logical coupling relationships between signal devices in different sections, enabling regional coordinated monitoring and preventative maintenance.

10. A railway signal intelligent monitoring system based on multi-source data fusion according to claim 1, characterized in that, The integrated monitoring and interaction platform supports multi-dimensional playback, which can synchronously play back electrical signal waveforms, video recordings, and radar scan trajectories before and after a fault in a unified three-dimensional view, providing complete data link support for manual verification of the cause of the fault.

Images