A system for detecting and alarming foreign matter invading the sight line of a public overpass bridge

Abstract

The application discloses a kind of public overpass bridge foreign matter intrusion alarm and detection system, comprising: sensing subunit, video acquisition module, edge analysis module, cloud analysis module and data monitoring module.Sensing subunit is laid in bridge key position, for sensing collision and generating alarm signal;Video acquisition module is used to capture and intelligently identify intrusion behavior;Edge analysis module carries out local space-time fusion and preliminary analysis to multi-source information;Cloud analysis module carries out correlation verification, credibility evaluation and risk prediction;Data monitoring module carries out hierarchical alarm push and closed-loop process management according to risk level.The application realizes real-time, accurate monitoring and active early warning to public overpass bridge foreign matter intrusion behavior through multi-source sensing, edge-cloud collaborative intelligent analysis and closed-loop management, significantly improves the reliability and emergency response efficiency of system.

Description

Technical Field

[0001] This invention belongs to the field of foreign object intrusion detection, and in particular relates to a foreign object intrusion alarm and detection system for a road-rail overpass. Background Technology

[0002] With the rapid development of high-speed rail and highway networks, the number of road-rail overpasses is increasing daily. As three-dimensional intersections of railways and highways, their safe operation is crucial. Currently, for such open and complex scenarios, safety monitoring largely relies on manual inspections, single video surveillance systems, or traditional perimeter alarm technologies. Railway systems often use fiber optic vibration and overhead contact lines for the protection of enclosed lines, while highway bridges rely more on intelligent video analytics. While these technologies have some applications in their respective fields, when directly applied to road-rail overpasses, they are often independent and fragmented, making it difficult to achieve comprehensive, highly reliable, real-time monitoring of intrusions such as falling objects and thrown objects from the bridge deck.

[0003] However, existing technological solutions still have significant shortcomings in addressing the unique needs of road-rail overpasses: First, various monitoring methods, such as vibration sensing and video analysis, typically operate independently, resulting in isolated information and a lack of effective multi-source data fusion mechanisms, leading to high false alarm rates and difficulties in event confirmation. Second, the system's intelligence level is limited, mostly only capable of post-event alarms, lacking in-depth mining and learning of historical data, and unable to predict risk trends or provide proactive warnings. Third, the energy supply and computing power of front-end sensing devices are limited, making it difficult to operate stably for extended periods in harsh environments and complete complex local real-time analyses. Fourth, the alarm information push and emergency response processes lack intelligent linkage and closed-loop management, affecting the timeliness of handling. Therefore, there is an urgent need for a dedicated system that can deeply integrate sensing, possess edge intelligence, support cloud-based collaborative analysis, and achieve closed-loop management. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the present invention provides a foreign object encroachment alarm and detection system for road-rail overpasses, comprising: The sensing subunits are deployed at key locations on the bridge to detect collisions with foreign objects and generate initial alarm signals. The video acquisition module is used to capture visual information of the monitored area and perform real-time analysis of the video stream to identify foreign object intrusion behavior and obtain structured recognition results containing target information. The edge analysis module, connected to the sensing subunit and the video acquisition module, is used to receive the initial alarm signal and the structured recognition result, perform local spatiotemporal correlation and intelligent analysis, and obtain the fused preliminary alarm data. The cloud-based analysis module is connected to the edge analysis module to receive and aggregate the preliminary alarm data, perform multi-source information correlation verification, credibility assessment and risk prediction analysis, and obtain confirmed high-confidence alarm and risk warning information. The data monitoring module is connected to the cloud analysis module to receive the high-confidence alarms and risk warnings, and pushes the alarm information to relevant responsible personnel according to preset rules, while tracking and recording the handling process.

[0005] Preferably, the sensing subunit includes a vibration sensor unit and a signal conditioning unit; The vibration sensor unit includes multiple vibration sensors that are distributed and fixedly installed at the connection between the bridge bottom plate and the guardrail, for real-time acquisition of simulated electrical signals of vibration caused by the impact of foreign objects; The signal conditioning unit is connected to the vibration sensor unit and is used to amplify, filter, and perform analog-to-digital conversion on the vibration analog electrical signal to obtain a digitized vibration signal.

[0006] Preferably, the video acquisition module includes a camera unit and an intelligent analysis unit; The camera unit includes surveillance cameras installed on both sides of the railway line, used to continuously acquire video image data covering the railway construction clearance area; The intelligent analysis unit is connected to the camera unit and has an embedded AI processing unit loaded with a foreign object recognition algorithm. It is used to decode and standardize the video image data, and use the foreign object recognition algorithm to perform target detection and intrusion judgment to obtain a structured recognition result containing target category, location coordinates and timestamp.

[0007] Preferably, the edge analysis module includes a collision sensing unit and a local computing unit; The collision sensing unit includes an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, and an edge computing subunit; The energy harvesting subunit is used to harvest energy from ambient light and bridge vibration; The energy management subunit is connected to the energy harvesting subunit and is used to perform power point tracking and voltage conversion regulation on the acquired energy; The energy storage subunit is connected to the energy management subunit and is used to store electrical energy and supply power to the collision sensing terminal unit. The sensing subunit is used to collect and output the initial alarm signal; The edge computing subunit is connected to the sensing subunit; The local computing unit is connected to the video acquisition module and the edge computing subunit, and is used to run data fusion and event analysis algorithms to perform spatiotemporal correlation matching and intelligent analysis on the initial alarm signal and the structured recognition result to obtain the preliminary alarm data.

[0008] Preferably, the edge computing subunit includes an AI subunit and a hardware acceleration subunit; The AI ​​subunit incorporates a lightweight multimodal feature fusion network model; The hardware acceleration subunit provides computational support for the AI ​​subunit; The AI ​​subunit is used to extract features from the initial alarm signal and perform feature layer fusion and correlation confidence calculation with the structured recognition results transmitted by the local computing unit.

[0009] Preferably, the cloud-based analysis module includes a data fusion unit, a credibility assessment unit, and a risk prediction unit; The data fusion unit is used to perform timestamp alignment and spatial coordinate normalization on multiple received preliminary alarm data, and to perform pairing and correlation judgment based on preset spatiotemporal correlation rules, calculate correlation confidence, and obtain alarm events that have been cross-validated by multiple sources. The credibility assessment unit is connected to the data fusion unit and is used to perform a comprehensive credibility assessment on the alarm event based on the association confidence and the consistency index of multi-source information, filter low-confidence alarms, and obtain the high-confidence alarms. The risk prediction unit is used to build a prediction model based on historical alarm data, environmental data, and operational data using time series analysis and machine learning algorithms, and output a risk probability prediction for foreign object intrusion events occurring in specific future periods and areas, thereby obtaining the risk warning information.

[0010] Preferably, the prediction model includes a temporal feature extraction branch, a spatial feature extraction branch, and a spatiotemporal feature fusion and prediction output layer; The time feature extraction branch is used to mine the time-series patterns of historical alarm data; The spatial feature extraction branch is used to analyze the spatial risk correlation between different monitoring points; The spatiotemporal feature fusion and prediction output layer is used to deeply fuse the extracted temporal and spatial features and output the risk probability prediction value and potential high-risk areas and time periods.

[0011] Preferably, the data monitoring module includes an alarm push unit and a process management unit; The alarm push unit is used to automatically select and combine multiple notification channels according to the event type and risk level in the high-confidence alarm and risk warning information, based on the preset hierarchical alarm push mechanism, and push alarm information containing event details and access links to the corresponding responsible personnel. The process management unit is used to establish a handling tracking file for each alarm message, recording the entire process status and time from instruction issuance and personnel signature to result feedback, forming a traceable closed-loop management log.

[0012] Preferably, the alarm push unit has a preset alarm-response rule table, which defines the mapping relationship between event type, risk level and push object, push channel and response time limit; for extremely high risk alarms, the alarm push unit is configured to trigger multi-channel strong reminder notifications to multiple types of responsible personnel at the same time.

[0013] Preferably, the data monitoring module further includes a geographic information linkage unit connected to the alarm push unit, used to associate alarm location information with an electronic map and highlight the alarm location on the electronic map.

[0014] Compared with the prior art, the present invention has the following advantages and technical effects: The foreign object intrusion alarm and detection system for road-rail overpasses provided by this invention deploys a sensor network and an intelligent video analysis terminal. It performs spatiotemporal fusion and preliminary intelligent discrimination of multi-source information at the edge, followed by deep correlation verification, credibility assessment, and risk prediction via a cloud platform. Finally, it achieves hierarchical and precise push notifications and closed-loop process management through a data monitoring center. This system effectively overcomes the shortcomings of existing technologies, significantly improving the real-time performance, accuracy, and reliability of intrusion event monitoring, and reducing the false alarm rate. Through the combination of edge computing and cloud intelligence, it enhances adaptability to complex environments and the overall intelligence level of the system, achieving a leap from passive alarm to proactive risk warning. Simultaneously, the standardized emergency response closed-loop process greatly improves handling efficiency, providing strong technical support for the safe operation of road-rail overpasses. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the functional composition of the collision alarm sensing unit according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the functional composition of the video intelligent analysis unit in an embodiment of the present invention; Figure 4 This is a system function interaction diagram according to an embodiment of the present invention. Detailed Implementation

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0018] like Figure 1 and Figure 4 As shown, this embodiment provides a foreign object intrusion alarm and detection system for a road-rail overpass, including: The sensing subunits are deployed at key locations on the bridge to detect collisions with foreign objects and generate initial alarm signals. The video acquisition module is used to capture visual information of the monitored area and perform real-time analysis of the video stream to identify foreign object intrusion behavior and obtain structured recognition results containing target information. The edge analysis module, connected to the sensing subunit and video acquisition module, is used to receive initial alarm signals and structured recognition results, perform local spatiotemporal correlation and intelligent analysis, and obtain preliminary alarm data after fusion. The cloud-based analytics module communicates with the edge analytics module to receive and aggregate preliminary alarm data, perform multi-source information correlation verification, credibility assessment and risk prediction analysis, and obtain confirmed high-confidence alarms and risk warning information. The data monitoring module communicates with the cloud analysis module to receive high-confidence alarms and risk warnings, and pushes the alarm information to relevant responsible personnel according to preset rules, while tracking and recording the handling process.

[0019] Furthermore, the sensing subunit includes a vibration sensor unit and a signal conditioning unit; The vibration sensor unit includes multiple vibration sensors that are distributed and fixedly installed at the connection between the bridge base plate and the guardrail, used to collect simulated electrical signals of vibration caused by the impact of foreign objects in real time; The signal conditioning unit is connected to the vibration sensor unit and is used to amplify, filter, and convert the analog vibration signal to digital form to obtain a digitized vibration signal.

[0020] Furthermore, in this embodiment, the sensing subunit is a sensor network, which is deployed at key locations on the bridge to detect collisions with foreign objects and generate alarm signals. Specifically, the system's on-site data acquisition component consists of a collision sensor array and a high-definition video surveillance network. Based on the bridge's structural mechanical characteristics, the collision sensor array is distributed and fixedly installed at key locations such as the bridge's base plate and guardrail connections to collect vibration signals caused by foreign object impacts in real time.

[0021] The sensing process includes signal acquisition and preprocessing, feature extraction, and initial event judgment. Analog electrical signals are amplified, filtered, and converted from analog to digital by signal conditioning circuits. Subsequently, the digital signal undergoes time-domain and frequency-domain analysis to extract features including signal amplitude, duration, dominant frequency components, and energy integral. The signal energy E can be obtained through the integral formula... Estimation is performed within a time window T, where x(t) is the time-domain vibration signal. The extracted feature vector is compared with a preset threshold or a lightweight classification model. If the features match the preset impact pattern, preliminary alarm data containing timestamps, location identifiers, and feature vectors is generated.

[0022] The cameras in the high-definition video surveillance network are installed on poles on both sides of the railway line. Their field of view is precisely designed to ensure that there are no blind spots in the entire railway construction clearance area under the bridge, so as to continuously acquire video image data of the monitored area.

[0023] Furthermore, the video acquisition module includes a camera unit and an intelligent analysis unit; The camera unit includes surveillance cameras installed on both sides of the railway line to continuously acquire video image data covering the railway construction clearance area; The intelligent analysis unit is connected to the camera unit and has an embedded AI processing unit loaded with a foreign object recognition algorithm. It is used to decode and standardize video image data, and use the foreign object recognition algorithm to perform target detection and intrusion judgment to obtain structured recognition results containing target category, location coordinates and timestamp.

[0024] Furthermore, the video acquisition module in this embodiment is a video acquisition intelligent analysis terminal, which includes a camera deployed at the monitoring location and an intelligent analysis terminal, used to capture visual information and identify foreign object intrusion behavior in real time; The video intelligent analysis terminal performs real-time analysis of the surveillance video stream to identify intruding foreign objects. The process includes image preprocessing and target detection, intrusion determination, and result structuring. After decoding and image standardization, the video stream uses a deployed deep learning target detection model for forward inference to locate potential foreign objects and output their bounding boxes and initial confidence scores. To determine intrusion, the bounding box of the detected target is mapped to a pre-defined railway construction clearance area, and the overlap area ratio between the target and this area is calculated. If the overlap area ratio satisfies... The conditions, among which If the set overlap threshold is not met, it is determined to be an intrusion. The final output is a structured recognition result containing the target category, timestamp, intrusion location coordinates, and evidence image.

[0025] Furthermore, the edge analysis module includes a collision sensing unit and a local computing unit; The collision sensing terminal unit includes an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, an edge computing subunit, and a communication subunit; The energy harvesting subunit is used to harvest energy from ambient light and bridge vibration; The energy management subunit is connected to the energy harvesting subunit and is used to perform power point tracking and voltage conversion regulation on the harvested energy; The energy storage subunit is connected to the energy management subunit and is used to store electrical energy and power the collision sensing terminal unit; The sensing subunit is used to collect and output the initial alarm signal; The edge computing subunit is connected to the sensing subunit; The local computing unit is connected to the video acquisition module and the edge computing subunit to run data fusion and event analysis algorithms, perform spatiotemporal correlation matching and intelligent analysis on the initial alarm signal and structured recognition results, and obtain preliminary alarm data.

[0026] The communication subunit is used to send preliminary alarm data wirelessly and to perform protocol adaptation to ensure compatibility with the receiving end.

[0027] Furthermore, the edge computing subunit includes an AI subunit and a hardware acceleration subunit; The AI ​​subunit incorporates a lightweight multimodal feature fusion network model; The hardware acceleration subunit provides computational support for the AI ​​subunit; The AI ​​subunit is used to extract features from the initial alarm signal and perform feature layer fusion and correlation confidence calculation with the structured recognition results transmitted from the local computing unit.

[0028] Furthermore, such as Figure 2 As shown, the edge analysis in this embodiment is based on the edge processing unit deployed on site, including a collision alarm sensing end and a local computing unit. The collision alarm sensing end integrates an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, an edge computing subunit, and a communication subunit. The energy harvesting subunit includes solar photovoltaic panels and a vibration energy harvesting device for harvesting energy from the environment; The energy management subunit is used to regulate electrical energy; The energy storage subunit is a battery or supercapacitor, used to store energy to maintain the continuous operation of the system.

[0029] Furthermore, the AI ​​subunit in this embodiment has a built-in data fusion and event analysis algorithm model, which runs with the support of the hardware acceleration subunit. It performs temporal and spatial correlation matching and intelligent analysis on sensor-triggered events and video recognition results, thereby achieving local identification and classification of intrusion events.

[0030] Specifically, the edge computing part of the system includes a collision alarm sensing terminal and a video intelligent analysis terminal installed on the field side.

[0031] The collision alarm sensing unit has a built-in signal conditioning circuit, microprocessor and communication module. Its signal input terminal is connected to the collision sensor. Specifically, it includes an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, an edge computing subunit and a communication subunit.

[0032] The energy harvesting subunit is composed of a solar photovoltaic panel and a vibration energy harvesting device, and is used to harvest energy from ambient light and bridge vibration. The energy management subunit is connected to the energy harvesting subunit and is used to perform maximum power point tracking, voltage conversion and power distribution on the harvested energy. The energy storage subunit is connected to the energy management subunit and uses a rechargeable battery to store electrical energy and provide a stable operating voltage for other modules in the terminal. The sensing subunit integrates a vibration sensor to convert mechanical vibrations into analog electrical signals; The input of the edge computing subunit is connected to the perception subunit. Its AI subunit, supported by the hardware acceleration subunit, performs feature extraction and intelligent analysis on the digital signals converted by the perception subunit. It then compares the analysis results with preset thresholds or models to generate preliminary collision alarm event data. This AI subunit employs a lightweight multimodal feature fusion network model with built-in data fusion and event analysis algorithms. It performs temporal and spatial correlation matching and intelligent analysis of sensor-triggered events and video recognition results, enabling local identification and classification of intrusion events.

[0033] The model includes vibration signal feature extraction and image feature extraction branches, and performs pruning and quantization optimization for edge devices. The outputs of the two branches are attention-weighted and fused at the feature layer to fuse the features. The calculation can be expressed as ,in and These are vibration and image feature vectors, and attention weights, respectively. It is dynamically generated from a lightweight subnetwork.

[0034] During the analysis, for a vibration sensor alarm event A video recognition event The system calculates its absolute time difference. Spatial coordinate mapping is accomplished through preset calibration parameters: the pixel coordinates in the video recognition results are transformed using perspective based on camera intrinsic and extrinsic parameters, and then back-projected onto the bridge's global three-dimensional coordinate system to obtain the physical coordinates. Location of the vibration event This refers to the preset sensor installation coordinates. The spatial distance between the two... Using the Euclidean distance formula calculate.

[0035] Based on the aligned spatiotemporal data, the algorithm performs a two-level association decision. First, a rule-based hard screening is conducted: the system presets a sliding time window. and a spatial overlap threshold If the event satisfies and If the conditions are met, the event is considered a potential related event pair and proceeds to the next stage; otherwise, it is excluded.

[0036] Subsequently, events that pass the initial screening are subjected to model-based soft decision-making and intelligent association. This process is accomplished by a lightweight multimodal feature fusion network model. This model receives not only the spatiotemporal parameters of the initial screening decision, but also... It further integrates the time-frequency feature vector extracted from the original vibration signal. and visual feature vectors extracted from video images The model learns and models complex spatiotemporal correlation patterns through its built-in attention mechanism and other structures. For patterns at the threshold boundary, i.e. and near and For fuzzy interval events, the model can transcend the limitations of fixed thresholds and perform more refined evaluations. Its final output is a comprehensive association confidence score. , where M is the model inference function and Θ is the model parameters. This score quantifies the probability that two events originate from the same intrusion source. If If the threshold value is exceeded based on historical data, the system confirms the correlation between the two and automatically labels the fused alarm event with the specific foreign object category and its confidence level based on the results output by the video recognition branch, thereby generating highly reliable local preliminary alarm data.

[0037] The communication subunit connects to the edge computing subunit to reliably and efficiently transmit locally generated preliminary alarm data to the cloud platform. Its core consists of a low-power wide-area network communication module and its driving circuitry, responsible for wireless data transmission and reception, as well as bidirectional adaptation of communication protocols.

[0038] More specifically, the video intelligent analysis terminal in this embodiment integrates a dedicated computing unit with AI reasoning capabilities. This unit is loaded with a trained foreign object recognition algorithm model, which can perform real-time decoding and online analysis of the video stream transmitted from the connected camera, automatically identify foreign object targets that intrude into the boundary in the image, and output a structured recognition result including target type, location coordinates, timestamp, and evidence image.

[0039] Furthermore, the cloud-based analytics module includes a data fusion unit, a credibility assessment unit, and a risk prediction unit; The data fusion unit is used to perform timestamp alignment and spatial coordinate normalization on multiple received preliminary alarm data, and to perform pairing and correlation judgment based on preset spatiotemporal correlation rules, calculate correlation confidence, and obtain alarm events that have been cross-validated by multiple sources. The credibility assessment unit is connected to the data fusion unit and is used to conduct a comprehensive credibility assessment of alarm events based on the correlation confidence and the consistency index of multi-source information, filter low-confidence alarms, and obtain high-confidence alarms. The risk prediction unit is used to build a prediction model based on historical alarm data, environmental data, and operational data, using time series analysis and machine learning algorithms. It outputs a risk probability prediction for foreign object intrusion events that will occur in specific future periods and areas, and obtains risk warning information.

[0040] Furthermore, the prediction model includes a temporal feature extraction branch, a spatial feature extraction branch, and a spatiotemporal feature fusion and prediction output layer; The time feature extraction branch is used to mine time-series patterns in historical alarm data; The spatial feature extraction branch is used to analyze the spatial risk correlation between different monitoring points; The spatiotemporal feature fusion and prediction output layer is used to deeply fuse the extracted temporal and spatial features and output the risk probability prediction value and potential high-risk areas and time periods.

[0041] Furthermore, in this embodiment, the cloud analysis module is specifically a cloud analysis platform. The cloud analysis platform is used to receive and aggregate data reported from various endpoints, and to perform correlation verification, credibility assessment, risk prediction, and trend analysis of multi-source alarm information. Specifically, the data fusion and event analysis algorithm includes aligning sensor signals and video recognition results with timestamps and mapping spatial coordinates, determining whether they belong to the same intrusion event by setting a sliding time window and a spatial overlap threshold, and automatically labeling alarm categories and confidence levels based on video recognition results.

[0042] The cloud-based analytics platform is equipped with predictive models that integrate time series analysis and spatial correlation algorithms to analyze the correlation between historical alarm data patterns and environmental factors, enabling predictive warnings for potential risk areas and high-incidence periods.

[0043] The cloud-based analysis platform in this embodiment can dynamically adjust the sensing sensitivity or sampling frequency of the end-side analysis based on the prediction results, and issue risk alerts to the maintenance department.

[0044] The system's cloud-based data processing component is deployed on a remote server cluster, primarily responsible for the fusion analysis and risk assessment of multi-source information. First, it receives alarm events and identification results reported from various edge terminals. Based on the time and space information of the event, it performs logical comparison and cross-validation of correlated alarms originating from vibration sensors and video analysis terminals. By applying the consistency principle of multi-source information, it comprehensively determines whether an intrusion event has actually occurred, and filters out single false alarms caused by environmental interference and other factors. Second, based on long-term accumulated historical alarm data, environmental monitoring data, and train operation data, it uses machine learning algorithms to build and continuously optimize predictive models to assess the potential risk probability of foreign object intrusion events under different external conditions and generate corresponding risk warning signals.

[0045] Specifically, the predictive model deployed on the cloud-based analytics platform is a hybrid intelligent model that integrates time series analysis and spatial correlation analysis, aiming to achieve accurate prediction and early warning of foreign object intrusion risks. Structurally, the model mainly includes a time feature extraction branch, a spatial feature extraction branch, and a spatiotemporal feature fusion and prediction output layer.

[0046] In the time dimension, the model integrates time series analysis algorithms to uncover temporal patterns and periodic regularities hidden in historical alarm data. This branch receives multi-dimensional input features arranged in time series, including but not limited to the number, type, and timestamps of historical alarm events, as well as associated environmental factor time series, such as wind speed, precipitation, and visibility. The model constructs features through a sliding time window and employs an attention-based temporal encoder or long short-term memory network to capture long-term and short-term dependencies. Its core process can be characterized as processing historical state sequences... The encoding is used to extract condensed temporal feature representations. .

[0047] In the spatial dimension, the model integrates spatial correlation algorithms to analyze and quantify the spatial correlation of risks between different monitoring points. This branch models the bridge and its surrounding area as a graph structure, where nodes represent monitoring points and edges represent spatial adjacency or functional associations between points. The spatial correlation algorithm learns the risk propagation and diffusion patterns between nodes by calculating spatial autocorrelation indices or utilizing graph neural networks. For any two points... and Its spatial correlation strength Based on geographical distance Topological relationships are obtained through kernel functions or learning, such as using a Gaussian kernel function. The output of the spatial feature extraction branch is a feature representation that reflects the global spatial risk distribution. .

[0048] The improvement of the prediction model involved in this embodiment lies in its deep fusion mechanism of spatiotemporal features. The model does not simply concatenate time and spatial features, but rather achieves cross-dimensional interaction and enhancement through a spatiotemporal fusion module. This module receives time-series features. Spatial features Furthermore, spatiotemporal joint features are computed through methods such as cross-attention mechanisms or tensor fusion. This joint feature simultaneously encodes coupled information on "when" and "where" the risk is higher. Finally, the prediction output layer decodes based on the zst method, outputting a predicted probability of foreign object intrusion events occurring in each monitoring area or across the entire bridge within a specific future time period. The model identifies potential high-risk areas and peak periods. Its training objective is to minimize the difference between the predicted risk probability and the actual occurrence of alerts, continuously optimizing it through iterative analysis using historical data.

[0049] This embodiment integrates time series analysis and spatial correlation algorithms. The prediction model can not only rely on the historical patterns of a single node, but also combine the spatial correlation of the entire network to make a comprehensive judgment. This enables more accurate and reliable predictive early warning of potential risk areas and high-incidence periods, providing intelligent decision support for preventive maintenance and dynamic resource allocation.

[0050] The cloud-based analytics platform in this embodiment processes multi-source alarm information following a progressive process of correlation verification, credibility assessment, risk prediction, and trend analysis. In the correlation verification stage, the platform first performs timestamp alignment and spatial coordinate normalization on alarm events from different edge analysis units, unifying various events under the same spatiotemporal reference system. Subsequently, based on preset spatiotemporal correlation rules, vibration sensing alarms and video recognition alarms occurring within similar time windows and in adjacent or identical monitoring areas are paired. The correlation determination comprehensively considers the time difference. Spatial distance and through a fusion function Calculate the association confidence, where and These represent the characteristic intensity of the vibration signal and the confidence level of video recognition, respectively. If the C value exceeds a set threshold, it is determined to be multi-source evidence of the same intrusion event and is associated with it.

[0051] During the credibility assessment phase, the system performs cross-validation and comprehensive analysis on the associated multi-source alarms. The assessment model considers not only the quality of individual sensor data but also emphasizes the degree of consistency among multi-source information. Consistency Indicators The final credibility can be calculated by comparing the degree of agreement between descriptions of the same event from different sources. The association confidence level C and the consistency index Weighted synthesis, i.e. The weighting coefficient and This assessment is learned from historical data. The results are used to filter out isolated, low-reliability alarms caused by occasional environmental disturbances.

[0052] For risk prediction and trend analysis, the platform integrates historical alarm databases, environmental data, and operational data. The risk prediction module utilizes time series analysis and machine learning algorithms to build predictive models. The model outputs the probability of a foreign object intrusion event occurring in a specific time period and a specific bridge section. Meanwhile, the trend analysis module clusters and statistically analyzes historical alarm events to identify periodic changes in alarm frequency, migration patterns of high-incidence areas, and the evolution trend of the proportion of various intrusive foreign objects, providing a basis for decision-making in preventive maintenance and resource deployment.

[0053] Furthermore, the data monitoring module includes an alarm push unit and a process management unit; The alarm push unit is used to automatically select and combine multiple notification channels according to the event type and risk level in high-confidence alarms and risk warning information, based on the preset hierarchical alarm push mechanism, and push alarm information containing event details and access links to the relevant responsible personnel. The process management unit is used to establish a handling tracking file for each alarm message, recording the status and time of the entire process from instruction issuance and personnel signature to result feedback, forming a traceable closed-loop management log.

[0054] Furthermore, the alarm push unit has a preset alarm-response rule table, which defines the mapping relationship between event type, risk level and push target, push channel and response time limit; for extremely high risk alarms, the alarm push unit is configured to trigger multi-channel strong reminder notifications to multiple types of responsible personnel at the same time.

[0055] Furthermore, the data monitoring module also includes a geographic information linkage unit, which is connected to the alarm push unit to associate alarm location information with the electronic map and highlight the alarm location on the electronic map.

[0056] Furthermore, the data monitoring module in this embodiment is a data monitoring center. The data monitoring center is used to receive alarm events and analysis conclusions confirmed by the cloud, and push alarm information to relevant responsible personnel according to a preset process, while tracking and recording the handling process.

[0057] Specifically, the data monitoring center has a tiered alarm push mechanism that can automatically select and combine one or more notification channels, such as SMS, application messages, platform pop-ups, or audible and visual alarms, based on the risk level of the alarm event, to ensure that information is delivered to the responsible personnel with the appropriate permissions in a timely manner.

[0058] Specifically, the tiered alarm push mechanism is implemented through a preset alarm-response rule table, which defines the mapping relationship between event type, risk level and push target, push channel and response time limit; for extremely high risk alarms, the system simultaneously triggers strong reminders to multiple types of responsible personnel through multiple channels.

[0059] Specifically, all push notifications include a standardized event summary and a link to detailed data access, and are linked with a geographic information system to highlight the alarm location on an electronic map.

[0060] Specifically, the data monitoring center records the entire process from alarm push to handling feedback, forming a traceable closed-loop management log for response performance analysis and system optimization.

[0061] Specifically, the system's monitoring and command and control section is equipped with an integrated monitoring platform, which is used to centrally display system status, real-time alarms, risk warnings, and handling progress.

[0062] When the platform receives a valid alarm or high-risk warning confirmed by the cloud data processing department, it automatically sends the alarm details and handling instructions to the relevant personnel responsible for line inspection, equipment maintenance and train dispatch through multiple channels such as SMS, dedicated application messages and audio-visual prompts, in accordance with the predefined emergency plan and through the integrated communication gateway.

[0063] The system's built-in process management module establishes an independent handling tracking file for each alarm message, recording the entire process from instruction issuance, personnel signature, on-site handling to result feedback and its precise time, forming a traceable and auditable complete handling closed loop to ensure that security incidents can be effectively responded to and handled within the specified time limit.

[0064] The foreign object intrusion alarm and detection system for railway-highway overpasses provided in this embodiment forms a complete closed loop of monitoring, analysis, early warning and handling through the collaborative operation of multiple levels, including on-site perception, edge computing, cloud analysis and command and dispatch.

[0065] As a preferred implementation method, such as Figures 1 to 4 As shown, this embodiment provides a foreign object intrusion alarm and detection system for a road-rail overpass. The system includes a collaborative sensor network, a video acquisition and intelligent analysis terminal, an edge analysis system, a cloud analysis platform, and a data monitoring center. The sensor network is deployed at key locations on the bridge to detect foreign object collisions and generate alarm signals. The video acquisition and intelligent analysis terminal captures visual information through cameras deployed at optimal monitoring positions and uses an integrated intelligent analysis terminal to perform real-time analysis of the video stream to identify various foreign object intrusion behaviors. When the data monitoring center receives a valid alarm or high-risk warning confirmed by the cloud, it automatically pushes alarm details and handling instructions to relevant personnel simultaneously through an integrated communication gateway.

[0066] The edge-side analysis unit, deployed on-site, comprises a collision alarm sensing unit and a video intelligent analysis unit. The collision alarm sensing unit further integrates an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, an edge computing subunit, and a communication subunit. The energy harvesting subunit utilizes solar photovoltaic panels and vibration energy harvesting devices to extract energy from the environment; the energy management subunit is responsible for power regulation and testing, including charging logic, discharging logic, and various testing units; the energy storage subunit, typically a battery or supercapacitor, stores energy to ensure continuous system operation; the sensing subunit connects to and processes signals from the sensor network, including vibration sensors and signal conditioning circuitry; the edge computing subunit embeds AI and hardware acceleration subunits, performing local data fusion and logical judgment on alarm signals from sensors and visual recognition results reported by the video analysis terminal to generate preliminary local alarms; the communication subunit is responsible for data interaction with the cloud platform, supporting wireless transmission and protocol adaptation.

[0067] like Figure 3 As shown, the video intelligent analysis unit is used to acquire video streams and identify foreign objects. It includes an energy harvesting subunit composed of solar photovoltaic panels, a charge / discharge controller, an energy storage device, a monitoring camera, and an intelligent analysis terminal. The charge / discharge controller manages the energy of the energy harvesting subunit, realizing MPPT tracking, power distribution, and voltage regulation; the energy storage device stores electrical energy; the monitoring camera is used for high-definition video acquisition and all-weather monitoring; the intelligent analysis terminal embeds an AI processing unit and a foreign object identification model, which is used to perform real-time analysis of the video stream and output structured foreign object intrusion identification results.

[0068] One of the sources of on-site data collection for the system is the encroachment data collection terminal, which is responsible for collecting and monitoring the status of data along the railway line and the bridge clearance area, providing basic data input for cloud analysis.

[0069] The cloud-based analytics platform receives and aggregates preliminary alarm events and structured identification results reported by various edge analysis units and intrusion data acquisition terminals. It first performs in-depth multi-source data fusion analysis. Using precise timestamp alignment and spatial coordinate mapping technology, the platform performs spatiotemporal correlation matching and logical verification on collision alarms from the vibration sensor network and foreign object identification events from the video intelligent analysis terminal. Based on preset fusion rules, it completes cross-validation and comprehensive credibility assessment of multi-source information, thereby outputting a filtered and confirmed high-confidence final alarm.

[0070] Building upon this data fusion, the platform further connects to or integrates with existing intelligent analysis and control platforms for railway cross-railways, enabling standardized push and sharing of alarm information, event locations, and on-site images. This supports collaborative emergency response and resource allocation for railway train dispatching, track maintenance, and other systems. Simultaneously, based on accumulated historical alarm data, environmental monitoring data, and train operation plans, the platform utilizes built-in machine learning models to conduct risk trend analysis and prediction, and synchronizes predictive warnings to the control platform, providing decision support for safety risk prevention and preventative maintenance across the entire railway network. Finally, the platform confirms alarm and warning signals and their analysis conclusions, sending them to the system's data monitoring center.

[0071] The data monitoring center is responsible for the final reception and processing of alarm information. It receives alarms from the cloud in real time and, according to preset procedures, promptly and accurately pushes information including event details, location, evidence, and risk level to relevant personnel such as line inspection, equipment maintenance, and train dispatching personnel through multiple channels. At the same time, it tracks and records the entire processing loop, providing support for system management and optimization.

[0072] The edge computing subunit is composed of a processing unit integrating AI functionality and a dedicated hardware acceleration unit. This AI subunit incorporates an optimized and lightweight deep learning model specifically designed for feature extraction and pattern recognition of real-time sensor signals and video analysis results, enabling intelligent classification of foreign objects and target association analysis. To meet real-time requirements, the accompanying hardware acceleration subunit provides high-performance parallel computing support for the AI ​​model, significantly improving inference speed and energy efficiency. In actual operation, this combination first performs millisecond-level time synchronization and spatial coordinate unification of multi-source information. Then, with the support of the acceleration module, the AI ​​subunit quickly calculates and determines the correlation between vibration events and video recognition targets within a preset spatiotemporal window. Based on the confidence level, it outputs the locally fused judgment result, thereby efficiently completing the local identification and preliminary classification of intrusion events.

[0073] Further optimization of the solution involves a cloud-based analytics platform that integrates time-series analysis with spatial correlation algorithms. By analyzing historical alarm data patterns across monitoring nodes and their correlation with environmental factors, it enables predictive early warnings for potential risk areas and high-risk periods. The platform can dynamically adjust the sensing sensitivity or sampling frequency of the edge analysis based on the prediction results and issue risk alerts to maintenance departments in advance, extending the response from passive alarms to proactive prevention.

[0074] To further optimize the solution, the data monitoring center features a tiered alarm push mechanism. Based on the risk level of the alarm event, it automatically selects and combines different notification channels, such as SMS, dedicated application messages, platform interface pop-ups, and even audible and visual alarms, ensuring that warning information at different levels is delivered to the responsible personnel with the appropriate permissions in a timely manner, thereby guaranteeing the timeliness and effectiveness of emergency response. This mechanism is specifically implemented through a pre-set alarm-response rule table, which defines the mapping relationship between different event types, risk levels, push targets, push channels, and response time limits. For extremely high-risk alarms, such as large foreign objects intruding into the clearance area, the system will simultaneously trigger strong multi-channel alerts to the dispatch center and on-site inspection personnel. All push messages include a standardized event summary and a link to access detailed data, and can be linked with a geographic information system to highlight the alarm location on an electronic map. Simultaneously, the data monitoring center records the entire process from alarm push to handling feedback, forming a traceable closed-loop management log for subsequent response performance analysis and system optimization.

[0075] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A foreign object intrusion alarm and detection system for a road-rail overpass, characterized in that, include: The sensing subunits are deployed at key locations on the bridge to detect collisions with foreign objects and generate initial alarm signals. The video acquisition module is used to capture visual information of the monitored area and perform real-time analysis of the video stream to identify foreign object intrusion behavior and obtain structured recognition results containing target information. The edge analysis module, connected to the sensing subunit and the video acquisition module, is used to receive the initial alarm signal and the structured recognition result, perform local spatiotemporal correlation and intelligent analysis, and obtain the fused preliminary alarm data. The cloud-based analysis module is connected to the edge analysis module to receive and aggregate the preliminary alarm data, perform multi-source information correlation verification, credibility assessment and risk prediction analysis, and obtain confirmed high-confidence alarm and risk warning information. The data monitoring module is connected to the cloud analysis module to receive the high-confidence alarms and risk warnings, and pushes the alarm information to relevant responsible personnel according to preset rules, while tracking and recording the handling process.

2. The system according to claim 1, characterized in that, The sensing subunit includes a vibration sensor unit and a signal conditioning unit; The vibration sensor unit includes multiple vibration sensors that are distributed and fixedly installed at the connection between the bridge bottom plate and the guardrail, for real-time acquisition of simulated electrical signals of vibration caused by the impact of foreign objects; The signal conditioning unit is connected to the vibration sensor unit and is used to amplify, filter, and perform analog-to-digital conversion on the vibration analog electrical signal to obtain a digitized vibration signal.

3. The system according to claim 1, characterized in that, The video acquisition module includes a camera unit and an intelligent analysis unit; The camera unit includes surveillance cameras installed on both sides of the railway line, used to continuously acquire video image data covering the railway construction clearance area; The intelligent analysis unit is connected to the camera unit and has an embedded AI processing unit loaded with a foreign object recognition algorithm. It is used to decode and standardize the video image data, and use the foreign object recognition algorithm to perform target detection and intrusion judgment to obtain a structured recognition result containing target category, location coordinates and timestamp.

4. The system according to claim 1, characterized in that, The edge analysis module includes a collision sensing unit and a local computing unit; The collision sensing unit includes an energy harvesting subunit, an energy management subunit, an energy storage subunit, a sensing subunit, and an edge computing subunit; The energy harvesting subunit is used to harvest energy from ambient light and bridge vibration; The energy management subunit is connected to the energy harvesting subunit and is used to perform power point tracking and voltage conversion regulation on the acquired energy; The energy storage subunit is connected to the energy management subunit and is used to store electrical energy and supply power to the collision sensing terminal unit. The sensing subunit is used to collect and output the initial alarm signal; The edge computing subunit is connected to the sensing subunit; The local computing unit is connected to the video acquisition module and the edge computing subunit, and is used to run data fusion and event analysis algorithms to perform spatiotemporal correlation matching and intelligent analysis on the initial alarm signal and the structured recognition result to obtain the preliminary alarm data.

5. The system according to claim 4, characterized in that, The edge computing subunit includes an AI subunit and a hardware acceleration subunit; The AI ​​subunit incorporates a lightweight multimodal feature fusion network model; The hardware acceleration subunit provides computational support for the AI ​​subunit; The AI ​​subunit is used to extract features from the initial alarm signal and perform feature layer fusion and correlation confidence calculation with the structured recognition results transmitted by the local computing unit.

6. The system according to claim 1, characterized in that, The cloud-based analytics module includes a data fusion unit, a credibility assessment unit, and a risk prediction unit. The data fusion unit is used to perform timestamp alignment and spatial coordinate normalization on multiple received preliminary alarm data, and to perform pairing and correlation judgment based on preset spatiotemporal correlation rules, calculate correlation confidence, and obtain alarm events that have been cross-validated by multiple sources. The credibility assessment unit is connected to the data fusion unit and is used to perform a comprehensive credibility assessment on the alarm event based on the association confidence and the consistency index of multi-source information, filter low-confidence alarms, and obtain the high-confidence alarms. The risk prediction unit is used to build a prediction model based on historical alarm data, environmental data, and operational data using time series analysis and machine learning algorithms, and output a risk probability prediction for foreign object intrusion events occurring in specific future periods and areas, thereby obtaining the risk warning information.

7. The system according to claim 6, characterized in that, The prediction model includes a temporal feature extraction branch, a spatial feature extraction branch, and a spatiotemporal feature fusion and prediction output layer; The time feature extraction branch is used to mine the time-series patterns of historical alarm data; The spatial feature extraction branch is used to analyze the spatial risk correlation between different monitoring points; The spatiotemporal feature fusion and prediction output layer is used to deeply fuse the extracted temporal and spatial features and output the risk probability prediction value and potential high-risk areas and time periods.

8. The system according to claim 1, characterized in that, The data monitoring module includes an alarm push unit and a process management unit; The alarm push unit is used to automatically select and combine multiple notification channels according to the event type and risk level in the high-confidence alarm and risk warning information, based on the preset hierarchical alarm push mechanism, and push alarm information containing event details and access links to the corresponding responsible personnel. The process management unit is used to establish a handling tracking file for each alarm message, recording the entire process status and time from instruction issuance and personnel signature to result feedback, forming a traceable closed-loop management log.

9. The system according to claim 8, characterized in that, The alarm push unit has a preset alarm-response rule table, which defines the mapping relationship between event type, risk level and push target, push channel and response time limit; for extremely high risk alarms, the alarm push unit is configured to trigger multi-channel strong reminder notifications to multiple types of responsible personnel at the same time.

10. The system according to claim 1, characterized in that, The data monitoring module also includes a geographic information linkage unit, which is connected to the alarm push unit to associate alarm location information with an electronic map and highlight the alarm location on the electronic map.

Images