A cable tunnel edge side multi-modal fusion acquisition and cooperative early warning method

Abstract

The application discloses a kind of cable tunnel edge side multimodal fusion acquisition and collaborative early warning method, belong to power cable tunnel fire safety monitoring technical field.The present application is aimed at the problems of existing cable fire early warning scheme, such as early warning lag, high false alarm and missed alarm rate of single mode detection, cloud processing response delay, edge computing resource conflict, constructs multi-thread parallel acquisition architecture, synchronously collects visual, olfactory and electrical signal mode data and completes timing alignment;Extracting fire precursor features to construct a multi-modal fusion feature vector, calculate the comprehensive risk index;Match the cable fire development stage to execute four-level collaborative early warning judgment;Adaptive sliding window and hierarchical cooling mechanism are used to realize early warning state management;Video acquisition push and intelligent detection unit are fully decoupled, and parallel operation without interference is realized.The present application can realize early warning of cable fire in whole cycle, greatly advance disposal window period, significantly reduce false alarm rate and compress early warning response time.

Description

Technical Field

[0001] This invention relates to the field of fire safety monitoring technology for power cable tunnels, specifically a method for multimodal fusion acquisition and collaborative early warning at the edge of a cable tunnel. Background Technology

[0002] Cable fires, with thermal runaway at cable joints as the core cause, exhibit a clear, progressive development pattern throughout their entire lifecycle. From early heat accumulation and insulation pyrolysis, to worsening insulation degradation and expanded thermal runaway, culminating in the appearance of open flames and the fire spreading longitudinally along the cable, a complete development process exists. However, existing mainstream solutions, primarily smoke detectors and fixed-temperature sensors, can only respond after open flames and substantial smoke production occur. They are completely unable to detect early warning signs such as heat accumulation and insulation pyrolysis before a fire breaks out, missing the golden window for fire response and failing to achieve early warning and proactive prevention.

[0003] Existing single-mode detection schemes have inherent defects, lacking the ability to fuse multi-dimensional features. Detection modes based on a single gas or a single temperature measurement are susceptible to interference from environmental factors such as tunnel ventilation disturbances and background temperature rise of electrical equipment, resulting in significant false alarms and missed alarms. At the same time, existing systems generally separate electrical monitoring from fire early warning systems, failing to incorporate three-phase current imbalance, which reflects cable insulation deterioration and phase-to-phase faults, into the early warning model, creating significant technical blind spots and making it impossible to improve early warning reliability through cross-validation of multi-dimensional features.

[0004] Existing centralized cloud processing architectures cannot meet the real-time requirements of early warning. Mainstream solutions adopt a centralized architecture of "front-end data acquisition - cloud processing," but underground tunnel networks have poor stability and limited bandwidth, making them prone to data transmission delays, packet loss, or even interruptions. The golden window for dealing with cable fires is only tens of seconds. The transmission and computing delays caused by cloud processing directly lead to delayed early warnings, and in extreme cases, complete failure.

[0005] Meanwhile, existing edge-side solutions cannot meet the dual needs of multi-channel video streaming and high-frequency AI detection. Video acquisition and streaming are deeply coupled with AI detection, constantly competing for the limited computing power of edge devices. The early warning mechanism does not match the full-cycle development pattern of cable fires and lacks an effective data anti-interference and status management mechanism. It is prone to false alarms and repeated reporting due to instantaneous data jitter, and cannot achieve smooth degradation and cancellation of the early warning status, which seriously interferes with operation and maintenance work.

[0006] In summary, existing technologies cannot meet the core fire prevention and control requirements of cable tunnels in special scenarios. Therefore, a multi-modal fusion acquisition and collaborative early warning method for the edge side of cable tunnels is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a method for multimodal fusion acquisition and collaborative early warning at the edge of cable tunnels, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a method for multimodal fusion acquisition and collaborative early warning at the edge of a cable tunnel, comprising the following steps:

[0009] S1. Construct a multi-threaded parallel acquisition architecture, synchronously acquire visual modal data, olfactory modal data and electrical signal modal data through mutually independent acquisition threads, and complete the time sequence alignment of all acquired data through a unified timestamp;

[0010] S2. To address the mechanism of cable tunnel fires, fire precursor features are extracted from three types of modal data to construct a multimodal fusion feature vector. Based on the fusion feature vector, a comprehensive hazard index R is calculated as an auxiliary criterion for early warning decision-making.

[0011] S3. Based on multimodal fusion feature vectors, at least four levels of collaborative early warning judgment are executed according to the full life cycle development stages of cable tunnel fires. The triggering conditions of each level of early warning are independent of each other and the judgment is executed synchronously. When multiple levels of early warning conditions are met at the same time, the highest level of early warning is reported and all triggering records are retained.

[0012] S4. To address data jitter caused by environmental disturbances and load fluctuations in cable tunnels, an adaptive sliding window mechanism and a graded cooling mechanism are adopted to manage the early warning status, suppress false alarms and duplicate reporting, and automatically downgrade and cancel the early warning status based on the sliding window average.

[0013] S5. The video frame acquisition and push unit and the intelligent detection unit are fully decoupled. The two complete data interaction through a shared frame buffer with a mutex lock, realizing the parallel and interference-free operation of continuous multi-channel video stream push and high-frequency AI detection.

[0014] As a further preferred embodiment of this technical solution: In S1, the specific implementation method of multi-threaded parallel acquisition is as follows:

[0015] The visual modality acquisition thread pulls thermal imaging video streams and visible light video streams from dual-spectrum cameras via the RTSP protocol, and supports concurrent acquisition of up to 4 video streams per node, with an acquisition cycle of 1 second.

[0016] The olfactory modality acquisition thread reads the gas sensor module via RS485 or Modbus RTU protocol to acquire real-time concentration data of CO, O2, TVOC, PM2.5 and PM10, with an acquisition cycle of 1 second.

[0017] The electrical signal mode acquisition thread reads the three-phase ammeter via RS485 or Modbus RTU protocol, and acquires real-time data of the A / B / C three-phase current. The acquisition cycle is 1 second.

[0018] All acquisition threads run independently, and each frame of acquired data is bound to a timestamp of the acquisition time, thus completing the time sequence synchronization and alignment of multi-source data.

[0019] As a further preferred embodiment of this technical solution, in S2, the specific implementation method of feature extraction and fusion is as follows:

[0020] Features extracted from visual modal data include: the highest temperature in the cable area of ​​the thermal imaging image. Heat source area Temperature change rate And the flame presence indicator output by visible light AI detection. With detection confidence ;

[0021] The formula for calculating the rate of temperature change is as follows:

[0022] (1)

[0023] In formula (1), This is the data collection time interval, in seconds.

[0024] Features extracted from olfactory modality data include: CO concentration. TVOC concentration and the change in both per unit time. , ;

[0025] Features extracted from electrical signal modal data include: the three-phase current imbalance U calculated based on IEEE standards; and a multi-mode fusion feature vector constructed based on these features.

[0026] (2)

[0027] The comprehensive risk index R is calculated based on the fused feature vectors, and the calculation formula is as follows:

[0028] (3)

[0029] In formula (3), For the normalized mapping function of the corresponding feature, These are the weighting coefficients for each feature.

[0030] As a further preferred embodiment of this technical solution: In S3, the four-level collaborative early warning mechanism corresponds to the four typical stages of the entire life cycle of a cable tunnel fire, and the specific triggering conditions for each level of early warning are as follows:

[0031] Level 1 warning, corresponding to the early heat accumulation stage of cable joints, can be triggered if any of the following conditions are met:

[0032] Highest temperature in thermal imaging ≥32℃, heat source area A≥800 pixels², CO concentration ≥10ppm;

[0033] Level 2 warning, corresponding to the stage of cable insulation deterioration and thermal runaway propagation, can be triggered if any of the following conditions are met:

[0034] The heat source temperature or heat source area growth rate is ≥1.3 times, and the CO concentration... ≥20ppm, three-phase current imbalance U≥5%;

[0035] A Level 3 warning is triggered when an open flame appears in the cable insulation layer, and can be activated if any of the following conditions are met:

[0036] Visible light AI detected a flame with a confidence level ≥ 0.25 and CO concentration. ≥35ppm, three-phase current imbalance U≥10%;

[0037] Level 4 warning corresponds to the stage where the flame spreads longitudinally along the cable, and the following conditions must be met simultaneously:

[0038] The flame area growth rate is ≥1.3 times, and the temperature continues to rise or the CO concentration... ≥50ppm.

[0039] As a further preferred embodiment of this technical solution: In S4, the specific implementation of the adaptive sliding window mechanism is as follows:

[0040] Maintain a historical queue of length N frames for each feature, with N=30 by default, corresponding to 30 seconds of historical data;

[0041] Each time a new frame of data is collected, it is appended to the end of the queue. If the queue length exceeds N, the oldest data at the head of the queue is removed to keep the queue length constant.

[0042] When calculating the growth rate of feature quantities, the historical average value in the queue excluding the latest frame is used as the calculation benchmark, rather than the previous value of a single frame, to smooth out instantaneous fluctuations in data.

[0043] A warning is confirmed to be triggered only when the average value of the feature quantity in M ​​consecutive frames continuously exceeds the warning threshold of the corresponding level.

[0044] As a further preferred embodiment of this technical solution: In S4, the specific implementation method of the graded cooling and automatic warning cancellation mechanism is as follows:

[0045] Each level of warning is configured with an independent cooldown timer. After a warning of the same level is triggered, it enters a cooldown period. The default cooldown period is 5 seconds. Even if the triggering conditions are met again during the cooldown period, it will not be reported again.

[0046] The cooldown time for high-level alerts is set to half that of low-level alerts to ensure timely response to high-risk situations.

[0047] When the triggering conditions for a certain level of warning continue to disappear and the duration exceeds the cancellation delay time (default cancellation delay time is 10 seconds), the warning status will be automatically updated to "cancelled" after the feature quantity is confirmed to have returned to the normal range based on the sliding window mean.

[0048] As a further preferred embodiment of this technical solution: In S5, the specific operation mode of decoupling video streaming and AI detection is as follows:

[0049] The video frame acquisition and push unit continuously acquires video frames from the camera at a fixed rhythm of 15fps. After encoding them into JPEG format, the frames are directly pushed to the cloud server via TCP protocol without waiting for the processing results from the intelligent detection unit.

[0050] At the same time, a mutex lock is added to the shared frame buffer, and the lock is released immediately after the latest frame is written into the buffer;

[0051] The intelligent detection unit acquires a mutex lock on the shared frame buffer at a fixed frequency of once per second. After reading the latest frame from the buffer, it immediately releases the lock and performs flame / smoke AI detection on the valid frames read. The detection results are not fed back to the video acquisition and push process, so as not to affect the video streaming rhythm.

[0052] When the intelligent detection unit detects a flame, it will report visual warning data, including alarm level, flame area, and detection confidence level, to the cloud server through an independent interface.

[0053] As a further preferred embodiment of this technical solution: the three-phase current imbalance U is calculated using the IEEE standard, and the calculation formula is as follows:

[0054] U = (maximum deviation / average three-phase current) × 100% (4)

[0055] Among them, the average value of the three-phase current is:

[0056] (5)

[0057] In formula (5), These are the real-time values ​​of the three-phase currents A, B, and C, with a maximum deviation of [value missing]. .

[0058] Compared with the prior art, the beneficial effects of the present invention are:

[0059] 1. This invention achieves early warning throughout the entire lifecycle of cable fires, significantly advancing the warning and response window. Addressing the shortcomings of traditional smoke / heat detection solutions, which only respond after open flames or visible smoke appear, failing to identify early heat accumulation at cable joints and missing the optimal response window, this invention precisely matches the entire lifecycle of a cable fire—from early heat accumulation, insulation deterioration and thermal runaway, to the appearance of open flames and the spread of fire along the cable—and constructs a four-level collaborative early warning mechanism. This mechanism integrates multi-dimensional early anomaly detection, including thermal imaging temperature rise detection, identification of characteristic gases from cable insulation pyrolysis, and monitoring of electrical fault precursors. It can detect potential cable thermal runaway hazards before open flames appear, thus securing a golden window for maintenance and response, thereby reducing the risk of uncontrolled spread of cable tunnel fires.

[0060] 2. This invention employs multimodal fusion and cross-validation, significantly reducing false alarm rates and improving the reliability of early warnings in complex environments. Addressing the pain points of false alarms and blind spots in single-modal detection caused by ventilation disturbances, load fluctuations, and environmental background interference within tunnels, this invention constructs a parallel acquisition architecture for three independent modalities: visual, olfactory, and electrical signals. A unified timestamp is used to synchronize and align the temporal data of multi-source heterogeneous data. Precursor features strongly correlated with the cable fire occurrence mechanism are extracted to construct a multimodal fusion feature vector. After normalization and weighted fusion, a comprehensive hazard index is calculated, achieving multi-dimensional feature complementarity and cross-validation. Simultaneously, an adaptive sliding window mechanism smooths out instantaneous data jitter, using the average of multiple consecutive frames to confirm early warning triggering, suppressing false alarms at their technical root. This significantly improves the anti-interference capability and reliability of early warnings in complex, enclosed cable tunnel environments.

[0061] 3. This invention fills the gap in early warning technology for electrical fires, enabling proactive prevention of the root causes of fires. Addressing the shortcomings of existing fire warning systems that do not incorporate electrical characteristics into their warning models and lack early detection capabilities for core fire causes such as cable insulation degradation and phase-to-phase faults, this invention incorporates three-phase current imbalance calculated based on IEEE standards into a multi-modal warning system. This imbalance is used as a core electrical precursor to cable insulation degradation, and differentiated warning thresholds are set to match the fire development stage. This allows for early identification and warning when insulation breakdown and phase-to-phase short circuit risks occur in cables. It overcomes the technical limitations of traditional fire protection systems that prioritize fire response over prevention of underlying causes, and improves the early warning and prevention capabilities of the entire chain of "electrical anomaly - heat accumulation - thermal runaway - combustion" in cable tunnels.

[0062] 4. This invention features edge-side localized computing processing, which greatly compresses the early warning response time and significantly avoids network dependency risks. Addressing the pain points of traditional solutions that rely on centralized cloud processing, network failures and insufficient bandwidth can easily lead to early warning delays and cannot meet the second-level response requirements for cable fires, this invention adopts an edge-side sinking computing architecture. All data collection, feature fusion, and early warning judgment processes are completed locally on edge industrial control computers distributed along the tunnel, without waiting for the original data to be uploaded to the cloud for processing.

[0063] 5. The fully decoupled architecture of this invention solves the conflict of edge computing resources and ensures the long-term stable parallel operation of the system. It addresses the pain points of limited computing power of edge devices, multiple video streaming and AI intelligent detection competing for computing resources, resulting in video stuttering, detection delay and system instability. The video frame acquisition and push unit and the intelligent detection unit are fully decoupled. The two complete data interaction through a shared frame buffer with a mutex lock. The video streaming process does not need to wait for the AI ​​detection results and runs continuously and stably at a fixed 15fps. The AI ​​detection unit executes detection tasks independently at a fixed frequency. The two processes run in parallel without any interference.

[0064] 6. This invention's fully closed-loop hierarchical early warning management mechanism effectively adapts to the unmanned operation and maintenance needs of cable tunnels. Addressing the pain points of traditional solutions—such as ambiguous early warning levels and inability to match the requirements of unmanned and remote operation and maintenance management in cable tunnels—it establishes a four-level independent synchronous judgment collaborative early warning mechanism. The triggering conditions for each level of early warning are independent; when multiple levels of conditions are triggered simultaneously, the highest level is reported and all trigger records are retained. Simultaneously, each level of early warning is independently configured with a hierarchical cooling timer to suppress duplicate reporting. Automatic downgrading and cancellation of the early warning status are achieved based on the sliding window average, forming a closed-loop management process encompassing early warning triggering, status management, hierarchical response, and automatic cancellation. Through these settings, it can greatly adapt to hierarchical response strategies for different hazard levels. Level 1 and 2 early warnings can trigger automatic recording and operation and maintenance notifications, while level 3 and 4 early warnings can link with emergency devices such as fire extinguishing and ventilation systems within the tunnel, fully meeting the core requirements of unmanned operation and maintenance management of cable tunnels. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the operation of a multimodal fusion acquisition and collaborative early warning method for the edge side of a cable tunnel according to the present invention. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0067] Example

[0068] Please see Figure 1 This invention provides a technical solution: a method for multimodal fusion acquisition and collaborative early warning at the edge of a cable tunnel, comprising the following steps:

[0069] S1. Construct a multi-threaded parallel acquisition architecture, synchronously acquire visual modal data, olfactory modal data and electrical signal modal data through mutually independent acquisition threads, and complete the time sequence alignment of all acquired data through a unified timestamp;

[0070] Among them, the visual modal data includes thermal imaging video streams and visible light video streams aligned with the cable tray; the olfactory modal data includes characteristic gas concentration data corresponding to the pyrolysis of cable insulation; and the electrical signal modal data includes real-time data of the A / B / C three-phase currents of the cable power supply circuit.

[0071] S2. To address the mechanism of cable tunnel fires, fire precursor features are extracted from three types of modal data to construct a multimodal fusion feature vector. Based on the fusion feature vector, a comprehensive hazard index R is calculated as an auxiliary criterion for early warning decision-making.

[0072] S3. Based on multimodal fusion feature vectors, at least four levels of collaborative early warning judgment are executed according to the full life cycle development stages of cable tunnel fires. The triggering conditions of each level of early warning are independent of each other and the judgment is executed synchronously. When multiple levels of early warning conditions are met at the same time, the highest level of early warning is reported and all triggering records are retained.

[0073] S4. To address data jitter caused by environmental disturbances and load fluctuations in cable tunnels, an adaptive sliding window mechanism and a graded cooling mechanism are adopted to manage the early warning status, suppress false alarms and duplicate reporting, and automatically downgrade and cancel the early warning status based on the sliding window average.

[0074] S5. The video frame acquisition and push unit and the intelligent detection unit are fully decoupled. The two complete data interaction through a shared frame buffer with a mutex lock, realizing the parallel and interference-free operation of continuous multi-channel video stream push and high-frequency AI detection.

[0075] In this embodiment, specifically: in S1, the specific implementation method of multi-threaded parallel acquisition is as follows:

[0076] The visual modal acquisition thread pulls thermal imaging video streams from dual-spectrum cameras (for detecting temperature rise of cable joints and wire harnesses) and visible light video streams (for AI flame detection or smoke detection) via the RTSP protocol, and supports concurrent acquisition of up to 4 video streams per node with an acquisition cycle of 1 second.

[0077] The olfactory modality acquisition thread reads the gas sensor module via RS485 or Modbus RTU protocol to acquire real-time concentration data of CO, O2, TVOC, PM2.5 and PM10, with an acquisition cycle of 1 second.

[0078] Among them, CO and TVOC are characteristic gases of cable insulation material pyrolysis, and the decrease in O2 reflects oxygen-consuming combustion in the tunnel;

[0079] The electrical signal mode acquisition thread reads the three-phase ammeter via RS485 or Modbus RTU protocol, and acquires real-time data of the three-phase currents A / B / C. The acquisition cycle is 1 second. The change in the three-phase current imbalance reflects cable insulation deterioration or phase-to-phase fault.

[0080] All acquisition threads run independently, and each frame of acquired data is bound to a timestamp of the acquisition time, thus completing the time sequence synchronization and alignment of multi-source data.

[0081] In this embodiment, specifically: in S2, the feature extraction and fusion are implemented as follows:

[0082] Features extracted from visual modal data include: the highest temperature in the cable area of ​​the thermal imaging image. Heat source area Temperature change rate (Reaction cable joint thermal runaway velocity), and the flame presence indicator output by visible light AI detection. With detection confidence ;

[0083] The formula for calculating the rate of temperature change is as follows:

[0084] (1)

[0085] In formula (1), This is the data collection time interval, in seconds.

[0086] Features extracted from olfactory modality data include: CO concentration. (Early signs of cable insulation pyrolysis), TVOC concentration (Volatilization characteristics of organic insulating materials), and the change in both per unit time. , ;

[0087] Features extracted from electrical signal modal data include: three-phase current imbalance U (reflecting the degree of electrical insulation degradation) calculated based on IEEE standards. A multi-modal fusion feature vector is constructed based on these features.

[0088] (2)

[0089] The comprehensive risk index R is calculated based on the fused feature vectors, and the calculation formula is as follows:

[0090] (3)

[0091] In formula (3), This is the normalization mapping function for the corresponding feature, used to linearly map the original dimensional values ​​to the interval [0,1]. The weighting coefficients for each feature are pre-calibrated according to the importance of early warning signs of cable tunnel fires, with the weights ordered as follows: visual heat source features > electrical signal features > gas features.

[0092] In this embodiment, specifically: In S3, the four-level collaborative early warning mechanism corresponds to the four typical stages of the entire life cycle of a cable tunnel fire, and the specific triggering conditions for each level of early warning are as follows:

[0093] Level 1 warning (electrical / thermal source abnormality) corresponds to the early heat accumulation stage of the cable joint and can be triggered if any of the following conditions are met:

[0094] Highest temperature in thermal imaging ≥32℃ (exceeding the normal temperature rise range of the tunnel environment), heat source area A ≥800 pixels² (local hot spot formation), CO concentration ≥10ppm (early characteristics of the start of pyrolysis in insulating materials);

[0095] Level 2 warning (thermal runaway propagation) corresponds to the stage of cable insulation degradation and thermal runaway propagation, and can be triggered if any of the following conditions are met:

[0096] Heat source temperature or heat source area growth rate ≥ 1.3 times (accelerated thermal runaway), CO concentration ≥20ppm, three-phase current imbalance U≥5% (electrical characteristics caused by power supply insulation degradation);

[0097] Level 3 warning (flame appearance) corresponds to the stage where open flame appears on the cable insulation layer. It can be triggered if any of the following conditions are met:

[0098] Visible light AI detected a flame with a confidence level ≥ 0.25 and CO concentration. ≥35ppm (significant insulation burning), three-phase current imbalance U≥10% (risk of insulation breakdown or phase-to-phase short circuit).

[0099] Level 4 warning (fire spreading along the cable), corresponding to the stage of flames spreading longitudinally along the cable, requires the following conditions to be met simultaneously:

[0100] The flame area growth rate is ≥1.3 times (expanding along the tunnel direction), and the temperature continues to rise or the CO concentration... ≥50ppm.

[0101] In this embodiment, specifically: in S4, the adaptive sliding window mechanism is implemented as follows:

[0102] Maintain a historical queue of length N frames for each feature, with N=30 by default, corresponding to 30 seconds of historical data;

[0103] Each time a new frame of data is collected, it is appended to the end of the queue. If the queue length exceeds N, the oldest data at the head of the queue is removed to keep the queue length constant.

[0104] When calculating the growth rate of feature quantities, the historical average value in the queue excluding the latest frame is used as the calculation benchmark, rather than the previous value of a single frame, to smooth out instantaneous fluctuations in data.

[0105] Furthermore, the warning level is confirmed to be triggered only when the average value of the feature quantity in M ​​consecutive frames continuously exceeds the warning threshold of the corresponding level. The default value of M is 5, and the value of M can be adaptively adjusted within the range of 3 to 10 frames according to the intensity of tunnel ventilation disturbance.

[0106] In this embodiment, specifically: in S4, the implementation method of the graded cooling and automatic warning cancellation mechanism is as follows:

[0107] Each level of warning is configured with an independent cooldown timer. After a warning of the same level is triggered, it enters a cooldown period. The default cooldown period is 5 seconds. Even if the triggering conditions are met again during the cooldown period, it will not be reported again.

[0108] The cooldown time for high-level alerts (Level 3 and Level 4) is set to half that of low-level alerts (Level 1 and Level 2) to ensure timely response to high-risk situations.

[0109] When the triggering conditions for a certain level of warning continue to disappear and the duration exceeds the cancellation delay time (default cancellation delay time is 10 seconds), the warning status will be automatically updated to "cancelled" after the feature quantity is confirmed to have returned to the normal range based on the sliding window mean.

[0110] In this embodiment, specifically: in S5, the specific operation mode of decoupling video streaming and AI detection is as follows:

[0111] The video frame acquisition and push unit continuously acquires video frames from the camera at a fixed rhythm of 15fps. After encoding them into JPEG format, the frames are directly pushed to the cloud server via TCP protocol without waiting for the processing results from the intelligent detection unit.

[0112] At the same time, a mutex lock is added to the shared frame buffer, and the lock is released immediately after the latest frame is written into the buffer;

[0113] The intelligent detection unit acquires a mutex lock on the shared frame buffer at a fixed frequency of once per second. After reading the latest frame from the buffer, it immediately releases the lock and performs AI detection of flames or smoke on the valid frames read. The detection results are not fed back to the video acquisition and push process, so as not to affect the video streaming rhythm.

[0114] When the intelligent detection unit detects a flame, it will report visual warning data, including alarm level, flame area, and detection confidence level, to the cloud server through an independent interface.

[0115] In this embodiment, specifically: the three-phase current imbalance U is calculated using the IEEE standard, and the calculation formula is as follows:

[0116] U = (maximum deviation / average three-phase current) × 100% (4)

[0117] Among them, the average value of the three-phase current is:

[0118] (5)

[0119] In formula (5), These are the real-time values ​​of the three-phase currents A, B, and C, with a maximum deviation of [value missing]. ;

[0120] Among them, attention is required when U ≥ 2%, a level 2 warning is triggered when U ≥ 5%, and a level 3 warning is triggered when U ≥ 10%.

[0121] In this embodiment, specifically: an edge computing device (industrial control computer) is deployed with a monitoring node every 100-200 meters along the cable tunnel. The hardware configuration of a single node includes:

[0122] x86 or ARM architecture industrial control computer with ≥4GB memory, equipped with RS485 interface and Ethernet interface;

[0123] One to two dual-spectrum cameras aligned with the cable tray, supporting visible light 704×576@15fps and thermal imaging 320×240@15fps acquisition, and supporting Hikvision ISAPI interface to obtain temperature matrix;

[0124] ModbusRTU protocol gas sensor module, collects CO, O2, TVOC, PM2.5, PM10, and is installed in the middle of tunnels or areas with dense cable joints;

[0125] Modbus RTU protocol three-phase ammeter, collects A / B / C three-phase current of power distribution circuit in the monitored section;

[0126] All nodes are connected to the cloud server of the tunnel operation and maintenance management center via industrial Ethernet;

[0127] One point that needs further explanation is that the IO relay module (optional) is used to link the fire extinguishing device or ventilation equipment in the tunnel.

[0128] Actual performance verification

[0129] In the actual deployment of a 500-meter cable tunnel, the key performance indicators were measured and verified, and the results are as follows:

[0130] Video streaming performance: A single node can stably stream 4 video streams (2 visible light streams + 2 thermal imaging streams) at 15fps, running continuously for 72 hours without frame rate loss or stuttering.

[0131] Intelligent detection performance: The intelligent detection unit completes flame / smoke detection at a fixed frequency of once per second, with a detection accuracy of ≥99%, and is completely uninterrupted to video streaming;

[0132] Early warning response performance: From the detection of abnormal features at the edge to the triggering of early warning reporting, the end-to-end response time is ≤80ms, which is far lower than the industry's conventional cloud processing solutions;

[0133] Equipment resource usage: When all functions are running in parallel, the CPU utilization rate of the industrial control computer is stable at less than 55%, the memory usage is ≤2GB, and sufficient computing power redundancy is reserved.

[0134] Early warning reliability: Compared with the traditional single-mode smoke detector solution, the multi-mode fusion early warning of this invention reduces the false alarm rate by 72%, and the thermal imaging temperature rise level warning is 4 to 7 minutes earlier than the smoke detector alarm, perfectly realizing the early warning of cable fires.

[0135] Working principle: This invention addresses the industry pain points of cable tunnels being narrow and enclosed, with strong environmental interference, and traditional single-mode fire early warning schemes suffering from lagging identification, high false alarm rates, and large response delays. It uses edge computing as the core carrier to construct a complete technical system that integrates multi-source synchronous acquisition, multi-modal feature fusion, full-cycle hierarchical early warning, adaptive anti-interference management, and decoupled parallel operation.

[0136] This invention establishes independent 1-second cycle acquisition threads for three modalities: visual, olfactory, and electrical signals. It simultaneously acquires bispectral video, characteristic gas concentrations, and three-phase current data, achieving time-series alignment of multi-source data through a unified timestamp, thus resolving the problem of heterogeneous data time-series misalignment at its source. Subsequently, targeting the entire lifecycle mechanism of cable fires, it extracts precursor features strongly correlated with fire development from the three modalities, constructs a multimodal fusion feature vector, and calculates a comprehensive hazard index through normalization and weighted fusion. This achieves multi-dimensional feature complementarity, eliminating blind spots in single-modal recognition.

[0137] This invention matches four typical stages of a fire, from early heat accumulation to fire spread, and sets up a collaborative early warning mechanism with four levels of independent synchronous judgment. When multiple conditions are triggered simultaneously, the highest level is reported, achieving a tiered early warning system throughout the entire fire cycle. Simultaneously, an adaptive sliding window smooths out instantaneous data jitter, confirming early warning triggers with the average value of consecutive frames. A graded cooling mechanism suppresses duplicate reporting, and the early warning status is automatically downgraded and deactivated based on the sliding window average, reducing the risk of false alarms at the source. Furthermore, this invention fully decouples video streaming and AI detection, completing data interaction through a shared frame buffer with a mutex lock, enabling two processes to run in parallel without interference and ensuring long-term system stability.

[0138] The overall solution adopts an edge-side sinking computing architecture, which significantly shortens the early warning response time. Compared with the traditional single-mode smoke detection solution, the early warning time is 4 to 7 minutes earlier and the false alarm rate is reduced by 72%, which is suitable for the core needs of fire prevention and control and operation and maintenance management in cable tunnels.

[0139] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A cable tunnel edge side multi-modal fusion acquisition and collaborative early warning method, characterized in that, Includes the following steps: S1. Construct a multi-threaded parallel acquisition architecture, synchronously acquire visual modal data, olfactory modal data and electrical signal modal data through mutually independent acquisition threads, and complete the time sequence alignment of all acquired data through a unified timestamp; S2. To address the mechanism of cable tunnel fires, fire precursor features are extracted from three types of modal data to construct a multimodal fusion feature vector. Based on the fusion feature vector, a comprehensive hazard index R is calculated as an auxiliary criterion for early warning decision-making. S3. Based on multimodal fusion feature vectors, at least four levels of collaborative early warning judgment are executed according to the full life cycle development stages of cable tunnel fires. The triggering conditions of each level of early warning are independent of each other and the judgment is executed synchronously. When multiple levels of early warning conditions are met at the same time, the highest level of early warning is reported and all triggering records are retained. S4. To address data jitter caused by environmental disturbances and load fluctuations in cable tunnels, an adaptive sliding window mechanism and a graded cooling mechanism are adopted to manage the early warning status, suppress false alarms and duplicate reporting, and automatically downgrade and cancel the early warning status based on the sliding window average. S5. The video frame acquisition and push unit and the intelligent detection unit are fully decoupled. The two complete data interaction through a shared frame buffer with a mutex lock, realizing the parallel and interference-free operation of continuous multi-channel video stream push and high-frequency AI detection.

2. The cable tunnel edge side multi-modal fusion acquisition and collaborative early warning method according to claim 1, characterized in that: In S1, the specific implementation method of multi-threaded parallel acquisition is as follows: The visual modality acquisition thread pulls thermal imaging video streams and visible light video streams from dual-spectrum cameras via the RTSP protocol, and supports concurrent acquisition of up to 4 video streams per node, with an acquisition cycle of 1 second. The olfactory modality acquisition thread reads the gas sensor module via RS485 or Modbus RTU protocol to acquire real-time concentration data of CO, O2, TVOC, PM2.5 and PM10, with an acquisition cycle of 1 second. The electrical signal mode acquisition thread reads the three-phase ammeter via RS485 or Modbus RTU protocol, and acquires real-time data of the A / B / C three-phase current. The acquisition cycle is 1 second. All acquisition threads run independently, and each frame of acquired data is bound to a timestamp of the acquisition time, thus completing the time sequence synchronization and alignment of multi-source data.

3. The cable tunnel edge side multi-modal fusion acquisition and collaborative early warning method according to claim 1, characterized in that: In S2, the specific implementation of feature extraction and fusion is as follows: Features extracted from the visual modality data include: the highest temperature of the cable area in the thermal image frame , the heat source area , the temperature change rate , and the flame existence flag and detection confidence of the visible light AI detection output The formula for calculating the rate of temperature change is as follows: （1） In equation (1), is the acquisition time interval in seconds; Features extracted from the olfactory modality data include: CO concentration , TVOC concentration , and the amount of change per unit time of both , ; Features extracted from electrical signal modal data include: the three-phase current imbalance U calculated based on IEEE standards; and a multi-mode fusion feature vector constructed based on these features. （2） The comprehensive risk index R is calculated based on the fused feature vectors, and the calculation formula is as follows: （3） In formula (3), is a normalized mapping function of the corresponding feature, is a weight coefficient of each feature.

4. The cable tunnel edge side multi-modal fusion acquisition and collaborative early warning method according to claim 1, characterized in that: In S3, the four-level collaborative early warning mechanism corresponds to the four typical stages of the entire life cycle of a cable tunnel fire. The specific triggering conditions for each level of early warning are as follows: Level 1 warning, corresponding to the early heat accumulation stage of cable joints, can be triggered if any of the following conditions are met: Highest temperature in thermal imaging ≥32℃, heat source area A≥800 pixels², CO concentration ≥10ppm; Level 2 warning, corresponding to the stage of cable insulation deterioration and thermal runaway propagation, can be triggered if any of the following conditions are met: The heat source temperature or heat source area growth rate is ≥1.3 times, and the CO concentration... ≥20ppm, three-phase current imbalance U≥5%; A Level 3 warning is triggered when an open flame appears in the cable insulation layer, and can be activated if any of the following conditions are met: Visible light AI detected a flame with a confidence level ≥ 0.25 and CO concentration. ≥35ppm, three-phase current imbalance U≥10%; Level 4 warning corresponds to the stage where the flame spreads longitudinally along the cable, and the following conditions must be met simultaneously: The flame area growth rate is ≥1.3 times, and the temperature continues to rise or the CO concentration... ≥50ppm.

5. The method for multi-modal fusion acquisition and collaborative early warning at the edge of a cable tunnel according to claim 1, characterized in that: In S4, the adaptive sliding window mechanism is implemented as follows: Maintain a historical queue of length N frames for each feature, with N=30 by default, corresponding to 30 seconds of historical data; Each time a new frame of data is collected, it is appended to the end of the queue. If the queue length exceeds N, the oldest data at the head of the queue is removed to keep the queue length constant. When calculating the growth rate of feature quantities, the historical average value in the queue excluding the latest frame is used as the calculation benchmark, rather than the previous value of a single frame, to smooth out instantaneous fluctuations in data. A warning is confirmed to be triggered only when the average value of the feature quantity in M ​​consecutive frames continuously exceeds the warning threshold of the corresponding level.

6. The method for multi-modal fusion acquisition and collaborative early warning at the edge of a cable tunnel according to claim 1, characterized in that: In S4, the specific implementation method of the graded cooling and automatic warning cancellation mechanism is as follows: Each level of warning is configured with an independent cooldown timer. After a warning of the same level is triggered, it enters a cooldown period. The default cooldown period is 5 seconds. Even if the triggering conditions are met again during the cooldown period, it will not be reported again. The cooldown time for high-level alerts is set to half that of low-level alerts to ensure timely response to high-risk situations. When the triggering conditions for a certain level of warning continue to disappear and the duration exceeds the cancellation delay time (default cancellation delay time is 10 seconds), the warning status will be automatically updated to "cancelled" after the feature quantity is confirmed to have returned to the normal range based on the sliding window mean.

7. The method for multi-modal fusion acquisition and collaborative early warning at the edge of a cable tunnel according to claim 1, characterized in that: In S5, the specific operation method of decoupling video streaming and AI detection is as follows: The video frame acquisition and push unit continuously acquires video frames from the camera at a fixed rhythm of 15fps. After encoding them into JPEG format, the frames are directly pushed to the cloud server via TCP protocol without waiting for the processing results from the intelligent detection unit. At the same time, a mutex lock is added to the shared frame buffer, and the lock is released immediately after the latest frame is written into the buffer; The intelligent detection unit acquires a mutex lock on the shared frame buffer at a fixed frequency of once per second. After reading the latest frame from the buffer, it immediately releases the lock and performs flame / smoke AI detection on the valid frames read. The detection results are not fed back to the video acquisition and push process, so as not to affect the video streaming rhythm. When the intelligent detection unit detects a flame, it will report visual warning data, including alarm level, flame area, and detection confidence level, to the cloud server through an independent interface.

8. The method for multi-modal fusion acquisition and collaborative early warning at the edge of a cable tunnel according to claim 1, characterized in that: The three-phase current imbalance U is calculated using the IEEE standard, and the calculation formula is as follows: U = (maximum deviation / average three-phase current) × 100% (4) Among them, the average value of the three-phase current is: （5） In formula (5), These are the real-time values ​​of the three-phase currents A, B, and C, with a maximum deviation of [value missing]. .

Images