Highway tunnel monitoring method, system, equipment and medium

Abstract

The invention discloses an expressway tunnel monitoring method, system and device and a medium, and relates to the technical field of tunnel monitoring. Tunnel environment data are collected in real time through a multi-source heterogeneous sensor array, and the data at least comprise video images, millimeter wave radar point cloud, laser radar three-dimensional coordinates, temperature and humidity and CO concentration parameters. According to the system, an improved YOLOv7 algorithm is used for carrying out target real-time detection, an optical flow method is combined to predict the motion trail of a target object, a tunnel dynamic characteristic spectrum is constructed, real-time monitoring of the tunnel environment is achieved, by deploying a risk level assessment engine, the system can extract space-time correlation characteristics, and the risk level assessment efficiency is improved. A comprehensive risk index is generated in combination with a fuzzy logic decision tree, and when the risk index exceeds a dynamic threshold value, the system triggers a grading early warning mechanism, links tunnel emergency equipment, synchronously generates an emergency plan and pushes the emergency plan to an operation and maintenance terminal to guarantee tunnel operation safety.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel monitoring, and specifically to a highway tunnel monitoring method, system, device and medium. Background Technique

[0002] With the rapid development of safety perception and early warning technologies for highway tunnels, traditional monitoring systems are gradually becoming difficult to meet the increasingly complex and changeable traffic conditions. Traditional methods mainly rely on a single type of sensor, such as fixed cameras or a single type of sensor array. These systems have certain limitations and lags when facing complex environmental changes.

[0003] To solve the above problems, modern technology has proposed the integrated application of multi-source heterogeneous sensors. Through the collaborative work of various devices such as video images, millimeter-wave radars, lidars, temperature and humidity sensors, and gas concentration sensors, more comprehensive environmental data collection has been achieved, thereby improving safety. However, although the collection of these data is detailed, the complexity of data processing and fusion among various sensors is high, resulting in problems with data processing efficiency and accuracy. In addition, existing early warning systems often issue alarms based on fixed thresholds, lacking dynamics and being unable to effectively respond to emergencies in real time. Therefore, how to efficiently extract risk information from multi-source heterogeneous data, construct a dynamic risk assessment system, and link emergency devices has become the focus of current research.

[0004] In summary, the existing technology still needs to be further improved and perfected in multi-modal data fusion, real-time risk assessment, and emergency response mechanisms to meet the safety monitoring requirements of high-density traffic and complex environments in highway tunnels. Summary of the Invention

[0005] The purpose of the present invention is to provide a highway tunnel monitoring method, system, device and medium to solve the problems raised in the above background technique.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A highway tunnel monitoring method, including the following steps:

[0007] Real-time collection of tunnel environmental data through a multi-source heterogeneous sensor array, and the data at least includes video images, millimeter-wave radar point clouds, lidar three-dimensional coordinates, temperature and humidity, and CO concentration parameters;

[0008] Use an adaptive noise suppression algorithm to preprocess the original data and establish a data confidence weight allocation model;

[0009] Construct a tunnel dynamic feature map based on a multi-modal data fusion strategy, in which an improved YOLOv7 algorithm is used for real-time detection of target objects, combined with the optical flow method to calculate the prediction of the movement trajectory;

[0010] Deploy a risk level assessment engine, extract spatio-temporal correlation features through a convolutional neural network, and input them into a preset fuzzy logic decision tree to generate a comprehensive risk index;

[0011] When the risk index exceeds the dynamic threshold, trigger a hierarchical early warning mechanism and link tunnel emergency equipment, and synchronously generate an event handling plan and push it to the operation and maintenance terminal.

[0012] As a specific solution of the technical solution of this application, the multi-modal data fusion strategy includes:

[0013] Establish a feature alignment module based on the attention mechanism to achieve spatio-temporal alignment of sensor data through cross-modal feature interaction;

[0014] Adopt a dual-channel residual network to process structured data and unstructured data respectively;

[0015] Design a time series sliding window mechanism to dynamically adjust the data sampling frequency and fusion period.

[0016] As a specific solution of the technical solution of this application, the risk level assessment engine includes:

[0017] A three-dimensional point cloud density anomaly monitoring sub-module, which uses an improved DBSCAN algorithm to identify abnormal vehicle aggregation states;

[0018] A visibility attenuation model, which combines Mie scattering theory to calculate the smoke diffusion trend;

[0019] A multi-target tracking trajectory prediction unit, which constructs a vehicle kinematic model based on the LSTM network.

[0020] A highway tunnel monitoring system, the system includes:

[0021] An edge computing node cluster, deployed in key tunnel monitoring areas, integrating FPGA acceleration chips to achieve data preprocessing;

[0022] A central analysis server, equipped with a GPU array to run a deep learning inference engine, including:

[0023] A data quality verification module, which dynamically eliminates abnormal sensor data;

[0024] A virtual-real fusion scene reconstruction module, which constructs a tunnel digital twin model;

[0025] An emergency linkage control unit, which connects ventilation, lighting, and escape indication devices through an industrial bus protocol;

[0026] A visualization interaction terminal, which provides a three-dimensional situation awareness interface and historical data traceability function.

[0027] As a specific solution of the technical solution of this application, the edge computing base point configuration includes:

[0028] Multi-protocol adaptation interfaces, compatible with RS485, CAN, and Zigbee communication standards;

[0029] Dynamic power consumption correlation module, adjusting the power supply voltage according to data traffic;

[0030] Hardware-level encryption unit, collecting SM4 national cryptographic algorithm to ensure transmission security.

[0031] A highway tunnel monitoring device, which includes at least one heterogeneous processor, including a CPU, an NPU, and an IP core dedicated to image processing;

[0032] A memory, used to store an executable program according to any of the methods described in claims 1-3;

[0033] Industrial communication module, supporting 5G slice network and LoRa wide-area networking;

[0034] Electromagnetic interference-resistant chassis, compliant with the GB / T17626 electromagnetic compatibility standard.

[0035] A highway tunnel monitoring medium, storing a computer instruction set;

[0036] Including a tunnel scenario knowledge graph database, storing characteristic vectors of typical accident cases;

[0037] Integrated online learning module, dynamically updating the parameters of the risk assessment model according to newly collected data;

[0038] Encapsulated device driver library, supporting the plug-and-play configuration of mainstream industrial sensors.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] This highway tunnel monitoring method, system, device, and medium collect real-time video images, millimeter-wave radar point clouds, lidar three-dimensional coordinates, temperature and humidity, and CO concentration parameters inside and outside the tunnel through an integrated multi-source heterogeneous sensor array, and preprocess the original data through an adaptive noise suppression algorithm. Combining with the data confidence weight assignment model improves the reliability and accuracy of the data.

[0041] Meanwhile, the system uses the improved YOLOv7 algorithm for real-time target detection, combines the optical flow method to predict the movement trajectory of the target object, constructs a tunnel dynamic feature map, realizes real-time monitoring of the tunnel environment. By deploying a risk level assessment engine, the system can extract spatio-temporal correlation features, generate a comprehensive risk index in combination with a fuzzy logic decision tree. When the risk index exceeds the dynamic threshold, the system will trigger a hierarchical early warning mechanism, link tunnel emergency equipment, and synchronously generate an emergency plan, which is pushed to the operation and maintenance terminal to ensure the safety of tunnel operation.

[0042] The system of the present invention preprocesses data through a cluster in the edge computing stage, has multi-protocol adaptation interfaces and a dynamic power consumption correlation module to improve data processing efficiency and energy utilization efficiency. At the same time, the central analysis server is equipped with a GPU array and runs a deep learning inference engine to ensure efficient decision support. The visualization interaction terminal provides a three-dimensional situation awareness interface and a historical data traceability function, enabling operation and maintenance personnel to intuitively understand the real-time status and historical situation of the tunnel. Brief Description of the Drawings

[0043] Figure 1 It is a schematic flow chart of the tunnel monitoring method of the present invention;

[0044] Figure 2 It is a schematic flow chart of the multi-modal data fusion strategy of the present invention;

[0045] Figure 3 It is a schematic flow chart of the risk level assessment engine of the present invention. Detailed Embodiments

[0046] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present invention.

[0048] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0049] In the present invention, unless otherwise clearly specified and defined, terms such as "installed", "connected", "connected to", "fixed", etc. shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components or the interaction relationship between two components, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] In the present invention, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "below" and "beneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0051] In the description of this specification, the description with reference to terms such as "an embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0052] As Figures 1 - 3 shown, the present invention provides a technical solution: a method for monitoring a highway tunnel, including the following steps:

[0053] The tunnel environment data is collected in real time through a multi-source heterogeneous sensor array, and the data includes at least video images, millimeter wave radar point cloud, laser radar three-dimensional coordinates, temperature and humidity, and CO concentration parameters; it should be clear that in the embodiment of the present application, the video camera is installed at a certain interval on the top or side wall of the tunnel, covering the entire tunnel area, collecting video image data for intuitive monitoring of traffic conditions and personnel activities in the tunnel. The millimeter wave radar and the video camera work together to obtain information such as the distance, speed and angle of the target object by transmitting and receiving millimeter wave signals, and form millimeter wave radar point cloud data, while the laser radar obtains the three-dimensional coordinate information of the object in the tunnel by laser scanning, and constructs a high-precision three-dimensional environmental model. The temperature and humidity sensors are distributed throughout the tunnel to monitor the temperature and humidity changes in the tunnel in real time, and provide data support for the early warning of abnormal conditions such as fire, and the CO concentration sensor is also installed in the tunnel to monitor the carbon monoxide concentration. When the concentration is too high, it can timely discover potential fires or vehicle exhaust accumulation. In this application, sensors are installed according to a certain layout and density to ensure the comprehensiveness and accuracy of data collection. For example, the interval between video cameras can be determined according to the length of the tunnel and monitoring needs, generally around 50-100 meters. The layout of millimeter-wave radar and lidar needs to consider their detection range and accuracy requirements to achieve effective monitoring of vehicles and physical objects in the tunnel.

[0054] Adopt adaptive noise suppression algorithm to preprocess the raw data, and establish data confidence weight distribution model; It should be clear that the adaptive noise suppression algorithm preprocesses the collected raw data to improve data quality and reliability, such as the least mean square algorithm and its improved version, to evaluate and offset the noise in the data. In this application, taking LMS algorithm as an example, its basic principle is to adjust the coefficient of the filter by minimizing the mean square value of the error signal, so as to achieve effective suppression of noise. The specific steps are as follows:

[0055] Initialize the filter coefficient vector w(0), usually set to the zero vector;

[0056] For each input signal sample x(n), calculate the filter output y(n)=wH(n)x(n), where wH(n) represents the conjugate transpose of w(n);

[0057] Calculate the error signal e(n)=d(n)-y(n), where d(n) is the desired signal, that is, the original signal not contaminated by noise;

[0058] According to the error signal e(n) and the input signal x(n), the filter coefficient vector w(n+1)=w(n)+μx*(n)e(n) is updated, where μ is the step size factor and x*(n) is the conjugate of x(n);

[0059] Repeat the above steps until the filter converges, i.e., the error signal e(n) reaches the minimum value.

[0060] Meanwhile, establish a data confidence weight assignment model to evaluate and quantify the reliability of different sensor data. According to factors such as the noise level of the data, the accuracy of the sensor, and environmental interference, assign a confidence weight to each piece of data. For example, for the video image data after noise suppression, if its clarity is high and the target recognition is accurate, a higher confidence weight is given. For the millimeter-wave radar point cloud data that is greatly affected by environmental interference, further processing and verification are required, and its confidence weight is relatively low. In this way, comprehensively consider the reliability of multiple data sources to provide a more accurate basis for subsequent data fusion and analysis.

[0061] Construct a tunnel dynamic feature map based on the multi-modal data fusion strategy, in which the improved YOLOv7 algorithm is used for real-time detection of target objects, combined with the optical flow method to calculate the motion trajectory prediction; it should be clear that in the embodiments of the present application, based on the multi-modal data fusion strategy, different types of pre-processed data are fused to construct a tunnel dynamic feature map. The specific steps are as follows:

[0062] Ensure the consistency of different sensor data in time and space. For example, synchronize the video image, millimeter-wave radar point cloud, and lidar three-dimensional coordinate data in time so that they correspond to the tunnel scene at the same moment; at the same time, perform coordinate transformation to unify the data in different coordinate systems into a common coordinate system for spatial fusion;

[0063] Extract representative features from each type of data. For video images, deep learning methods such as convolutional neural networks (CNNs) can be used to extract target features in the images, such as the shape, color, license plate, etc. of the vehicle; for millimeter-wave radar point clouds, extract motion features such as the position, speed, and acceleration of the target object; for lidar three-dimensional coordinate data, extract three-dimensional structure features of the object, such as size and shape.

[0064] Use appropriate fusion algorithms, such as weighted average fusion, Kalman filter fusion, etc., to fuse the features of different data sources. Taking weighted average fusion as an example, the fused feature vector can be expressed as: , where \(F_i\) represents the feature vector of the \(i\)-th data source, \(w_i\) is the corresponding data confidence weight, and \(n\) is the total number of data sources. In this way, the reliability and feature information of different data sources are comprehensively considered to obtain a more comprehensive and accurate representation of the tunnel's dynamic features. The fused feature data is organized according to the time series to form a tunnel dynamic feature map. This map can reflect the dynamic changes of the target objects in the tunnel in real time, including their positions, movement trajectories, appearance features, etc., providing a basis for subsequent real-time detection of target objects and prediction of movement trajectories. Based on the constructed tunnel dynamic feature map, an improved YOLOv7 algorithm is used for real-time detection of target objects. YOLOv7 is an advanced object detection algorithm with the characteristics of fast speed and high accuracy. It is improved, such as optimizing the network structure, adjusting hyperparameters, etc., to better adapt to the object detection task in the tunnel environment. Specific improvements can include: adding an attention mechanism module to enable the network to pay more attention to the key feature regions of the target object and improve the monitoring accuracy, adjusting the combination methods of convolutional layers, pooling layers, etc. to adapt to the scale changes and complex backgrounds of target objects in the tunnel scene. According to the characteristics of tunnel monitoring data, hyperparameters such as the learning rate and batch size are adjusted to accelerate the training convergence speed of the model and improve the detection efficiency. At the same time, the optical flow method is combined to calculate the prediction of the movement trajectory of the target object. The optical flow method is a motion estimation method based on image sequences that can calculate the motion vectors of pixel points in the image to obtain the movement trajectory of the target object. The specific steps are as follows:

[0065] Perform optical flow calculation on consecutive video image frames to obtain the optical flow vectors of each pixel point;

[0066] According to the optical flow vectors, determine the movement direction and speed of the target object in the image;

[0067] Use prediction algorithms such as Kalman filtering, combined with the target position in the current frame and the motion parameters estimated by optical flow, to predict the possible position of the target in the next frame, thereby realizing the real-time prediction of the target movement trajectory.

[0068] Deploy a risk level assessment engine to extract spatio-temporal correlation features through a convolutional neural network and input them into a preset fuzzy logic decision tree to generate a comprehensive risk index. It should be clear that the convolutional neural network can automatically learn the spatial and temporal features in the data and has advantages in extracting spatio-temporal correlation features in tunnel monitoring data. The specific process is as follows;

[0069] Take the data in the tunnel dynamic feature map as input, and through operations such as convolutional layers and pooling layers, extract the local and global features in the data and capture the change patterns of the target object in time and space;

[0070] Furthermore, through structures such as fully connected layers, the extracted features are integrated and mapped to obtain a spatio-temporal correlation feature representation that can reflect the tunnel safety risk. Then, the extracted spatio-temporal correlation features are input into a preset fuzzy logic decision tree to generate a comprehensive risk index. The fuzzy logic decision tree is a model based on fuzzy set theory and decision tree structure, which can handle data with uncertainty and ambiguity and is suitable for complex scenarios such as tunnel safety risk assessment. The specific steps are as follows;

[0071] Determine the index system for risk assessment, including traffic flow, vehicle speed, environmental parameters, etc. Each index corresponds to a fuzzy set, such as low, medium, high, etc.;

[0072] Construct a decision tree structure. According to expert knowledge and historical data, determine the role and weight of each index in risk assessment, as well as the risk level division under different index combinations;

[0073] Take the extracted spatio-temporal correlation features as input. Through the branch judgment of the decision tree and combined with the inference rules of fuzzy logic, calculate the comprehensive risk index. The comprehensive risk index can be expressed as: , where m and n respectively represent the number of risk assessment indicators and the number of fuzzy sets, u ij is the membership degree of the i-th index in the j-th fuzzy set, w ij is the corresponding weight, and f ij is the risk contribution factor corresponding to this fuzzy set.

[0074] When the risk index exceeds the dynamic threshold, trigger a hierarchical early warning mechanism and link it with tunnel emergency equipment, and simultaneously generate an event handling plan and push it to the operation and maintenance terminal. It should be clear that in the embodiments of the present application, the setting of the dynamic threshold can be adjusted according to factors such as historical data, tunnel operation conditions, and safety standards to adapt to different tunnel environments and operation requirements. The hierarchical early warning mechanism divides risks into different levels, such as low risk, medium risk, high risk, etc. Each level corresponds to corresponding early warning measures and emergency response strategies. At the same time, link tunnel emergency equipment, such as ventilation systems, lighting systems, fire protection systems, etc., and take corresponding emergency measures. For example, when the detected CO concentration is too high, automatically turn on the ventilation equipment to strengthen air circulation and reduce the CO concentration; when a fire hazard is found, start the fire sprinkler system and adjust the lighting system to guide the evacuation of personnel. In addition, synchronously generate an event handling plan and push it to relevant operation and maintenance personnel through the operation and maintenance terminal. The event handling plan includes detailed emergency handling steps, division of responsibilities, resource allocation, etc., to help operation and maintenance personnel quickly and effectively respond to safety events in the tunnel and ensure the safe operation of the tunnel

[0075] The multi-modal data fusion strategy includes:

[0076] Build a feature alignment module based on the attention mechanism to achieve spatio-temporal alignment of sensor data through cross-modal feature interaction;

[0077] Adopt a dual-channel residual network to process structured data and unstructured data respectively;

[0078] Design a time series sliding window mechanism to dynamically adjust the data sampling frequency and fusion period. It should be clear that in the embodiments of this application, in multi-modal data fusion, a feature alignment module based on the attention mechanism is built to achieve spatio-temporal alignment of sensor data through cross-modal feature interaction. The specific steps are as follows:

[0079] For the input feature dimension, initialize three weight matrices WQ, WK, and WV, whose dimensions are usually the same as the input feature dimension. For example, if the input feature dimension is 768, the dimensions of these three matrices are all 768, 768. Convert data of different modalities into a unified feature representation. For example, for video images, millimeter-wave radar point clouds, and lidar three-dimensional coordinate data, extract their feature vectors respectively. Assuming the feature dimension of each modality is 768, the dimension of the input feature matrix X is the sequence length, 768. Multiply the input feature by the weight matrix to obtain the query matrix Q, the key matrix K, and the value matrix V: where the dimensions of Q, K, and V are the same as the input feature matrix, calculate the dot product of the query matrix and the transpose of the key matrix, and divide by the scaling factor (dk is the dimension of the key vector), to obtain the attention score matrix: , this score reflects the correlation between different positions. To effectively process different types of multi-modal data, a dual-channel residual network is adopted to process structured data and unstructured data respectively. The specific structure is as follows:

[0080] Used to process data with clear structure and rules, such as millimeter-wave radar point clouds and lidar three-dimensional coordinate data. This channel contains several convolutional layers and fully connected layers. The spatial features of the data are extracted through convolutional operations, and then the features are integrated and classified through fully connected layers. For example, use deep residual network structures such as ResNet, where each residual block contains two convolutional layers and a shortcut connection to alleviate the problem of gradient disappearance and improve the training effect of the network;

[0081] Used to process unstructured data such as video images. This channel is mainly composed of a convolutional neural network, and through multi-layer convolutional and pooling operations, automatically learn the feature hierarchy in the image. Similarly, residual connections are adopted to enhance the expression ability and training stability of the network;

[0082] After extracting features from the two channels separately, the features of the two types of data are fused through a fusion layer. Methods such as feature concatenation, weighted summation, or automatically learning the fusion method through a learning module can be used to combine the features of structured and unstructured data to form a more comprehensive feature representation for subsequent tasks such as risk assessment and target detection.

[0083] To adapt to the dynamic changes of data in the tunnel environment, a time series sliding window mechanism is designed to dynamically adjust the data sampling frequency and fusion period. The specific implementation is as follows:

[0084] Set a time window with a fixed length, and collect and process data within this window. For example, the window length can be set to 10 seconds, and data is collected once per second, so there are 10 data samples within the window. Dynamically adjust the data sampling frequency according to the actual situation in the tunnel and the data change rate. In the case of large traffic flow and rapid environmental changes, increase the sampling frequency to obtain more detailed data; while in the case of small traffic flow and stable environment, reduce the sampling frequency to reduce data processing volume and computing resource consumption. For example, when it is detected that the number of vehicles suddenly increases or the environmental parameters change sharply, increase the sampling frequency from once per second to five times per second. The fusion period determines the frequency of multi-modal data fusion. Dynamically adjust the fusion period according to the data change situation within the sliding window and the system's requirement for real-time performance. When the data changes violently, shorten the fusion period to update the fusion result in a timely manner; when the data is relatively stable, extend the fusion period to reduce the computing overhead. For example, when the standard deviation of the data within the sliding window exceeds a certain threshold, adjust the fusion period from once every 5 seconds to once every 2 seconds.

[0085] The risk level assessment engine includes:

[0086] A three-dimensional point cloud density anomaly monitoring sub-module that uses an improved DBSCAN algorithm to identify abnormal vehicle aggregation states;

[0087] A visibility attenuation model that combines Mie scattering theory to calculate the smoke diffusion trend;

[0088] A multi-target tracking trajectory prediction unit that constructs a vehicle kinematic model based on the LSTM network. It should be clear that in the embodiment of the present application, this sub-module uses an improved DBSCAN algorithm to identify abnormal vehicle aggregation states. The specific steps are as follows:

[0089] Preprocess the three-dimensional point cloud data obtained by the lidar, remove noise points and invalid points to obtain a clean point cloud data set. For each point in the point cloud data set, calculate the point density within its neighborhood. The density calculation formula is: , where pi and pj represent the coordinates of the i-th and j-th points respectively, N is the number of points in the neighborhood, and σ is the bandwidth parameter of the Gaussian kernel, which is used to control the smoothness of density calculation. According to the normal vehicle distribution, a density threshold p is set. th . When the point density in a certain area exceeds this threshold, it is considered that there may be abnormal vehicle aggregation. Using the improved DBSCAN algorithm, the areas with density higher than the threshold are clustered to identify the areas of abnormal vehicle aggregation. The improved DBSCAN algorithm improves the recognition accuracy of vehicle aggregation states in tunnel environments by adjusting parameters such as the neighborhood radius and the minimum number of points. The visibility attenuation model combines the Mie scattering theory to calculate the smoke diffusion trend. The specific process is as follows: According to the physical properties of smoke particles, its scattering coefficient bscat is calculated. In the Mie scattering theory, the scattering coefficient is related to factors such as the size of the particles, the refractive index, and the wavelength of the incident light. The calculation formula is: , where n is the refractive index of the smoke particles, λ is the wavelength of the incident light, N(r) is the particle size distribution function, and Q scat (r) is the scattering efficiency factor of a single particle. The extinction coefficient b sct is the sum of the scattering coefficient and the absorption coefficient. In a smoke environment, assuming that the absorption coefficient is relatively small, it can be approximately considered that the extinction coefficient is equal to the scattering coefficient, that is, b cxt ≈b scat . The relationship between the visibility V and the extinction coefficient is: . When the smoke diffusion causes the extinction coefficient to increase, the visibility decreases accordingly. By real-time monitoring the scattering characteristics of smoke particles, the change of visibility is calculated, so as to evaluate the influence degree of smoke on tunnel visibility and provide a basis for risk assessment. The multi-object tracking trajectory prediction unit is based on the vehicle kinematic model of the LSTM network architecture to realize the prediction of vehicle movement trajectories. The specific steps are as follows:

[0090] Taking the historical trajectory data of the vehicle as input, including information such as the position, speed, and acceleration of the vehicle. Assuming the input sequence is X = {x1, x2,..., xt}, where xt represents the state vector of the vehicle at time t. A multi-layer LSTM network is constructed, and each layer contains multiple LSTM units. The LSTM units control the flow of information through the input gate, forget gate, and output gate, and can effectively capture the long-term dependencies in the time series. The hidden layer state ht and cell state ct of the network are updated at each time step, and the formulas are as follows:

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] Among them, σ is the sigmoid function, ⊙ represents element-wise multiplication, and W and b are the weight and bias parameters of the network. Through the trained LSTM network, the historical trajectory data of the vehicle is input to predict its future trajectory position, and the output sequence is , which represents the position prediction of the vehicle in the next n time steps. The prediction results can be used to monitor the measured motion state in real time, detect abnormal trajectories in a timely manner, and provide important support for tunnel safety monitoring.

[0097] A highway tunnel monitoring system, the system includes:

[0098] An edge computing node cluster, deployed in key monitoring areas of the tunnel, integrating FPGA acceleration chips to achieve data preprocessing;

[0099] A central analysis server, equipped with a GPU array to run a deep learning inference engine, including:

[0100] A data quality verification module, dynamically eliminating abnormal sensor data;

[0101] A virtual-real fusion scenario reconstruction module, constructing a digital twin model of the tunnel;

[0102] An emergency linkage control unit, connecting ventilation, lighting, and escape indication devices through an industrial bus protocol;

[0103] A visualization interaction terminal, providing a three-dimensional situation awareness interface and historical data traceability function. It should be clear that in the embodiments of the present application, the edge computing node cluster is deployed in key monitoring areas of the tunnel, and its main function is to collect and preprocess various types of data in the tunnel in real time to reduce the burden on the central server and reduce data transmission latency. The cluster integrates FPGA acceleration chips and utilizes their powerful parallel processing capabilities and programmability to achieve fast preprocessing of sensor data. For example, FPGA can accelerate operations such as video image data encoding, decoding, and feature extraction, improving data processing efficiency. By performing preliminary data screening and processing at the edge nodes, valuable key data is transmitted to the central server, thereby optimizing the performance of the entire system. The central analysis server is equipped with a GPU array for running a deep learning inference engine to further analyze and process a large amount of data from the edge nodes. It includes the following key modules: Data verification module: This module is responsible for dynamically eliminating abnormal sensor data to ensure the accuracy and reliability of subsequent analysis. Data verification is usually based on statistical methods or machine learning models to identify and exclude outliers. For example, the mean and standard deviation of the data can be calculated, and then according to the Z-score formula, it can be determined whether a data point is an outlier: , where x is the data point, μ is the mean, and σ is the standard deviation. If the Z value exceeds a set threshold (such as 3), the data point is considered an outlier and is removed. Virtual-real fusion scenario reconstruction module: This module uses digital twin technology to build a digital twin model of the tunnel. By combining the real-time data of the physical tunnel with the virtual model, it realizes the real-time monitoring and simulation prediction of the tunnel state. The process of building the digital twin model includes steps such as three-dimensional modeling of the tunnel, integration and mapping of sensor data, and dynamic update of the model. For example, the three-dimensional point cloud data obtained by lidar can be used to accurately restore the geometric structure of the tunnel, while video images and other sensor data are used to enrich the details and real-time status information of the model. Emergency linkage control unit: This unit is connected to devices such as ventilation, lighting, and escape indicators in the tunnel through industrial bus protocols (such as Modbus, Profibus, etc.) to achieve rapid linkage control in case of emergencies. When the monitoring system detects an abnormal event (such as a fire, traffic accident, etc.), the emergency linkage control unit can automatically adjust the operating state of the ventilation system to remove smoke according to the preset emergency strategy, control the lighting and escape indicator devices to guide the evacuation of personnel, thereby minimizing accident losses and casualties. The visual interaction terminal provides users with an intuitive three-dimensional situation awareness interface, enabling users to view information such as the traffic conditions and equipment operating status in the tunnel in real time. By combining the digital twin model with the actual monitoring data, this interface displays the real-time state of the tunnel in the form of three-dimensional graphics, and users can conveniently perform operations such as zooming and rotating to view the details of different areas. In addition, the visual interaction terminal also has a historical data traceability function, and users can query and playback the tunnel operation data for a certain period in the past as needed for event analysis, fault troubleshooting, etc. For example, managers can view information such as traffic flow changes and vehicle driving trajectories before and after an accident through this terminal, providing strong support for accident handling and traffic management.

[0104] The edge computing base point configuration includes:

[0105] Multi-protocol adaptation interface, compatible with RS485, CAN, and Zigbee communication standards;

[0106] Dynamic power consumption correlation module, adjusting the power supply voltage according to the data flow;

[0107] The hardware-level encryption unit collects data and uses the SM4 national cryptographic algorithm to ensure transmission security. The multi-protocol adaptation interface is an important part of the edge computing node. It can be compatible with multiple communication standards to ensure that different types of sensors and devices can communicate effectively with the edge computing node. Specifically, this interface is compatible with the following communication standards: A commonly used serial communication interface with good anti-interference ability and long communication distance, suitable for the harsh environment inside the tunnel. Its communication rate is 100 kbps, and the maximum transmission distance can reach 1,200 meters. Controller Area Network, which is an efficient and reliable communication protocol, widely used in the fields of automotive and industrial control. Its communication rate is 1 Mbps, and it can achieve a multi-master and multi-slave communication mode to ensure real-time data transmission. A low-power and short-distance wireless communication technology, suitable for fields such as sensor networks and smart homes. Its communication rate is 250 kbps, operates in the 2.4 GHz frequency band, and has the functions of self-organizing network and multi-hop routing. The multi-protocol adaptation interface realizes data conversion and interaction between different communication standards through corresponding conversion circuits and protocol stack software, ensuring that various sensor data can be uniformly accessed to the edge computing node for processing. The dynamic power consumption correlation module is the key to energy saving of the edge computing node. It can adjust the supply voltage in real time according to the data traffic to achieve the purpose of energy saving. The specific implementation is as follows: By monitoring the data transmission rate and data volume of the communication interface, the current data traffic information is obtained in real time. For example, when the data traffic is large, it means that the sensor is transmitting data frequently. At this time, a higher supply voltage is required to ensure the stability of data processing and transmission; while when the data traffic is small, the supply voltage can be appropriately reduced to reduce power consumption. According to the relationship between data traffic and power consumption, the supply voltage is dynamically adjusted. The power consumption calculation formula is: where P represents power consumption, V represents supply voltage, and I represents current. In order to minimize power consumption while ensuring the normal operation of the system, the following strategy can be adopted: Set a basic supply voltage Vbase. When the data traffic exceeds a certain threshold, the supply voltage increases by a certain proportion; conversely, when the data traffic is lower than the threshold, the supply voltage decreases proportionally. For example, when the data traffic exceeds 100 kbps, the supply voltage increases from 3.3 V to 3.6 V; when the data traffic is lower than 50 kbps, the supply voltage decreases from 3.3 V to 3.0 V.

[0108] A highway tunnel monitoring device, which includes at least one heterogeneous processor, including a CPU, an NPU, and an IP core dedicated to image processing;

[0109] A storage device for storing an executable program according to any one of claims 1-3;

[0110] An industrial-grade communication module that supports 5G slice network and LoRa wide-area networking;

[0111] An anti-electromagnetic interference chassis, compliant with the GB / T17626 electromagnetic compatibility standard. In the application example, the CPU: As a general-purpose processor, it is responsible for running the operating system and coordinating the work of each component. Usually adopting a multi-core architecture, such as the Intel Core i7 series, it has the characteristics of high performance and low power consumption, and can handle complex control logic and data management tasks. The neural network processor is specifically used to accelerate the operation of deep learning algorithms. For example, in tasks such as object detection and behavior recognition, the NPU can significantly improve the processing speed and efficiency, and its performance can reach hundreds of TOPS (trillions of operations per second). For the processing and analysis of video image data, dedicated intellectual property (IP) cores are designed. These IP cores can efficiently execute operations such as image encoding and decoding, feature extraction, etc., reduce the load on the CPU and NPU, and improve the overall performance of the system. The storage is used to store the programs and data required for the device to run, and its configuration is as follows: According to the requirements of the tunnel monitoring system, the capacity of the storage is usually between hundreds of GB and several TB. For example, a combination of a 512GB solid-state drive and a 1TB mechanical hard drive can not only ensure fast read and write speeds but also provide a large amount of data storage. The read and write speed of the solid-state drive can reach hundreds of MB per second, while the read and write speed of the mechanical hard drive is relatively low, but it can provide a larger storage space and is suitable for long-term data storage. To ensure the integrity and reliability of the data, the storage adopts redundancy technologies such as RAID to prevent data loss or damage. The communication module supports multiple communication methods to meet the data transmission requirements in the tunnel environment: Through the slicing technology of the 5G network, it provides a high-bandwidth and low-latency communication channel for the tunnel monitoring system. The sliced network can allocate independent network resources according to different service requirements to ensure the stability and security of data transmission. Inside the tunnel, the LoRa technology can achieve long-distance and low-power wireless communication. Its transmission distance can reach several kilometers and is suitable for the collection and transmission of sensor data. LoRa has high receiving sensitivity and can penetrate the tunnel structure to ensure stable signal reception.

[0112] A highway tunnel monitoring medium stores a computer instruction set;

[0113] It includes a tunnel scenario knowledge graph database that stores typical accident case feature vectors;

[0114] Integrates an online learning module to dynamically update the parameters of the risk assessment model according to newly collected data;

[0115] Encapsulate the device driver library to support the plug-and-play configuration of mainstream industrial sensors. The tunnel scenario knowledge graph database is one of the core components of the monitoring medium, which is used to store the feature vectors of typical accident cases. These feature vectors are the key information extracted from historical accident data and can help the system quickly identify and respond to similar events. The extraction of feature vectors usually involves multi-dimensional analysis of accident data, including accident type, occurrence location, time, number of vehicles involved, environmental conditions, etc. For example, a simple feature vector can be represented as: , where u1 represents the accident type (such as fire, collision, etc.), u2 represents the occurrence location, u3 represents the time, u4 represents the number of vehicles involved, and u5 represents the environmental conditions (such as visibility, humidity, etc.). These feature vectors are clustered and classified through machine learning algorithms so that they can be quickly matched and identified when a new event occurs. The online learning module is another key component of the monitoring medium, which can dynamically update the parameters of the risk assessment model according to newly collected data. This enables the system to continuously adapt to new environments and conditions, improving the accuracy of prediction and the timeliness of response. The online learning module usually uses incremental learning algorithms, such as Stochastic Gradient Descent (SGD) or its variants, to update the model parameters. For example, assuming that the risk assessment model is a linear regression model, the parameter update formula can be expressed as: , where θt is the current model parameter, η is the learning rate, and ∇J(θt) is the gradient of the loss function with respect to the parameter. In this way, the system can adjust the model parameters in real time to adapt to new data patterns. The device driver library is an important part of the monitoring medium, which supports the plug-and-play configuration of mainstream industrial sensors. This means that when a new sensor is connected to the system, the driver library can automatically identify and configure the sensor without manual intervention. The driver library usually contains a series of predefined drivers, each corresponding to a sensor type, and provides a standardized interface for data acquisition and control. For example, for a temperature and humidity sensor, the corresponding driver in the driver library will define how to read the temperature and humidity data, as well as how to set the sampling frequency and other parameters of the sensor.

[0116] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended embodiments and their equivalents.

Claims

1. A highway tunnel monitoring method, characterized in that, It includes the following steps: Collect tunnel environment data in real time through a multi-source heterogeneous sensor array. The data at least includes video images, millimeter-wave radar point clouds, lidar three-dimensional coordinates, temperature and humidity, and CO concentration parameters; Use an adaptive noise suppression algorithm to preprocess the original data and establish a data confidence weight assignment model; Construct a tunnel dynamic feature map based on a multi-modal data fusion strategy. Among them, an improved YOLOv7 algorithm is used for real-time target detection, and the optical flow method is combined to calculate the motion trajectory prediction; Deploy a risk level assessment engine, extract spatio-temporal correlation features through a convolutional neural network, and input them into a preset fuzzy logic decision tree to generate a comprehensive risk index; When the risk index exceeds the dynamic threshold, trigger a hierarchical warning mechanism and link tunnel emergency equipment, and synchronously generate an event handling plan and push it to the operation and maintenance terminal.

2. The highway tunnel monitoring method according to claim 1, characterized in that: The multi-modal data fusion strategy includes: Establish a feature alignment module based on the attention mechanism to achieve spatio-temporal alignment of sensor data through cross-modal feature interaction; Use a dual-channel residual network to process structured data and unstructured data respectively; Design a time series sliding window mechanism to dynamically adjust the data sampling frequency and fusion period.

3. The highway tunnel monitoring method according to claim 1, characterized in that: The risk level assessment engine includes: A three-dimensional point cloud density anomaly monitoring sub-module that uses an improved DBSCAN algorithm to identify abnormal vehicle aggregation states; A visibility attenuation model that combines Mie scattering theory to calculate the smoke diffusion trend; A multi-target tracking trajectory prediction unit that constructs a vehicle kinematic model based on the LSTM network.

4. A highway tunnel monitoring system, characterized in that: The system includes: An edge computing node cluster deployed in key tunnel monitoring areas, integrating an FPGA acceleration chip to achieve data preprocessing; A central analysis server equipped with a GPU array to run a deep learning inference engine, including: A data quality verification module that dynamically eliminates abnormal sensor data; A virtual-real fusion scenario reconstruction module that constructs a tunnel digital twin model; An emergency linkage control unit that connects ventilation, lighting, and escape indication devices through an industrial bus protocol; A visualization interaction terminal that provides a three-dimensional situation awareness interface and a historical data traceability function.

5. The highway tunnel monitoring system according to claim 4, characterized in that: The edge computing base point configuration includes: A multi-protocol adaptation interface that is compatible with RS485, CAN, and Zigbee communication standards; A dynamic power consumption correlation module that adjusts the power supply voltage according to the data traffic; A hardware-level encryption unit that collects the SM4 national encryption algorithm to ensure transmission security.

6. A highway tunnel monitoring device, characterized in that: It includes at least one heterogeneous processor, including a CPU, an NPU, and a dedicated IP core for image processing; A memory for storing an executable program according to any of the methods described in claims 1-3; An industrial communication module that supports 5G slice networks and LoRa wide-area networking; An anti-electromagnetic interference chassis that complies with the GB / T17626 electromagnetic compatibility standard.

7. A monitoring medium for highway tunnels, characterized in that, Stores a computer instruction set; Includes a tunnel scenario knowledge graph database that stores typical accident case feature vectors; Integrates an online learning module that dynamically updates the risk assessment model parameters according to newly collected data; Encapsulates a device driver library that supports plug-and-play configuration of mainstream industrial sensors.

Images