A method for cable pathway monitoring and risk grading

Abstract

This invention relates to a cable channel monitoring and risk classification method based on condition-adaptive enhancement and cross-modal unified modeling, comprising: acquiring video modal data and sensor modal data, and establishing a globally unified time axis; enhancing the video modal data through a condition-adaptive imaging enhancement mechanism, and calculating the image quality score through weighted fusion; enhancing the enhanced video modal data through an attention mechanism to obtain final video features, and constructing a graph structure based on the topological relationship of sensor locations to obtain sensor features; constructing a cross-modal fusion mechanism to perform deep semantic fusion of the final video features and sensor features to obtain the final fused features; calculating multi-dimensional detection results, screening detection results that meet reliability conditions, and converting the detection results into risk levels and handling instructions to achieve cable channel monitoring and risk classification. Compared with existing technologies, this invention has advantages such as adaptability to complex working conditions, high accuracy in anomaly identification, high reliability, and long-term stable operation.

Description

Technical Field

[0001] This invention relates to the field of power cable channel monitoring technology, and in particular to a cable channel monitoring and risk classification method based on operating condition adaptive enhancement and cross-modal unified modeling. Background Technology

[0002] With the advancement of urban cable deployment and the continuous expansion of underground cable infrastructure, traditional manual inspections suffer from significant shortcomings in comprehensiveness and timeliness, leading to a substantial increase in maintenance pressure. The academic and engineering communities have developed two main research approaches in cable duct monitoring: one focuses on integrated intelligent systems for comprehensive monitoring and platform integration; the other focuses on specialized research in physical / signal modeling and sensor networks. Both approaches aim to improve the efficiency of detecting and issuing early warnings for anomalies such as water accumulation, smoke, external damage, and intrusion.

[0003] Existing monitoring systems are mostly based on fixed cameras and environmental sensors to collect video images and data such as temperature, humidity, gas concentration, water level, and vibration. The data is then processed by the front end or center to perform noise reduction, contrast enhancement, and inter-frame difference analysis. Finally, alarm information is output based on machine learning or deep learning models.

[0004] The common workflow of video intelligent recognition includes target detection, segmentation, tracking, and event determination. Some systems incorporate low-light enhancement or noise reduction to alleviate image quality issues. However, low-light conditions, fog, and strong reflections lead to edge and texture degradation, reducing target separability and resulting in significant false positives and false negatives. Existing enhancement algorithms mostly employ fixed-parameter strategies, which lack adaptability to complex conditions.

[0005] Multi-source data fusion typically employs timestamp alignment and rule-based fusion, comparing video recognition results with environmental measurement thresholds to generate anomaly conclusions. This approach relies on manually set fixed thresholds and empirical weights, lacking a unified, learnable mechanism for updating the confidence level and weights of cross-modal evidence, and thus failing to dynamically adjust the fusion strategy based on data quality.

[0006] Existing systems typically build models for single or a small number of tasks, resulting in insufficient feature sharing between tasks. The number of models expands rapidly with the increase in task types, leading to rising maintenance costs. Furthermore, as edge deployments increase, common implementations use lightweight models for basic detection, then upload intermediate results to the center for comprehensive evaluation and linkage. When the link is unstable or bandwidth is insufficient, inconsistencies can easily arise between edge alarm strategies and central evaluation criteria.

[0007] Chinese invention patent CN120873739A discloses a method for accurate identification of risk signals based on multimodal data fusion, comprising: parallel acquisition of structured business data, unstructured text data, time-series sensor data, and image / video data in a target scenario; extraction of essential features from each data source using a modality-specific neural network; extraction of contextual feature vectors from text data using a semantic encoder; extraction of multi-scale visual features from image / video data using a spatial pyramid network; extraction of dynamic pattern features from sensor data using a temporal convolution module; and extraction of entity association features from structured data using a graph embedding algorithm; construction of a cross-modal interactive graph model, using each modal feature as a node, learning the nonlinear association strength between modalities through a trainable attention mechanism and generating a feature association matrix; driving an adaptive fusion gating module based on the association matrix, dynamically calculating the real-time contribution weights of each modal feature and performing weighted fusion to generate a unified feature representation; inputting the fused features into a multi-level risk classification network, outputting a risk level probability distribution and type label; and jointly optimizing the parameters of the feature extraction module, interactive graph model, and classification network based on the difference between the classification results and the true labels. This invention generates real-time fusion weights through cross-modal correlation matrices, enabling adaptive feature weighting and latent risk correlation mining, significantly improving the accuracy and interpretability of complex scene recognition. However, it still suffers from several drawbacks: insufficient adaptability of acquired image data to different environments; lack of a deep semantic correlation model to effectively utilize complementary information between modalities; and the fact that the impact of image quality degradation on fusion weights and risk thresholds is not explicitly factored into the decision-making process. Furthermore, it struggles to guarantee alarm consistency and closed-loop handling under weak network or link failure conditions, resulting in insufficient system robustness.

[0008] In summary, there is currently a lack of a cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling to solve or partially solve the aforementioned problems. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling. This method aims to solve or partially solve the problems of insufficient adaptability of acquired image data to different environments, lack of deep semantic association model to effectively utilize complementary information between different modalities, failure to explicitly include the impact of image quality degradation on fusion weights and risk thresholds in the decision-making process, difficulty in ensuring alarm consistency and closed-loop handling under weak network or link failure conditions, and insufficient system robustness.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] This invention provides a method for monitoring and risk classification of cable channels based on adaptive enhancement of operating conditions and unified cross-modal modeling, specifically including the following steps:

[0012] S1. Acquire video modal data and sensor modal data, establish a globally unified time axis, and map the video modal data and sensor modal data onto a synchronized time grid;

[0013] S2. Enhance the video modal data through a working condition adaptive imaging enhancement mechanism, and calculate the image quality score by weighted fusion based on multi-dimensional image quality features;

[0014] S3. The enhanced video modal data undergoes spatiotemporal convolution and pooling operations, and is further enhanced through an attention mechanism to obtain the final video features. A graph structure is constructed based on the topological relationship of the sensor locations, and sensor features are obtained through a spatiotemporal graph convolutional network and a gated recurrent unit.

[0015] S4. Construct a cross-modal fusion mechanism based on explicit image quality score and modal confidence, and perform deep semantic fusion on the final video features and sensor features to obtain the final fused features;

[0016] S5. Calculate multi-dimensional detection results based on the final fusion features, screen the detection results that meet the reliability conditions, and convert the detection results into risk levels and handling instructions to realize cable channel monitoring and risk classification.

[0017] As a preferred technical solution, the time axis in step S1 is defined as follows:

[0018]

[0019] In the formula, To standardize the time step, , For time indexing, This represents the total number of time steps. The earliest timestamp among all modalities. This is the timestamp of the first frame of video modal data. For the first Timestamps of sensor modal data This is the minimum value checked by the sensor;

[0020] The video modal data is used to generate aligned frames using time nearest neighbor interpolation, with the following formula:

[0021]

[0022] In the formula, For the optimal frame index, For time axis;

[0023] The sensor modal data is used to generate aligned measurements using piecewise linear interpolation, with the following formula:

[0024]

[0025] In the formula, For each sensor modal data timestamp, This is a collection of physical parameters for environmental measurement. For time indexing, and To meet The index of adjacent sampling points.

[0026] As a preferred technical solution, the enhancement process in step S2 specifically includes:

[0027] Brightness and contrast enhanced:

[0028]

[0029] In the formula, To observe the image, To enhance the image, The maximum pixel value of the image. To measure according to the illumination Establish an adaptive Gamma parameter mapping function. As the baseline Gamma value, , To adjust the parameters, Illuminance threshold;

[0030] Defogging treatment:

[0031]

[0032]

[0033]

[0034] In the formula, To observe the image, For scene radiation rate, Transmittance, Atmospheric light value, For defogging intensity parameters, These are the pixel coordinates of a local region. It is a set of RGB color channels. For the c channel pixel value, This represents the atmospheric light value in channel c. The image after brightness enhancement. The image after dehazing. This is the lower limit threshold for transmittance. For The local area centered on;

[0035] High reflectivity treatment:

[0036]

[0037]

[0038] In the formula, For reflection detection operator, For the red channel of the dehazed image, For the green channel of the dehazed image, For the blue channel of the dehazed image, For high brightness threshold, Low brightness threshold, This is the image after reflection suppression. The image after dehazing. For Gaussian kernel, This indicates a convolution operation.

[0039] As a preferred technical solution, the image quality score in step S2 is obtained through weighted fusion:

[0040]

[0041]

[0042]

[0043]

[0044] In the formula, Image quality score As weight, For normalization function, For edge density, For local contrast, This is the global contrast index. , These are reference parameter values ​​for natural images. For image skewness parameters, For image variance, Image width, Image height, For Heaviside step function, For gradient magnitude, The threshold for edge detection. This is the image after reflection suppression. It is the local neighborhood mean. It is a numerically stable term.

[0045] As a preferred technical solution, the final video feature formula in step S3 is:

[0046]

[0047]

[0048] In the formula, For the final video features, To further enhance the feature representation ability of the spatiotemporal attention mechanism, These are the video features after spatiotemporal convolution and pooling operations. The time window length, For feature dimensions For learnable attention weight matrix, This is a matrix transpose operation used to calculate the correlation attention weight matrix between time steps. For activation functions;

[0049] The characteristic formula of the sensor is:

[0050]

[0051]

[0052]

[0053]

[0054] In the formula, For graph convolution operations, For gated loop unit, It is an adjacency matrix. For measuring points and The Euclidean distance between them For connection indicator functions, For distance attenuation parameters, It is a time series. For degree matrix, For learnable parameter matrix, As a sensor feature, through the hidden state at all time steps Obtained by performing time-sequential pooling.

[0055] As a preferred technical solution, the modal confidence scores in step S4 include video modal confidence scores and sensor modal confidence scores.

[0056] The video modal confidence score is calculated based on signal-to-noise ratio and temporal stability.

[0057]

[0058] In the formula, For video modal confidence, For activation function, This is the signal-to-noise ratio weighting coefficient. For signal-to-noise ratio, For time series stability weighting coefficients, For time series stability indicators;

[0059] The sensor modal confidence level is based on data consistency and anomaly detection results:

[0060]

[0061] In the formula, For sensor modal confidence, This is the data consistency weighting coefficient. To ensure data consistency across multiple sensors, These are the anomaly detection weighting coefficients. This represents the confidence level for anomaly detection.

[0062] As a preferred technical solution, the formula for the final fused features in step S4 is:

[0063]

[0064]

[0065]

[0066]

[0067] , ,

[0068] In the formula, For the final fusion features, This indicates that the output is fused through weighted summation and residual concatenation. For video modal confidence, For sensor modal confidence, The attention score is based on image quality priors and modal confidence constraints. For querying the matrix, The key matrix, For value matrices, For video features, For sensor characteristics, For feature splicing operations, The residual weights for the video channels. For activation function, For the attention dimension, and These are learnable weight parameters. For learnable parameters, The average value of the image quality score over time. It is a row vector consisting entirely of 1s. For the Sigmoid function, To query the transformation matrix, The key transformation matrix is... The transformation matrix is ​​a value transformation matrix.

[0069] As a preferred technical solution, the multidimensional detection result determination criteria in step S5 are as follows:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] In the formula, This is an indicator function; it returns 1 if the condition is true, and 0 otherwise. For classification confidence, As the confidence level standard, To establish consistency indicators, To ensure the accuracy of positioning standards, For image quality, As the minimum standard for image quality, For video modal confidence, To assess the credibility of video modal data, For sensor modal confidence, To ensure the reliability of sensor modal data, Enter the risk classification process at that time; For the probability distribution of each category, and The first The class's weight vector and bias terms. and The first The class's weight vector and bias terms. For transpose; For the final fusion feature; To predict the bounding box, For reference bounding box, For intersection, union, and comparison; As weight, For normalization function, For edge density, For local contrast, , These are reference parameter values ​​for natural images. For image skewness parameters, Let V be the image variance.

[0076] As a preferred technical solution, in step S5, the risk level is divided based on the range of risk score values, and the risk score is quantitatively calculated using a multi-factor weighted model.

[0077]

[0078] In the formula, To score risk, For the first The weighting coefficients of each risk factor For the corresponding risk assessment function, For the final fusion features, For the target category label, For bounding box coordinates, The data vector represents the sensor data; the risk factors include classification confidence factor, positioning accuracy factor, environmental severity factor, and temporal consistency factor.

[0079] The formula for the classification confidence factor is:

[0080]

[0081] In the formula, The predicted probability of the target category;

[0082] The formula for the positioning accuracy factor is:

[0083]

[0084] In the formula, For positional consistency indicators;

[0085] The formula for the environmental severity factor is:

[0086]

[0087] In the formula, For the first The weighting coefficients of each sensor For normalization function, For the first The measured values ​​of each sensor;

[0088] The formula for the time-series consistency factor is:

[0089]

[0090] In the formula, This is the time-series correlation coefficient.

[0091] As a preferred technical solution, the method further includes system verification and parameter optimization by establishing an edge-cloud collaborative feedback mechanism:

[0092] The system verification adopts a dual verification mechanism, including automatic verification and manual verification. The automatic verification re-evaluates the detection results through a pre-trained verification model, while the manual verification involves experts annotating key events and abnormal situations.

[0093] The parameter optimization includes the gradient of cross-modal attention weights, the gating threshold, and the risk scoring weights.

[0094] The cross-modal attention weights The gradient update is as follows:

[0095]

[0096] In the formula, For learning rate, For loss function, The gradient of the cross-modal attention weights;

[0097] The threshold The adaptive adjustment is based on ROC curve analysis, by maximizing the Youden exponent. Determine the optimal threshold:

[0098]

[0099] In the formula, For model sensitivity, Model specificity;

[0100] Risk scoring weights Bayesian updates are performed based on historical event statistics and expert feedback.

[0101]

[0102] In the formula, For prior weights, For the number of observed samples, Historical weight values The weight values ​​calculated for the observed samples;

[0103] The updated parameters are sent back to the edge device via a communication protocol. After receiving the parameter packet, the edge device performs a consistency check and deploys the new parameters through an A / B testing mechanism.

[0104] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0105] (1) Improve the adaptability to complex working conditions: This invention uses a working condition adaptive imaging mechanism to trigger corresponding enhancement processing according to different environmental parameters and calculates the image quality score as prior information for subsequent fusion weight allocation. This effectively solves the problem of false alarms and missed alarms in complex working conditions such as low illumination, fogging, high humidity and strong reflection, and realizes the adaptability and dynamic adjustment capability to complex working conditions, achieving the linkage between imaging quality and decision-making process.

[0106] (2) Deep fusion of cross-modal semantics: This invention adopts a unified spatiotemporal cross-modal modeling framework to unify the spatiotemporal features of video and environmental measurement maps-time features, constructs a globally unified time axis, and maps multi-source data with different sampling frequencies to a unified time axis through timestamp alignment and sliding window resampling. It adopts a cross-modal attention fusion mechanism to explicitly estimate the evidence weights of each modality, realizes deep semantic association under multimodal conditions, improves the accuracy and consistency of anomaly identification, and solves the problem of insufficient stability of anomaly judgment due to the inability to effectively utilize the complementary information between each modality.

[0107] (3) Improve the reliability of the decision-making mechanism: This invention adopts a technical means of gating and judging multi-dimensional parameters such as image quality, confidence level and positioning consistency, and selects the detection results that meet the conditions to enter the risk classification, and converts the detection results into risk level and disposal instructions. This solves the problem of output error caused by poor data feature quality, and achieves the technical effect of improving the reliability of output results, providing an important basis for subsequent parameter optimization and model improvement.

[0108] (4) Edge-end real-time closed-loop and cloud-end collaborative architecture: This invention constructs a closed-loop processing of “image quality-weight-risk” at the edge end, performs evidence verification, operation and maintenance scheduling and strategy learning in the cloud, and sends the optimized parameters back to the edge end. After receiving the parameter packet, the edge end performs consistency verification, realizing the technical effects of continuous learning and adaptive optimization of parameters, improving the stability and accuracy of long-term operation, and solving the problem that alarm consistency and handling closed loop are difficult to guarantee under weak network or link failure conditions, and the system has insufficient robustness. Attached Figure Description

[0109] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0110] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0111] Example 1

[0112] To address the problems existing in the aforementioned prior art, this embodiment provides a cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling, such as... Figure 1 As shown, it includes the following steps:

[0113] S1. Acquire video modal data and sensor modal data, establish a globally unified time axis, and map the video modal data and sensor modal data onto a synchronized time grid.

[0114] Multimodal data acquisition and time alignment are fundamental components of the system, aiming to acquire heterogeneous video and environmental measurement data (i.e., sensor data) from the field acquisition end and establish a unified time reference benchmark to provide a synchronized data stream for subsequent cross-modal fusion. This process is based on multi-sensor data fusion theory and time synchronization protocols, ensuring the traceability and consistency of different modal data in the time dimension through precise timestamp recording and metadata management.

[0115] The system continuously acquires video frame sequences and multi-source environmental measurement data from the field acquisition terminal. The video acquisition channel acquires RGB or infrared video frame sequences, denoted as... ,in Indicates the first t Frame image, Total number of frames For spatial resolution, This refers to the number of channels. Each frame of the image is equipped with a precise timestamp. superscript It is stated that microsecond-level synchronization is achieved through Network Time Protocol (NTP), while simultaneously recording the camera ID. and spatial coordinates Metadata. Environmental measurement acquisition covers a variety of physical parameters, denoted as [data set]. superscript This indicates different measurement channels, including illuminance. ,temperature relative humidity Combustible gas concentration Water level Vibration acceleration Each measurement is equipped with a timestamp. and measuring point markings The coordinates of the measuring point are as follows The acquisition process adheres to strict data quality control protocols: video frame integrity is ensured through hash verification to guarantee lossless transmission, and environmental measurements are checked for range limitations. and rate of change constraints Remove physically invalid values. Metadata is stored in a structured manner. This provides complete contextual information for subsequent processing.

[0116] The unified timeline alignment and resampling module addresses the differences in sampling frequency and time reference between multi-source heterogeneous data. By establishing a globally unified timeline and using interpolation resampling techniques, it maps video frame sequences and environmental measurement data onto a synchronized time grid, laying the foundation for subsequent unified spatiotemporal modeling. This process, based on signal processing theory and numerical analysis methods, ensures the accuracy of the resampling process and the temporal consistency of the data.

[0117] The system first establishes a globally unified timeline and sets a unified time step. The value is determined based on system real-time requirements and computational resource constraints, typically ranging from 50 to 200 milliseconds. The aligned time axis is defined as follows. ,in For time indexing, The earliest timestamp across all modalities is used to ensure consistency in the timeline's starting point. For video modalities, a nearest-time interpolation strategy is employed to generate aligned frames. The optimal frame index This method preserves the original quality of video frames and avoids distortion caused by pixel-level interpolation. For environmental measurement modalities (i.e., sensor modal data), since their sampling frequency is usually much lower than the video frame rate, a piecewise linear interpolation method is used to obtain aligned measurement values.

[0118]

[0119] in and To meet The adjacent sampling point index. Linear interpolation guarantees the assumption of continuity of physical quantities and has good approximate accuracy for slowly varying parameters such as temperature and humidity.

[0120] The aligned data is used to construct temporal features through a sliding window mechanism. Window length. Configured according to task characteristics, it covers sufficient temporal context to capture dynamic change patterns. The sliding window moves in steps along the timeline. Move to create overlapping sequences ,in A multichannel tensor representing environmental measurements. It is a video frame sequence. This is the current time index. To ensure data quality, the system employs a robust anomaly detection and repair mechanism: outliers are identified based on a statistical test using the median absolute deviation (MAD), with the following criteria: ,in This is a sensitivity parameter, typically set between 2.5 and 3.0. This is the current sampled value. The sample sequence is within a sliding window; detected outliers are imputed and repaired using local multinomial regression or temporal Kalman filtering to ensure the smoothness and physical plausibility of the sequence. Errors in alignment and resampling are evaluated using timestamp residuals. The system guarantees alignment error To ensure that the timing accuracy meets the requirements of subsequent cross-modal fusion, This represents the total number of sampling points. For the actual aligned timestamp, For the target timestamp.

[0121] S2. Enhance video modal data through a working condition adaptive imaging enhancement mechanism, and calculate image quality score through weighted fusion based on multi-dimensional image quality features.

[0122] For complex working conditions, adaptive imaging enhancement and quality assessment are performed on video images to output enhanced frames. Image quality score This process is based on multi-level adaptive transform theory and no-reference image quality assessment theory, and is implemented through cascaded enhancement algorithms and statistical learning methods.

[0123] First, the system establishes the theoretical foundation for condition-adaptive enhancement based on Retinex theory. This involves analyzing the observed images... Modeled as reflection component and lighting components The product of: The reflection and illumination components are separated by logarithmic domain transformation: This provides theoretical support for subsequent adaptive processing.

[0124] During the brightness and contrast enhancement phase, the system measures illuminance. Establish an adaptive Gamma parameter mapping function: ,in As the baseline Gamma value, , To adjust the parameters, This is the illuminance threshold. The enhancement transformation formula is: Dynamic adaptation to different illuminance conditions is achieved by adaptively adjusting the Gamma parameter. To observe the image, The image after brightness enhancement. This represents the maximum pixel value of the image.

[0125] When humidity At this time, the system activates a dehazing algorithm based on a physical scattering model. An atmospheric scattering model is used. ,in, To observe the image, For scene radiation rate, Transmittance, Atmospheric light value. Transmittance estimation uses an improved dark channel prior: ,in For defogging intensity parameters, These are the pixel coordinates of a local region. It is a set of RGB color channels. For the c channel pixel value, This represents the atmospheric light value in channel c. For The local area centered on the scene. The scene restoration formula is: The rationality of the defogging process is ensured through physical model constraints, among which... The image after brightness enhancement. The image after dehazing. This is the lower limit threshold for transmittance.

[0126] For conditions of strong reflection, the system constructs a reflection detection operator. :when and hour Otherwise, it is 0. For the red channel of the dehazed image, For the green channel of the dehazed image, For the blue channel of the dehazed image, For high brightness threshold, The threshold is set to low brightness. Reflection suppression is achieved through local adaptive filtering: ,in This is the image after reflection suppression. The image after dehazing. For Gaussian kernel, This indicates a convolution operation.

[0127] Image quality score The calculation is based on a weighted fusion of multi-dimensional image quality features. First, the Sobel operator is used to calculate the gradient magnitude: , , Image after reflection suppression, gradient magnitude Edge density is defined as: ,in For Heaviside step function, , For the image width and height, This is the edge detection threshold.

[0128] Local contrast calculation is based on Weber's contrast theory, where local contrast is defined as: ,in It is the local neighborhood mean. For numerically stable terms, This is the image after reflection suppression. The global contrast ratio is: .

[0129] The no-reference quality assessment uses Natural Scene Statistical (NSS) theory to calculate the normalized luminance coefficient: ,in and These represent the mean and standard deviation of the entire graph, respectively. The generalized Gaussian distribution parameters of the MSCN coefficients are obtained through... Modeling, in which Estimate by moment matching method and ,in Control the shape of the distribution. Control the variance of the distribution.

[0130] Ultimately, the image quality score Obtained through weighted fusion:

[0131]

[0132] in Ensure weight normalization. The Sigmoid normalization function, , Use the reference parameter values ​​for the natural image to ensure A higher value indicates better image quality. Through the cascaded processing of the above theoretical methods and mathematical derivations, the system can adaptively enhance images according to operating conditions and quantitatively calculate the prior image quality. This provides a reliable basis for quality assessment of subsequent cross-modal fusion.

[0133] In addition to the image quality evaluation method that integrates BRISQUE, edge density, and contrast, this invention can also use an end-to-end image quality evaluation network based on deep learning, or an image quality index based on frequency domain analysis for image quality evaluation.

[0134] S3. The enhanced video modal data undergoes spatiotemporal convolution and pooling operations, and is further enhanced through an attention mechanism to obtain the final video features. A graph structure is constructed based on the topological relationship of the sensor locations, and sensor features are obtained through a spatiotemporal graph convolutional network and a gated recurrent unit.

[0135] Cross-modal feature extraction and representation learning aims to map heterogeneous video spatiotemporal information and environmental measurement data to a unified feature space, providing semantically aligned representations for subsequent fusion. This process is based on spatiotemporal convolutional network theory and graph neural network theory, achieving deep representation learning of multimodal data through parallel feature extraction paths.

[0136] Video-side feature extraction employs a spatiotemporal feature extraction network to enhance the frame window. Processing is then performed. Let the input video sequence be... ,in The time window length, For spatial resolution, This refers to the number of channels. Spatiotemporal feature extraction is achieved through 3D convolution operations: ,in Represents 3D convolution. For activation function, and The first Layer weights and biases The value matrix is ​​used. After multiple layers of spatiotemporal convolution and pooling operations, the video feature representation is obtained. ,in For feature dimensions. The spatiotemporal attention mechanism further enhances feature representation capabilities: The final video features are ,in For learnable attention weight matrix, This is a matrix transpose operation used to calculate the correlation attention weight matrix between time steps. This is the activation function.

[0137] A graph with measurement points as nodes is established on the measurement side. , where the node set Represents sensor measurement points, edge set Determined by spatial topology or functional adjacency relationships. Adjacency matrix The construction is based on the spatial distance and electrical connection relationship between the measuring points: ,in For measuring points and The Euclidean distance between them For connection indicator functions, This is the distance decay parameter. The time series... The data is fed into a spatiotemporal graph convolutional network, where This represents the number of sensor nodes. To measure the number of channels. The graph convolution operation is defined as: ,in For degree matrix, This is a learnable parameter matrix. Temporal modeling is implemented using a gated recurrent unit (GRU): Sensor features are obtained by performing temporal pooling on the hidden states at all time steps: .

[0138] S4. Construct a cross-modal fusion mechanism based on image quality score and modal confidence to perform deep semantic fusion of the final video features and sensor features to obtain the final fused features.

[0139] The cross-modal attention fusion mechanism is the core innovation of the system. By explicitly introducing image quality priors and modal confidence into the attention calculation process, it achieves deep semantic fusion of video and environmental measurements. Based on Transformer attention theory and cross-modal learning theory, this mechanism ensures the effectiveness and robustness of the fusion process through improved attention calculation formulas and residual connection strategies.

[0140] The fusion process first constructs a query, key, and value matrix. The query matrix is ​​generated from video features: ,in To query the transformation matrix, For the attention dimension. The key matrix and value matrix are generated from sensor features: , ,in and These are the transformation matrices for the keys and values, respectively. The calculation formula for the traditional attention mechanism is as follows: Based on this, the present invention introduces image quality prior and modal confidence constraints.

[0141] The system estimates the confidence scores of each modality in parallel, providing a basis for weight allocation in subsequent fusion. Video modal confidence scores are calculated based on signal-to-noise ratio and temporal stability. ,in For signal-to-noise ratio, As a time series stability index, This is the signal-to-noise ratio weighting coefficient. This represents the temporal stability weighting coefficient. Sensor modal confidence is based on data consistency and anomaly detection results. ,in To ensure data consistency across multiple sensors, For anomaly detection confidence level, This is the data consistency weighting coefficient. To ensure data consistency across multiple sensors, This represents the weighting coefficient for anomaly detection.

[0142] The improved formula for calculating attention score is as follows: ,in The average time value of the image quality score: , The time window length, For the first The quality score of each video frame. It is a row vector consisting entirely of 1s. This is the sensor modal confidence vector. and These are learnable weight parameters. For activation function, For querying the matrix, The key matrix, Attention dimension. Image quality priors. Ensure that the attention weights for low-quality images are adjusted appropriately, and adjust the modality confidence term accordingly. This allows high-confidence sensor data to receive higher attention weight.

[0143] The fused output is achieved by connecting a weighted sum with the residuals: ,in This indicates a feature concatenation operation. For attention score, For video features, The residual weights for the video channels. For learnable parameters, The function is Sigmoid. Residual connections ensure that video information is effectively transmitted even when sensor data quality is poor. Final fused features. It incorporates complementary information from video and sensors, providing rich semantic representations for subsequent anomaly detection and risk classification. The confidence propagation mechanism further enhances the reliability of the fused features. This ensures that the confidence level of the final features matches the data quality of each modality.

[0144] In addition to cross-modal attention fusion, this invention can also employ multilayer perceptron fusion, graph neural network fusion, or probabilistic fusion methods based on variational autoencoders to achieve cross-modal fusion mechanisms.

[0145] S5. Calculate multi-dimensional detection results based on the final fusion characteristics, screen the detection results that meet the reliability conditions, and convert the detection results into risk levels and handling instructions to realize cable channel monitoring and risk classification.

[0146] Consistency verification and multi-dimensional gating mechanisms are key aspects of ensuring system reliability. A comprehensive quality control system is constructed by comprehensively evaluating multiple dimensions, including classification confidence, location consistency, image quality, and modal confidence. Based on multivariate decision theory and a Bayesian inference framework, this mechanism ensures that only high-confidence detection results enter the final risk grading process, thereby significantly reducing false alarm rates and improving the overall robustness of the system.

[0147] The system first determines the fusion features Calculate the confidence score for event classification The probability distributions of each category are obtained using a softmax classifier: ,in and The first The class's weight vector and bias terms. and The first The class's weight vector and bias terms. This is the transpose. Classification confidence is defined as the maximum probability value: This reflects the model's confidence in the prediction results. Simultaneously, the system calculates the positioning consistency index. By comparing predicted bounding boxes With reference bounding box overlap: , where IoU is the intersection-union ratio, which measures the accuracy of the positioning results.

[0148] Multi-dimensional gating conditions are implemented through logical AND operations: ,in This is an indicator function, equal to 1 when the condition is true, and 0 otherwise. The physical meaning of each dimension is as follows: Ensure that the classification results have sufficient confidence. To ensure the accuracy of target positioning, The image quality must meet the minimum standard. and The data reliability of video and sensor modalities are constrained separately, where For classification confidence, As the confidence level standard, To establish consistency indicators, To ensure the accuracy of positioning standards, For image quality, As the minimum standard for image quality, For video modal confidence, To assess the credibility of video modal data, For sensor modal confidence, The reliability of sensor modal data is determined only when all conditions are met simultaneously. Only then are the test results considered reliable and can proceed to the subsequent risk grading process.

[0149] When the gating condition is not met, i.e. The system initiates an intelligent rollback strategy. The rollback mechanism comprises three levels: first, parameter fine-tuning is performed, attempting re-evaluation by lowering some thresholds or adjusting weight parameters; second, evidence reconstruction is executed, correcting the current detection results using historical data and contextual information; finally, the result is marked as "awaiting manual review," recording detailed gating failure information including specific values ​​for each dimension and the cause of failure. The decision logic for the rollback process is: if Then adjust the classification threshold and re-evaluate; if Then trigger image enhancement and re-extract features; if or Then, a single-modal detection mode is activated. Through this hierarchical backoff strategy, the system ensures timely processing of high-confidence results while avoiding the omission of important events, achieving a good balance between accuracy and recall. Gating status and backoff information are fully recorded in the system log, providing crucial information for subsequent parameter optimization and model improvement.

[0150] The risk grading and response decision-making module transforms the detection results from gating verification into actionable risk levels and response instructions, achieving closed-loop control from anomaly identification to actual response. Based on multi-factor risk assessment theory and an expert system decision-making framework, this module uses quantified risk indicators and intelligent response strategies to ensure the system can automatically generate corresponding countermeasures according to different risk levels.

[0151] In addition to multi-dimensional hard gating, this invention can also employ soft gating mechanisms, reinforcement learning-based adaptive gating, or Bayesian inference-based probabilistic gating methods for gating determination.

[0152] Risk scoring is quantified using a multi-factor weighted model: ,in For the first The weighting coefficients of each risk factor For the corresponding risk assessment function, For the final fusion features, For the target category label, For bounding box coordinates, This is a sensor data vector. Key risk factors include: classification confidence factor. Reflecting the uncertainty of the test results, in the formula The predicted probability of the target category, the positioning accuracy factor. The formula measures the degree of deviation in target positioning, where... As a positional consistency indicator, environmental severity factor Taking into account the degree of anomaly in environmental parameters such as temperature, humidity, and gas concentration, in the formula, For the first The weighting coefficients of each sensor For normalization function, For the first Measurements from each sensor, time-series consistency factor The stability of the test results over time was evaluated, among which This is the time-series correlation coefficient. Weighting coefficient. Determined through historical data and expert knowledge, meeting the requirements. Normalization constraints ensure risk scoring .

[0153] Based on risk scoring Based on the numerical range, the system classifies the risk into five levels: Level I ( Level 1 indicates normal condition, requiring no special treatment; Level II ( Level III (Attention status, enhanced monitoring mode activated; Level III) Level IV is a warning status, generating an inspection work order and notifying relevant personnel; Level V is a dangerous condition that triggers automatic linkage equipment such as ventilation systems and lighting equipment; This is an emergency. Immediately activate all emergency response measures and report to the management center. Risk level determination uses hysteresis logic to avoid frequent switching: the current level is... The new computing level is Only when The risk level is updated only when needed, among which This is the hysteresis threshold.

[0154] Decision-making is based on intelligent matching of risk level and event type. The response strategy table is defined as follows: ,in Includes a specific sequence of actions to be taken. Risk level, This refers to the event type. For different risk levels, the system executes corresponding handling logic: when... When necessary, execute emergency response procedures, including equipment control commands, personnel evacuation signals, and activation of emergency lighting; when In time, the information is pushed to the operation and maintenance management system. All actions are logged in detail, including action type, execution time, and response result, forming a complete event handling chain. Time delay control for handling decisions ensures that the entire process from risk calculation to instruction issuance is completed within a preset time limit. For Level V emergency events, the response delay is required to be <5 seconds; for Level IV events, <10 seconds; and for Level III events, <30 seconds, thereby guaranteeing the system's real-time response capability and security protection effectiveness.

[0155] In addition, the present invention also provides cloud-based review and strategy optimization steps.

[0156] The cloud-based review and strategy optimization module is a key component for the system's continuous learning and adaptive improvement. By establishing a collaborative feedback mechanism between the edge and the cloud, it enables continuous optimization of system performance and knowledge accumulation. Based on distributed machine learning theory and online learning algorithms, this module forms a complete system optimization loop through steps such as evidence analysis, manual review, and parameter updates.

[0157] The cloud first systematically reviews the evidence and processing trajectory synchronized from the endpoint. The evidence data structure includes event identifiers. timestamp Original video frames Enhanced back frame Image quality score sequence Environmental measurement data Fusion characteristics Gating status Risk Score , handling actions Complete information is required. The review process employs a dual verification mechanism: automatic review re-evaluates the detection results using a pre-trained verification model and calculates the consistency index. Manual review involves experts annotating key events and anomalies to generate [the data]. Labels are used for model calibration. The results are for edge detection. This is the result of cloud verification. The verification results are divided into three categories: Confirmed, Corrected, and Uncertain, with detailed correction information recorded.

[0158] Based on the review results, the system executes operation and maintenance scheduling and resource allocation. The operation and maintenance work order generation algorithm establishes an optimization model based on event type, risk level, geographical location, and historical processing time. ,in For processing time, To reduce processing costs, and These are weighting coefficients. Constraints include personnel availability constraints. Time window constraints Skill matching constraints etc., among which For the task Assigned to personnel Indicator variables, For personnel Availability, For the task The optimal scheduling scheme is obtained by solving a mixed integer programming problem within a specific time window.

[0159] Policy learning and parameter optimization employ a combination of online learning and reinforcement learning. Key parameter updates include: cross-modal attention weights. gradient update ,in For loss function, For learning rate, The gradient of the cross-modal attention weights; the gating threshold. The adaptive adjustment is based on ROC curve analysis, by maximizing the Youden exponent. Determine the optimal threshold, where, For model sensitivity, Model specificity; risk score weights Bayesian updates are performed based on historical event statistics and expert feedback. ,in For prior weights, For the number of observed samples, Historical weight values The weight values ​​are calculated for the observed samples. The parameter update strategy uses an exponentially weighted moving average to ensure stability. ,in For update rate, The current parameter value. Update the parameter values ​​calculated based on the new data.

[0160] Updated parameters are transmitted back to the edge via a secure communication protocol. A version management mechanism is established for the parameter synchronization package, recording information such as parameter changes, effective time, and rollback strategy for each update. Upon receiving the parameter package, the edge performs consistency checks to confirm the correctness of the parameter format and the validity of the values. New parameters are gradually deployed using A / B testing, performance changes are monitored, and automatic rollback is implemented in case of anomalies. A smooth transition strategy is adopted for parameter activation to avoid the impact of sudden changes on system stability. The entire cloud-based review and strategy optimization process forms a complete closed loop of "evidence collection → review and verification → strategy learning → parameter update → effect evaluation," ensuring continuous improvement and adaptive optimization of the system during actual deployment, enhancing long-term stability and accuracy.

[0161] Example 2

[0162] Based on the foregoing embodiments, this embodiment provides a cable channel monitoring and risk classification system based on adaptive enhancement of operating conditions and unified cross-modal modeling, to realize the aforementioned cable channel monitoring and risk classification method based on adaptive enhancement of operating conditions and unified cross-modal modeling. The system specifically includes:

[0163] Data Acquisition and Alignment Module: This module is mainly used to acquire video and multi-source environmental measurement data from the cable tunnel site and establish a unified time reference. The video acquisition unit continuously acquires RGB / IR video frames and records frame-level timestamps and camera point identifiers; the environmental measurement unit acquires measurement data such as illuminance, temperature, humidity, gas, water level, and vibration and records sampling time and measurement point identifiers; the synchronization and resampling unit performs timestamp alignment, sliding window resampling, and basic cleaning on data with different sampling frequencies on a unified time axis, and outputs a time-aligned multi-source sequence.

[0164] Adaptive Imaging Enhancement Module: This module adaptively enhances and assesses the quality of video images under conditions of low light, fog, high humidity, and strong reflection. The brightness / contrast enhancement unit adaptively adjusts the enhancement strategy based on the illuminance estimation results and outputs enhanced frames; the defogging / de-vaporization and reflection suppression unit performs corresponding defogging processing and suppresses strong reflective areas based on humidity and scattering characteristics; the image quality evaluation unit calculates the overall image quality score Q and provides Q to subsequent modules for weight estimation and threshold gating.

[0165] Cross-modal spatiotemporal unified modeling and fusion module: This module extracts features from video and environmental measurements within a unified spatiotemporal framework and performs cross-modal fusion. The video spatiotemporal feature extraction unit extracts spatiotemporal feature tokens from the enhanced video frame sequence; the measurement graph-temporal feature construction unit constructs a graph structure based on the topological relationships of measurement points and extracts the temporal and correlation features of environmental measurements; the cross-modal attention fusion unit performs unified fusion of video and measurement features, with the fusion weights jointly constrained by the image quality score Q and the confidence scores of each modality.

[0166] Risk assessment and verification module: This module generates anomaly identification results and risk classifications based on fused features, maintaining consistency in the judgments. The anomaly identification and risk classification unit outputs the anomaly category, location, and risk level; the consistency verification and threshold gating unit receives the image quality score Q and the fusion confidence level, performs multi-dimensional gating judgments, and rolls back results that fail the consistency verification.

[0167] Edge-Cloud Collaboration and Handling Module: This module transforms edge-side alarms into handling actions and completes evidence preservation and strategy optimization. The edge-linked handling unit triggers local linkage and handling suggestions based on the risk level; the cloud-based review and maintenance scheduling unit synchronizes alarm evidence to the cloud, completes manual / automatic review and maintenance scheduling, and optimizes and updates system parameters and strategies based on historical results.

[0168] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for monitoring and risk classification of cable channels based on adaptive enhancement of operating conditions and unified cross-modal modeling, characterized in that, Specifically, the following steps are included: S1. Acquire video modal data and sensor modal data, establish a globally unified time axis, and map the video modal data and sensor modal data onto a synchronized time grid. The sensor modal data is the environmental measurement data. S2. Enhance the video modal data through a working condition adaptive imaging enhancement mechanism, and calculate the image quality score by weighted fusion based on multi-dimensional image quality features; S3. The enhanced video modal data undergoes spatiotemporal convolution and pooling operations, and is further enhanced through an attention mechanism to obtain the final video features. A graph structure is constructed based on the topological relationship of the sensor locations, and sensor features are obtained through a spatiotemporal graph convolutional network and a gated recurrent unit. S4. Construct a cross-modal fusion mechanism based on explicit image quality score and modal confidence, and perform deep semantic fusion on the final video features and sensor features to obtain the final fused features; The formula for the final fusion feature is: ， ， In the formula, For the final fusion features, This indicates that the output is fused through weighted summation and residual concatenation. For video modal confidence, For sensor modal confidence, The attention score is based on image quality priors and modal confidence constraints. For querying the matrix, The key matrix, For value matrices, For video features, For sensor characteristics, For feature splicing operations, The residual weights for the video channels. For activation function, For the attention dimension, and These are learnable weight parameters. For learnable parameters, The average value of the image quality score over time. It is a row vector consisting entirely of 1s. For the Sigmoid function, To query the transformation matrix, The key transformation matrix is... Value transformation matrix; S5. Calculate multi-dimensional detection results based on the final fusion features, screen the detection results that meet the reliability conditions, and convert the detection results into risk levels and handling instructions to realize cable channel monitoring and risk classification; The criteria for determining the multi-dimensional detection results are as follows: In the formula, This is an indicator function; it returns 1 if the condition is true, and 0 otherwise. For classification confidence, As the confidence level standard, To establish consistency indicators, To ensure the accuracy of positioning standards, For image quality, As the minimum standard for image quality, For video modal confidence, To assess the credibility of video modal data, For sensor modal confidence, To ensure the reliability of environmental measurement data, Enter the risk classification process at that time; For the probability distribution of each category, and The first The class's weight vector and bias terms. and The first The class's weight vector and bias terms. For transpose; For the final fusion feature; To predict the bounding box, For reference bounding box, For intersection, union, and comparison; As weight, For normalization function, For edge density, This is the global contrast index. , These are reference parameter values ​​for natural images. For image skewness parameters, For image variance; The risk level is defined based on the range of risk score values, and the risk score is quantified using a multi-factor weighted model. In the formula, To score risk, For the first The weighting coefficients of each risk factor For the corresponding risk assessment function, For the final fusion features, For the target category label, For bounding box coordinates, The data vector is a sensor data vector; the risk factors include classification confidence factor, positioning accuracy factor, environmental severity factor, and temporal consistency factor. The formula for the classification confidence factor is: In the formula, The predicted probability of the target category; The formula for the positioning accuracy factor is: In the formula, For positional consistency indicators; The formula for the environmental severity factor is: In the formula, For the first The weighting coefficients of each sensor For normalization function, For the first The measured values ​​of each sensor; The formula for the time-series consistency factor is: In the formula, This is the time-series correlation coefficient.

2. The cable channel monitoring and risk classification method based on adaptive enhancement and cross-modal unified modeling according to claim 1, characterized in that, In step S1, the time axis is defined as follows: In the formula, To standardize the time step, , For time indexing, This represents the total number of time steps. The earliest timestamp among all modalities. This is the timestamp of the first frame of video modal data. For the first timestamps of environmental measurement data This is the minimum value checked by the sensor; The video modal data is used to generate aligned frames using time nearest neighbor interpolation, with the following formula: In the formula, It is a video frame sequence. For the optimal frame index, For time axis; The environmental measurement data is generated using piecewise linear interpolation to produce aligned measurement values, with the following formula: In the formula, Timestamp for each environmental measurement data point. This is a collection of physical parameters for environmental measurement. For time indexing, and To meet The index of adjacent sampling points.

3. The cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling as described in claim 1, characterized in that, The enhancement process in step S2 specifically includes: Brightness and contrast enhanced: In the formula, To observe the image, To enhance the image, The maximum pixel value of the image. To measure according to the illumination Establish an adaptive Gamma parameter mapping function. As the baseline Gamma value, , To adjust the parameters, Illuminance threshold; Defogging treatment: In the formula, To observe the image, For scene radiation rate, Transmittance, Atmospheric light value, For defogging intensity parameters, These are the pixel coordinates of a local region. It is a set of RGB color channels. For the c channel pixel value, This represents the atmospheric light value in channel c. The image after brightness enhancement. The image after dehazing. This is the lower limit threshold for transmittance. For The local area centered on; High reflectivity treatment: In the formula, For reflection detection operator, For the red channel of the dehazed image, For the green channel of the dehazed image, For the blue channel of the dehazed image, For high brightness threshold, Low brightness threshold, This is the image after reflection suppression. The image after dehazing. For Gaussian kernel, This indicates a convolution operation.

4. The cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling as described in claim 1, characterized in that, In step S2, the image quality score is obtained through weighted fusion: In the formula, Image quality score As weight, For normalization function, For edge density, For local contrast, This is the global contrast index. , These are reference parameter values ​​for natural images. For image skewness parameters, For image variance, Image width, Image height, For Heaviside step function, For gradient magnitude, The threshold for edge detection. This is the image after reflection suppression. It is the local neighborhood mean. It is a numerically stable term.

5. The cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling as described in claim 1, characterized in that, The final video feature formula in step S3 is: In the formula, For the final video features, To further enhance the feature representation ability of the spatiotemporal attention mechanism, These are the video features after spatiotemporal convolution and pooling operations. The time window length, For feature dimension, For learnable attention weight matrix, This is a matrix transpose operation used to calculate the correlation attention weight matrix between time steps. For activation functions; The characteristic formula of the sensor is: In the formula, For graph convolution operations, For gated loop unit, It is an adjacency matrix. For measuring points and The Euclidean distance between them For connection indicator functions, For distance attenuation parameters, It is a time series. For degree matrix, For learnable parameter matrix, As a sensor feature, through the hidden state at all time steps Obtained by performing time-sequential pooling.

6. The cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling as described in claim 1, characterized in that, The modal confidence scores in step S4 include video modal confidence scores and sensor modal confidence scores. The video modal confidence score is calculated based on signal-to-noise ratio and temporal stability. In the formula, For video modal confidence, For activation function, This is the signal-to-noise ratio weighting coefficient. For signal-to-noise ratio, For time series stability weighting coefficients, For time series stability indicators; The sensor modal confidence level is based on data consistency and anomaly detection results: In the formula, For sensor modal confidence, This is the data consistency weighting coefficient. To ensure data consistency across multiple sensors, These are the anomaly detection weighting coefficients. This represents the confidence level for anomaly detection.

7. The cable channel monitoring and risk classification method based on working condition adaptive enhancement and cross-modal unified modeling as described in claim 1, characterized in that, The method also includes system verification and parameter optimization through the establishment of an edge-cloud collaborative feedback mechanism. The system verification adopts a dual verification mechanism, including automatic verification and manual verification. The automatic verification re-evaluates the detection results through a pre-trained verification model, while the manual verification involves experts annotating key events and abnormal situations. The parameter optimization includes the gradient of cross-modal attention weights, the gating threshold, and the risk scoring weights. The cross-modal attention weights The gradient update is as follows: In the formula, For learning rate, For loss function, The gradient of the cross-modal attention weights; The threshold The adaptive adjustment is based on ROC curve analysis, by maximizing the Youden exponent. Determine the optimal threshold: In the formula, For model sensitivity, Model specificity; Risk scoring weights Bayesian updates are performed based on historical event statistics and expert feedback. In the formula, For prior weights, For the number of observed samples, Historical weight values The weight values ​​calculated for the observed samples; The updated parameters are sent back to the edge device via a communication protocol. After receiving the parameter packet, the edge device performs a consistency check and deploys the new parameters through an A / B testing mechanism.

Images