A highway slope detection system based on a visual scheme

Abstract

The present application relates to the field of image processing and detection system, disclose a kind of highway slope detection system based on visual scheme, comprising: multi-modal image acquisition unit, image fingerprint extraction unit, time series information entropy calculation unit and cross-modal collaborative arbitration unit;Image fingerprint extraction unit generates first and second image fingerprint time series;Time series information entropy calculation unit forms first and second modal fingerprint entropy time series curve;Cross-modal collaborative arbitration unit executes arbitration rule set according to two time series curves, distinguishes and judges the three mutually exclusive system states of risk state, environmental noise state and equipment failure state, the present application establishes cross-modal collaborative arbitration mechanism based on image fingerprint entropy, realizes the distinction to real geological risk, environmental noise interference and sensor itself fault in image feature evolution mode, solves the technical problem that existing visual detection scheme is prone to false alarm in complex environment.

Description

Technical Field

[0001] This invention relates to a vision-based highway slope detection system, belonging to the field of image processing and detection system technology. Background Technology

[0002] Current mainstream technologies, especially deep learning-based algorithms, primarily focus on high-precision identification of clearly shaped objects in images, such as component defects on industrial production lines or structural cracks in civil engineering projects. The effectiveness of this approach relies on the fundamental premise that the target signal must present high signal-to-noise ratio morphological features in the image data. However, when this image detection method is applied to open and dynamic natural scenes, particularly for early risk warnings of processes like geological disasters, this premise no longer holds. The initial disturbance source of such risks, such as water seepage within a slope, does not produce clear morphological features in its early stages of evolution. In visible light images, it may not yet have formed cracks, only exhibiting subtle changes in surface texture; in thermal infrared images, it may only appear as a slight change in surface texture. Thermal anomalies with blurred outlines.

[0003] Furthermore, the image features of such early risk signals are very weak. At the same time, the inherent environmental noise in open environments, such as strong sunlight, rapidly changing light and shadow, and obstruction caused by rain and fog, produces visual features with intensity far greater than that weak risk signal. This makes it difficult for existing morphological recognition-based methods to respond to such scenarios with weak signals and strong noise before morphological evidence appears, thus hindering effective early warning. To address these challenges, the industry has tried to improve the accuracy of single-modal algorithms or detect the paths of anomalies in each modality separately. However, existing image processing tools typically analyze image data from different modalities as isolated chains of evidence. This approach makes it easy for weak signals in a single modality to be confused or drowned out by strong noise in that modality, causing the system to fail to confirm the signal or generate a large number of false alarms. Its inherent limitation is that the early risk signal is not essentially an independent object existing in a single modality, but rather a state change that is weakly correlated and occurs synchronously across multiple modalities such as visible light and thermal infrared.

[0004] Even those solutions that attempt to integrate data from multiple different types of sensors (such as laser, inclinometer, and vision) mostly rely on passive measurement of macroscopic physical displacement rather than proactive mining of early-stage image feature precursors in the evolution of risks. For example, Chinese invention patent CN117073764B discloses a monitoring system and method for the condition of roadbed slopes in mountainous areas. Although this solution integrates laser ranging, tilt detection, and AI vision detection modules, its core logic still relies on deploying positioning auxiliary modules (such as flexible vertical strips and reflective units) and tilt detection modules on the slope, and judging the physical deformation that has occurred through laser ranging and tilt angle changes. This approach is essentially a contact-based morphological monitoring that lags behind the initial disturbance of risk. Its AI vision is only used to assist in observing the deployed positioning auxiliary modules, rather than analyzing the texture or thermal radiation of the slope itself or other early signs. This type of solution also cannot solve the core problem of how to achieve early prediction based solely on the coordinated changes in image features before physical morphological defects appear.

[0005] Therefore, the technical problem to be solved by this invention is how to design an image processing and detection method that can extract and determine the weakly correlated collaborative features of this type of cross-modal data from multiple noisy and heterogeneous image data streams, and thus achieve early prediction of risk based solely on the collaborative changes in image features before any physical morphological disease appears. Summary of the Invention

[0006] This invention provides a vision-based highway slope detection system, the main purpose of which is to solve the problem that existing image detection methods rely on morphological features and process multimodal data in isolation, and cannot extract weakly correlated collaborative features across modes in strong noise environments to achieve early risk prediction.

[0007] To achieve the above objectives, the present invention provides a vision-based highway slope detection system, comprising:

[0008] A multimodal image acquisition unit is configured to acquire at least a first modality of visual image data and a second modality of visual image data of a highway slope.

[0009] The image fingerprint extraction unit, connected to the multimodal image acquisition unit, is configured to apply image feature extraction algorithms to the visual image data of the first modality and the visual image data of the second modality respectively, so as to generate their respective first image fingerprint time series sequence and second image fingerprint time series sequence.

[0010] The temporal information entropy calculation unit is configured to receive the first image fingerprint temporal sequence and the second image fingerprint temporal sequence, and calculate the information entropy value for them respectively, so as to form the first modality fingerprint entropy temporal curve and the second modality fingerprint entropy temporal curve.

[0011] A cross-modal collaborative arbitration unit is configured to execute an arbitration rule set based on the first modal fingerprint entropy time series curve and the second modal fingerprint entropy time series curve. This arbitration rule set is used to distinguish and determine three mutually exclusive system states, and includes the following rules: when it is determined that both the first modal fingerprint entropy time series curve and the second modal fingerprint entropy time series curve show a continuous non-periodic growth trend, the system state is determined to be a risk state; when it is determined that at least one time series curve shows instantaneous fluctuations or periodic fluctuations, but does not meet the conditions defined by the risk state, the system state is determined to be an environmental noise state; when it is determined that only the first modal fingerprint entropy time series curve or a single curve in the second modal fingerprint entropy time series curve shows a continuous non-periodic growth trend, the system state is determined to be an equipment failure state.

[0012] Preferably, the cross-modal collaborative arbitration unit is configured to determine a continuous non-periodic growth trend, specifically including: performing trend analysis on the time series curves of the first modality fingerprint entropy and the second modality fingerprint entropy within a sliding time window; calculating the trend slopes of the time series curves of the first modality fingerprint entropy and the second modality fingerprint entropy; and calculating the fitting residuals of the time series curves of the first modality fingerprint entropy and the second modality fingerprint entropy with a standard periodic template, wherein the standard periodic template is generated based on historical data of the respective time series curves in a stable state; when it is determined that the trend slopes of the two time series curves are both continuously higher than a preset slope threshold, and the fitting residuals are both continuously higher than a preset residual threshold, it is determined that the two time series curves simultaneously exhibit a continuous non-periodic growth trend.

[0013] Preferably, the visual image data of the first modality is visible light image data, and the visual image data of the second modality is thermal infrared image data; the image feature extraction algorithm applied by the image fingerprint extraction unit is specifically configured to: divide the visual image data of the first modality and the visual image data of the second modality into multiple image sub-regions respectively; calculate the image feature histogram for each image sub-region; and concatenate the image feature histograms of all image sub-regions corresponding to each modality to generate the first image fingerprint time series sequence and the second image fingerprint time series sequence.

[0014] Preferably, the temporal information entropy calculation unit is specifically configured to: construct a first historical fingerprint pool and a second historical fingerprint pool for the first image fingerprint temporal sequence and the second image fingerprint temporal sequence, respectively; include new image fingerprints in the first image fingerprint temporal sequence into the first historical fingerprint pool, and include new image fingerprints in the second image fingerprint temporal sequence into the second historical fingerprint pool; and calculate the information entropy values ​​of the first historical fingerprint pool and the second historical fingerprint pool, respectively, to form a first modality fingerprint entropy temporal curve and a second modality fingerprint entropy temporal curve.

[0015] Preferably, the cross-modal collaborative arbitration unit is further configured to: temporarily suspend the output of risk warnings and activate the fingerprint sequence volatility discriminator when the system state is determined to be a risk state; the fingerprint sequence volatility discriminator is configured to retrieve and analyze the first image fingerprint time series sequence and the second image fingerprint time series sequence that led to the risk state determination; and calculate the volatility V through the following steps: step a calculates the first-order difference vector sequences of the first image fingerprint time series sequence and the second image fingerprint time series sequence respectively; step b calculates the Euclidean norm of each difference vector in the first-order difference vector sequence sequence to form their respective scalar time series; step c calculates the statistical variance of the scalar time series within a preset time window and defines it as V; the cross-modal collaborative arbitration unit is further configured to execute a second-level arbitration rule: if and only if Only then is the risk status finally confirmed and an early warning signal issued, among which The preset low volatility threshold; when When this happens, the system state is determined to be under external interference.

[0016] Preferably, the system further includes an environmental stimulus response heterogeneity analysis unit, which is configured to operate independently of the temporal information entropy calculation unit and is used to: firstly detect the occurrence of a global environmental stimulus event in the visual image data of a first modality or a second modality through image analysis, wherein the global environmental stimulus event refers to a natural environmental change that causes a drastic change in the global statistical features of the image; upon detection of the global environmental stimulus event, spatially segment the visual image data of the first modality or the visual image data of the second modality into multiple sub-regions; extract and calculate the image fingerprint of the modality corresponding to each sub-region, and the change vector before and after the occurrence of the global environmental stimulus event; calculate the statistical variance or dispersion between the change vectors of multiple sub-regions to quantify the spatial statistical heterogeneity; and determine that the highway slope has a potential structural heterogeneity risk when the statistical variance or dispersion is higher than a preset heterogeneity threshold.

[0017] Preferably, the system further includes a channel confidence assessment unit, which is configured to: calculate the global image statistical features of the original image data of the first modality visual image data and the second modality visual image data before the image fingerprint extraction unit processes them; the global image statistical features include the full frame pixel variance or image contrast; determine whether each modal channel is in a data invalid state based on whether the global image statistical features are lower than a preset invalid threshold; and the cross-modal collaborative arbitration unit is further configured to: execute the arbitration rule set when it determines that all modal channels are not in a data invalid state; and switch its judgment logic to a single-channel emergency warning mode when it determines that only one modal channel is not in a data invalid state, that is, determine the risk state only based on the continuous non-periodic growth trend of the fingerprint entropy time series curve of the effective modal channel.

[0018] Preferably, the system further includes a spatial registration self-verification unit, which is configured to: during system initialization, store a set of static anchor points that can be stably identified in both the visual image data of the first modality and the visual image data of the second modality, and their initial registration relationship; periodically reuse the function of the image fingerprint extraction unit to re-detect the current coordinates of the static anchor points in the currently acquired visual image data of the first modality and the visual image data of the second modality; calculate the registration residual between the current coordinates and the initial registration relationship; and the operation of the cross-modal collaborative arbitration unit is gated by the registration residual: the cross-modal collaborative arbitration unit is allowed to execute the arbitration rule set only if and only if the registration residual is lower than a preset registration threshold.

[0019] Preferably, the system further includes an asymmetric entropy increase analysis unit, configured to: monitor the input of the cross-modal cooperative arbitration unit in parallel; receive the determination result when the system state is determined to be a device fault state; and then classify the asymmetric entropy increase event as a secondary risk event distinct from the risk state; the asymmetric entropy increase analysis unit is also configured to: reuse the function of the fingerprint sequence volatility discriminator to classify the secondary risk event: when determining the statistical variance of the fingerprint time series sequence of the modal channel that caused the asymmetric entropy increase event... Less than the low volatility threshold When this event is classified as a source corruption event, and when the statistical variance of this asymmetric entropy increase event is determined... Not less than the low volatility threshold When this occurs, it is classified as a scene foreign object intrusion event.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] 1. This method no longer relies on identifying specific morphologies in images. Instead, it extracts feature fingerprints from visible light and thermal infrared images separately and continuously calculates the information entropy evolution of each fingerprint sequence. Its judgment logic is constrained by a cross-modal collaborative mechanism: a risk state is confirmed only when the system observes that the fingerprint entropy of at least two modalities exhibits a continuous and non-periodic synchronous growth trend. This collaborative and continuous dual-judgment method, which considers the inherent stability of the image data stream, allows environmental noise that only causes single-modal fluctuations or instantaneous variations, such as changes in light and shadow, and transient rain and fog, to be separated at the statistical level of image features. This avoids the problems encountered by existing image detection methods. The system addresses the issue of susceptibility to high-frequency interference in open environments. Instead of immediately issuing a warning when a sustained and coordinated increase in multimodal fingerprint entropy is confirmed, a secondary discrimination mechanism is activated. This mechanism reuses the time-series image fingerprint that caused the entropy increase and calculates the statistical volatility of the sequence. Since changes in image features caused by internal risks exhibit smooth statistical characteristics, while changes caused by external high-intensity interference, such as construction, show dramatic and high-amplitude statistical characteristics, this mechanism distinguishes between the two types of events by comparing volatility thresholds. This progressive analysis, from signal perception to signal origin discrimination, helps the system avoid misjudging high-intensity human interference sources.

[0022] 2. This method also includes a parallel detection unit, which transforms global environmental excitation events (such as heavy rainfall or sudden changes in illumination) in the image data stream from noise to be filtered out into a detection medium. When such events are detected, the system spatially divides the image field of view into multiple sub-regions and calculates the change vector of the image fingerprint of each sub-region before and after the excitation. By calculating the statistical heterogeneity of the spatial distribution of these change vectors, the system can assess whether the slope's response to the same excitation is uniform. This method of using environmental excitation for dynamic stress analysis exposes latent structural heterogeneity that cannot be observed in static images. Furthermore, this system constructs an integrity verification mechanism for the image data stream itself. On the one hand, before calculating the fingerprint, the system first calculates... The global statistical features of the original image, such as the whole-frame pixel variance, are used to evaluate the real-time data confidence of each modal channel. When a channel, such as visible light, is found to be in an invalid data state due to obstruction, the judgment logic of the cross-modal collaborative analysis unit will automatically switch, for example, downgrading to a single-channel emergency warning mode that only relies on other effective channels, such as thermal infrared, in order to maintain the basic monitoring capability under extreme conditions. On the other hand, the system periodically reuses the image fingerprint extraction function to perform spatial registration residual calculation on the static anchor points common to different modal images, in order to self-check whether the physical pointing of the sensor has shifted. Only when the registration residual is lower than the threshold, that is, when the physical basis of the collaboration is confirmed to be valid, is the judgment of collaborative entropy increase allowed to be executed, thus avoiding the collapse of the logical foundation due to sensor physical misalignment. Attached Figure Description

[0023] Figure 1 This is a block diagram of the data flow and collaborative arbitration logic of the detection system of the present invention;

[0024] Figure 2 This is a schematic diagram illustrating the periodic fluctuation of fingerprint entropy under typical environmental noise interference.

[0025] Figure 3 This is a timing diagram for classifying and processing device faults caused by asymmetric entropy increase events according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and are not intended to limit the scope of protection of the invention.

[0027] This invention provides a vision-based highway slope detection system. The system operates within an image processing and detection framework. Its core components include: a multimodal image acquisition unit, an image fingerprint extraction unit, a temporal entropy calculation unit, and a cross-modal collaborative arbitration unit. The basic data flow path is defined as follows: the raw image data stream acquired by the multimodal image acquisition unit is sent to the image fingerprint extraction unit for feature vectorization. The generated temporal sequence of image fingerprints is then used by the temporal entropy calculation unit to calculate the evolution of its entropy. Finally, the cross-modal collaborative arbitration unit executes a collaborative decision logic based on multiple parallel entropy evolution curves to output a final analysis of the system state. The multimodal image acquisition unit is configured as the system's data source, used to acquire at least first-modal and second-modal visual image data of the highway slope. In a specific engineering deployment, the first-modal visual image data is configured as visible light image data. According to the data, the system consists of a fixed industrial-grade visible light camera facing the monitoring area, continuously acquiring data at a preset frame rate, such as one frame per minute. The second modality of visual image data is configured as thermal infrared image data, which is synchronously acquired by a thermal infrared imager with a similar field of view, installed alongside the visible light camera. These two units together constitute the system's data input. The image fingerprint extraction unit, connected to the output of the multimodal image acquisition unit, is responsible for converting high-dimensional, high-noise raw image data into low-dimensional, characteristic image fingerprint vectors. Based on the early risks of highway slopes, such as internal seepage, clear morphological features, such as cracks, are not immediately apparent in the image data. Instead, they manifest as subtle, gradual changes in texture or thermal radiation distribution. Therefore, the image feature extraction algorithm used in this unit is configured to avoid morphological recognition and instead focus on statistical features. The specific implementation procedure is as follows: When a frame is received, for example… When processing visible light image data of pixels, this unit first spatially divides it into multiple image sub-regions, for example... The unit first constructs a grid; then, for each image sub-region, it independently calculates its image feature histogram, such as a 256-level grayscale histogram or a 59-dimensional local binary pattern histogram; finally, the unit stitches the feature histograms of these 100 sub-regions in spatial order to generate a single, high-dimensional vector, which is defined as the image fingerprint at that moment; the same processing flow is applied in parallel to the synchronously acquired thermal infrared image data, so that the unit continuously generates its own first image fingerprint time series sequence and second image fingerprint time series sequence as time evolves.

[0028] The temporal information entropy calculation unit is configured to receive the two fingerprint temporal sequences mentioned above and transform them from instantaneous state descriptions into quantifications of the evolution of feature space complexity. For a healthy, stable slope, the changes in its image fingerprint over a long timescale are periodic, and the information entropy of its fingerprint set should remain convergent. However, a continuous, non-periodic physical disturbance, such as water seepage, will inevitably lead to the continuous emergence of new, unprecedented patterns in the image fingerprint, thus causing a continuous increase in the information entropy of its fingerprint set. To achieve this quantification, the specific configuration of this unit is as follows: it consists of the first image fingerprint temporal sequence and the second image fingerprint temporal sequence, respectively, and constructs a first historical fingerprint pool and a second historical fingerprint pool in system memory or storage. The historical fingerprint pool can be implemented as a first-in-first-out circular queue, and its capacity is calibrated to... ,For example It can be set to 1000 to correspond to the data of the past 1000 sampling points; when the image fingerprint extraction unit generates a new first image fingerprint, the unit adds it to the first historical fingerprint pool. If this fingerprint pool is full, then... The oldest fingerprint at any given moment is discarded; subsequently, the unit immediately checks the current fingerprint in the first historical fingerprint pool. Each fingerprint vector is used to calculate its information entropy value once, for example, by using clustering algorithms such as K-Means. The vectors are divided into There are several clusters, and the Shannon information entropy of each cluster is calculated based on its occupancy probability. In the formula, The number of clusters set for the clustering algorithm. For the first The number of fingerprints contained in each cluster accounts for a percentage of the total fingerprint pool capacity. The proportion; the calculated entropy value It is added as a data point to the end of the first modality fingerprint entropy time series curve; the exact same process is applied in parallel to the second image fingerprint time series sequence and its second historical fingerprint pool to form the second modality fingerprint entropy time series curve; in this way, the unit translates two feature vector streams into two entropy value time series curves.

[0029] The cross-modal collaborative arbitration unit is the core logical decision-making center of this system. It is configured to execute an arbitration rule set based on the first modal fingerprint entropy time series curve and the second modal fingerprint entropy time series curve. The design of this rule set aims to fundamentally separate real risk signals from environmental noise and source failures by utilizing the synergy between modalities. The arbitration rule set is deterministically defined as the distinction and determination of three mutually exclusive system states, and includes the following rules: State 1 (Risk State): The system state is determined to be a risk state if and only if both the first modal fingerprint entropy time series curve and the second modal fingerprint entropy time series curve show a continuous non-periodic growth trend; State 2 (Environmental Noise State): When at least one time series curve shows instantaneous fluctuations or periodic fluctuations... For example, when the characteristic fluctuations caused by changes in light and shadow or the cycle of sunlight can be fitted by a periodic template, but do not meet the strict conditions of simultaneity, continuity, and non-periodicity defined by the risk state, the system state is judged as an environmental noise state; State 3 (equipment failure state): when only a single curve in the first mode fingerprint entropy time series curve or the second mode fingerprint entropy time series curve shows a continuous non-periodic growth trend, for example, only the fingerprint entropy of the visible light camera source increases unidirectionally due to the slow accumulation of dust on the lens, while the thermal infrared fingerprint entropy remains stable, the system state is judged as an equipment failure state; this cross-modal AND logic arbitration ensures that only coordinated events that occur synchronously in the physical world and span the visible light feature space and the thermal infrared feature space will be identified as the highest priority risk.

[0030] To free the determination of a continuous, non-periodic growth trend in the aforementioned arbitration logic from subjective judgment, the cross-modal collaborative arbitration unit is further configured to operate within a sliding time window, the width of which is... The time series analysis can be set to 24 hours, and trend analysis can be performed on the time series curves of the first modality fingerprint entropy and the second modality fingerprint entropy respectively; the specific procedure includes: First, calculate the trend of the two time series curves in the current period. The slope of the trend within the curve can be obtained, for example, by linear fitting using the least squares method. and The second step is to calculate the fitting residuals of each of the two time series curves with a standard periodic template. and The standard periodic template, which can be a standard 24-hour curve representing the daily variation of entropy values ​​on a typical sunny day, is generated based on historical data from system initialization and confirmation of a stable state for each of its time series curves. The third step involves a judgment: the trend slope of both time series curves is consistently higher than a slope threshold close to zero if and only if the trend slope of both time series curves remains above this threshold. That is, the judgment condition is and Furthermore, the fitting residuals consistently exceeded a preset residual threshold that characterizes the degree of non-periodicity. That is, the judgment condition is and Only then was it finally determined that both time-series curves simultaneously exhibited a sustained non-periodic growth trend; among them, and This can be determined through a standardized calibration procedure, for example, by performing trend analysis on historical data under steady-state conditions and adding three times the standard deviation to the mean slope. The calibration is performed by calculating the fitting residuals between historical data and the periodic template under steady-state conditions, taking the mean and adding three times the standard deviation. This quantification procedure ensures the periodic fluctuations of the characteristics. The value may be positive, but The value will be very low and will not be misjudged.

[0031] Furthermore, to distinguish between gentle, gradual real geological risks in the feature space and severe, high-amplitude external disturbances, such as near-field construction machinery activities, which may also trigger multimodal collaborative entropy increases, a cross-modal collaborative arbitration unit is also configured to perform a secondary arbitration. Its procedure is as follows: when the system state is initially determined to be at risk, the risk warning is temporarily suspended, and a built-in fingerprint sequence volatility discriminator is immediately activated. The fingerprint sequence volatility discriminator is configured to retrieve and analyze the two image fingerprint time series sequences that led to this risk state determination, i.e., the feature vectors themselves, rather than the entropy values. The discriminator then calculates the statistical variance of the first-order difference sequence of these two fingerprint time series sequences within a preset time window, such as the past hour. The fingerprint sequence volatility discriminator calculates this statistical variance. The specific procedures include: First, acquiring the image fingerprint time sequence within a preset time window (e.g., the past hour). ,in for First, obtain the high-dimensional fingerprint vector at time step 1; second, calculate the first-order difference vector sequence of this vector sequence. ; Again, for each vector in the difference vector sequence Calculate its Euclidean norm (L2 norm) to obtain a scalar time series. scalar sequence The changes in the fingerprint vector at adjacent time points in the feature space were characterized; finally, the scalar time series was calculated. The statistical variance within the preset time window is defined as the volatility. ;Should The value characterizes the degree of abrupt change of the eigenvector in the feature space; the cross-modal collaborative arbitration unit is further configured to execute a second-level arbitration rule: if and only if At that time, among them Only after setting a preset low volatility threshold is the risk status finally confirmed and an early warning signal output; when If the characteristics of this situation change drastically and chaotically, the system state will be judged as an external disturbance state, and this warning will be blocked. By collecting fingerprint sequences from two scenarios—known internal slow risks, such as simulated seepage, and known external severe disturbances, such as construction videos—and calculating their statistical variances, we can determine the optimal scenarios. And select an intermediate value that can distinguish between the two types of variance as the calibration value.

[0032] To address the latent structural risks that cannot be observed in static images, this system also includes an independently operating environmental stimulus response heterogeneity analysis unit. The core logic of this unit is to reuse environmental noise, such as heavy rainfall, as a natural, active stimulus source. Its configuration is used to: firstly, through image analysis, such as monitoring instantaneous changes in global brightness or texture variance, detect the occurrence of a global environmental stimulus event in either the first or second modality of visual image data; upon detecting this event, such as recognizing the start of rainfall, the unit immediately spatially segments the image field of view into multiple sub-regions, for example… The grid; then, the unit extracts and calculates the image fingerprint corresponding to each sub-region before the global environmental stimulus event occurs ( The vector of change after () is calculated as ,in To motivate the moment before the event occurred, A preset time, such as 10 minutes, is set after the excitation event occurs. Then, the spatial statistical heterogeneity among these 100 change vectors is calculated, such as the statistical variance or dispersion of these 100 vectors. Finally, when the spatial statistical heterogeneity is higher than a heterogeneity threshold calibrated in a homogeneous scenario, the unit determines that there is a potential structural heterogeneity risk in the highway slope. This state indicates that the slope's response to the same excitation is not uniform, and outputs a latent risk warning that is different from the main risk.

[0033] To ensure the system's basic monitoring capabilities under extreme conditions such as degraded channel data quality, including dense fog and blizzards that render images invalid, this system also includes a channel confidence assessment unit. This unit is configured to analyze the data quality of the raw image data, such as visible light and thermal infrared data frames, before processing by the image fingerprint extraction unit. The specific procedure is as follows: Global image statistical features are calculated for each frame of raw image data. These global image statistical features can be computationally inexpensive indicators, such as the full-frame pixel variance or image contrast. The unit then determines whether the global image statistical features fall below a preset, near-zero invalidity threshold. For example, in a visible light frame obscured by dense fog... In an optical image, the full-frame pixel variance approaches 0 to determine whether each modal channel is in an invalid data state. Furthermore, the decision logic of the cross-modal collaborative arbitration unit is dynamically gated by this evaluation result: when it determines that all modal channels are not in an invalid data state, i.e., the channel confidence is high, it executes the arbitration rule set normally, i.e., the collaborative entropy increase logic; when it determines that only one modal channel, such as the thermal infrared channel, is not in an invalid data state, the decision logic of the arbitration unit is automatically and dynamically switched to the single-channel emergency warning mode, i.e., at this time, the risk state is determined only based on the fingerprint entropy time series curve of the effective modal channel, i.e., the thermal infrared, which shows a continuous non-periodic growth trend.

[0034] To ensure the effectiveness of the physical basis of collaborative analysis, namely the spatial alignment of sensors, during long-term operation, this system also includes a spatial registration self-calibration unit. This unit is configured to: during system initialization, either through operator calibration or automatic system algorithm identification, find and store a set of static anchor points that can be stably identified by feature algorithms such as SIFT in both the first and second modal visual image data. These anchor points include, for example, distant fixed building outline corner points or uniquely shaped rock corner points, and their initial registration relationships, such as a... affine transformation matrix This unit is configured to periodically, for example, when the thermal image is most stable in the early morning, reuse the function of the image fingerprint extraction unit, namely its built-in feature point detector, to re-detect the current coordinates of the static anchor points in the currently acquired image; subsequently, this unit calculates the current coordinates in the initial registration relationship. The registration residuals, such as reprojection errors, are considered. Ultimately, the operation of the cross-modal collaborative arbitration unit is gated by the registration residuals: the cross-modal collaborative arbitration unit is allowed to execute the arbitration rule set only if the registration residuals are below a preset, very small registration threshold, such as 2 pixels. If the residuals exceed the limit, the collaborative arbitration is forcibly suspended and a maintenance alarm for sensor physical misalignment is output.

[0035] To fully utilize all information in the data stream and achieve multi-target perception of equipment health and security status, this system also includes an asymmetric entropy increase analysis unit. This unit is configured to monitor the input of the cross-modal cooperative arbitration unit in parallel. Its core task is to analyze non-cooperative entropy increase events filtered out by the main cooperative logic. When the cross-modal cooperative arbitration unit determines that the system state is a device fault state, that is, when it determines that only one modality's fingerprint entropy shows a continuous non-periodic increase, this unit receives the determination result and then classifies the asymmetric entropy increase event as a secondary risk event that is distinct from the risk state. The asymmetric entropy increase analysis unit is also configured to further reuse the function of the fingerprint sequence volatility discriminator, that is, to use the aforementioned calculation of statistical variance. The logic is to classify secondary risk events at the characteristic level: when determining the statistical variance of this asymmetric entropy increase event... Less than the low volatility threshold This state, characterized by slow, gradual, and non-cooperative changes, is classified as a source contamination event, such as the slow accumulation of dust on a lens; however, when determining the statistical variance of this asymmetric entropy increase event... Not less than the low volatility threshold This state is characterized by drastic, high-amplitude, and non-coordinated changes in features, and is classified as a scene intrusion event, such as a person or animal entering the monitoring area.

[0036] Finally, to address the engineering problem of poor adaptability of fixed thresholds to different complexity scenarios, such as high-texture dense vegetation slopes and low-texture bare rock slopes, this system also includes an adaptive loading unit for analysis parameters. This unit is configured to first perform scene feature profiling during the system initialization phase. Its procedure is as follows: collect and analyze an initial fingerprint pool, such as fingerprint data from the first 24 hours after system startup, to obtain baseline statistical features. These baseline statistical features include the average statistical variance or baseline information entropy of the initial fingerprint pool. Based on these baseline feature values, the complexity of the highway slope scenario is classified, for example, into three levels: high complexity, medium complexity, and low complexity. The system has a built-in parameter library that pre-stores multiple sets of analysis thresholds corresponding to different scenario complexities, such as those for high complexity scenarios. It should be relatively high, while low complexity scenarios The values ​​should be relatively low; the adaptive loading unit for analysis parameters is configured to automatically select and load a matching set of analysis thresholds from the parameter library based on the classification results of scene complexity. These thresholds serve as preset slope thresholds, preset residual thresholds, and preset low volatility thresholds for use by the cross-modal collaborative arbitration unit, fingerprint sequence volatility discriminator, environmental stimulus response heterogeneity analysis unit, channel confidence assessment unit, or spatial registration self-verification unit in subsequent analysis operations. , preset heterogeneity threshold, preset invalid threshold, or preset registration threshold.

[0037] Example 1: In a specific deployment, the vision-based highway slope detection system of the present invention was applied to a monitoring scenario with complex geological conditions and intermittent human activities nearby. The challenge of this scenario is that the system must distinguish between a gradual risk evolution driven by internal factors and a high-amplitude image feature perturbation driven by external factors against the background of inherent high-frequency image noise in the open environment. The system was initialized according to the procedures in the specific implementation method. Its multimodal image acquisition unit was configured as a dual-modal system of visible light and thermal infrared. The adaptive loading unit of analysis parameters has classified the scenario complexity as high complexity based on the high vegetation coverage and light and shadow changes of the scenario, and loaded a matching set of analysis thresholds from the built-in parameter library, including a low-fluctuation threshold for secondary arbitration. Its value is set to On the 10th day of system operation, large construction machinery began earthmoving operations at the toe of the slope in the monitored area for several hours. This operation generated continuous dust, mechanical movement, and engine heat radiation in the images, causing continuous, non-periodic, and drastic changes in the characteristics of both the visible light fingerprint time series and the thermal infrared fingerprint time series generated by the image fingerprint extraction unit. Correspondingly, the time series information entropy calculation unit detected that both the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve exceeded their respective slope thresholds. With residual threshold The system exhibits a coordinated and continuous non-periodic growth trend. At this point, the first-level arbitration logic of the cross-modal collaborative arbitration unit is triggered, determining that the system state meets the preliminary conditions for a risk state and temporarily suspending the output of the warning. The arbitration unit then activates the fingerprint sequence volatility discriminator, which retrieves the first image fingerprint time series sequence and the second image fingerprint time series sequence that caused this entropy increase, and calculates the statistical variance of their first-order difference sequences. Calculated Value The cross-modal collaborative arbitration unit executes the secondary arbitration rules and applies the volatility value. With low volatility threshold loaded from the parameter library Comparison; due to The system determined that the image feature evolution process of the collaborative entropy increase event had high volatility statistical characteristics, which was inconsistent with the low volatility characteristics of internal geological risks. Therefore, it was determined to be an external interference state, and the output of this risk warning signal was ultimately blocked.

[0038] On the 35th day of system operation, no strong external interference was observed in the monitored area. However, slow and continuous groundwater seepage began to occur within the slope. This physical process was reflected in the image feature space, causing slow, non-periodic, subtle changes in the soil surface texture in the visible light image. Simultaneously, the corresponding area in the thermal infrared image also exhibited a blurred outline and a continuous low-temperature anomaly. The temporal information entropy calculation unit again detected that both modal fingerprint entropy time-series curves showed a coordinated and continuous non-periodic growth trend. The first-level arbitration logic of the cross-modal collaborative arbitration unit was triggered again, activating the fingerprint sequence volatility discriminator. This discriminator retrieved and analyzed the image fingerprint time-series sequence that caused the entropy increase and calculated its statistical variance. The result of this calculation Value The cross-modal collaborative arbitration unit executes the secondary arbitration rules because... The system determined that the image feature evolution process of the collaborative entropy increase event had low volatility statistical characteristics, which was consistent with the gradual evolution characteristics of internal geological risks. Therefore, the cross-modal collaborative arbitration unit finally confirmed the risk status and officially output an early risk warning signal. The system operation process showed that the cross-modal collaborative arbitration mechanism and the fingerprint sequence volatility identification mechanism could work together. The system first detected potential anomalies by confirming the collaborative entropy increase in the image feature space, and then identified the origin of the event by identifying the statistical volatility of the anomaly sequence. During the monitoring period, before any physical morphological disease appeared, the system output an early warning signal caused by internal risks based solely on the collaborative evolution and dynamic characteristics of image feature statistics.

[0039] Example 2: To verify the performance of the system of the present invention in distinguishing between real risks, environmental noise, and equipment failures, a test based on image sequence playback was conducted; the test platform used a set including an industrial-grade visible light camera, with its resolution set to [resolution value missing]. Pixels, and a long-wavelength uncooled thermal infrared imager, with a resolution set to The image acquisition system comprises pixels; this system was pre-operated continuously for one month at a typical highway slope site, collecting image sequence datasets including normal conditions, known environmental disturbances such as changes in sunlight and rainfall, and simulated equipment malfunctions such as partial lens occlusion, for this playback test; the test group system using the method of this invention is configured according to a specific implementation method, its image fingerprint extraction unit adopts the local binary mode algorithm, and the historical fingerprint pool capacity of the temporal information entropy calculation unit is... Set as Frame, Trend Analysis Sliding Time Window of Cross-Modal Collaborative Arbitration Unit Set as Frame, corresponding Hours, assuming a frame rate The frame rate per minute is determined by analyzing historical data from normal states, and a slope threshold is set accordingly. units / frame, residual threshold units, low volatility threshold of fingerprint sequence volatility discriminator Set as The experiment included playback and analysis of three main scenario sequences: Scenario 1, environmental noise simulation, selected a 2-hour image sequence containing rapid afternoon cloud drift causing drastic changes in illumination and a normal sunset temperature drop as input; During system operation, the visible light modal fingerprint entropy time series curve showed several brief, large-amplitude spikes, but failed to form a sustained growth trend, while the thermal infrared modal fingerprint entropy time series curve showed a gentle, periodic downward trend, consistent with the physical process of sunset cooling; The cross-modal collaborative arbitration unit analyzed the two curves, and since neither curve simultaneously met the condition of sustained and non-periodic growth, it did not satisfy... and and and The system state was determined to be an environmental noise state.

[0040] Scenario 2: Equipment Failure Simulation. A 4-hour image sequence simulating the slow accumulation of dust on a visible light camera lens is selected. In this sequence, the contrast of the visible light images gradually and unidirectionally decreases, while the thermal infrared images remain clear and stable. During system operation, the visible light modal fingerprint entropy time-series curve exhibits a continuous, non-periodic increase, with its slope... Greater than And residuals Greater than Meanwhile, the thermal infrared modal fingerprint entropy time series curve remained within a narrow range of fluctuations near the baseline level, and its slope... Less than The cross-modal collaborative arbitration unit, based on the arbitration rule set, determines that only a single mode, namely the visible light mode, exhibits a continuous aperiodic entropy increase, and the system state is judged as a device fault state. Furthermore, the asymmetric entropy increase analysis unit receives this determination and reuses the fingerprint sequence volatility discriminator to calculate the statistical variance of the visible light fingerprint sequence during that time period. ,get ,because This equipment failure was further classified as a source contamination incident.

[0041] Scenario 3: Simulating Risk Injection. A sequence of images showing a stable slope is selected, and in the latter half, image processing techniques are used to simulate the gradual textural changes in the visible light image caused by slow groundwater seepage. Specifically, this manifests as a slight darkening of color and a continuous, subtle cooling of the corresponding area in the thermal infrared image, with an amplitude of approximately [missing value]. When processing the first half of the sequence, both entropy curves remained stable. However, after entering the simulated risk injection region in the second half, both the visible light and thermal infrared modal fingerprint entropy time-series curves began to show a continuous, non-periodic increase, and both satisfied a slope greater than [missing value]. With residual greater than The cross-modal collaborative arbitration unit initially determines that the risk state conditions are met, and then activates the fingerprint sequence volatility discriminator to calculate the statistical variance of the two fingerprint sequences within that time period. They are respectively , Since the calculated variances are all less than the low volatility threshold. The secondary arbitration determined that the event had low volatility characteristics, and finally confirmed that the system status was at risk and issued an early warning. Table 1 shows examples of the system status and related parameters at key time points in the above three scenarios.

[0042] Table 1: Example of System Status Determination Test Results.

[0043]

[0044] The experimental results are shown in Table 1. The test group system using the method of the present invention processes image sequences containing different event types. The system can distinguish environmental noise, equipment failure and simulated low-volatility risk events based on the cooperative growth trend of cross-modal fingerprint entropy and the statistical volatility of fingerprint sequences. This shows that the cross-modal cooperative arbitration mechanism based on image fingerprint entropy, combined with second-level volatility recognition, provides a way to distinguish image feature changes from different sources in complex environments.

[0045] Example 3: This example combines Figures 1 to 3 This section describes a vision-based highway slope detection system, such as... Figure 1 As shown, the data stream originates from the multimodal image acquisition unit, which acquires, for example, visible light and thermal infrared image data, and transmits the data stream to the image fingerprint extraction unit. The fingerprint extraction unit generates first and second image fingerprint time-series sequences and then sends these sequences to the time-series information entropy calculation unit. This unit then forms the first and second modal fingerprint entropy time-series curves and inputs them into the cross-modal collaborative arbitration unit. The cross-modal collaborative arbitration unit, acting as the primary arbitrator, is responsible for determining collaborative entropy increase. Its determination result is differentiated into three paths: if it is a single-modal continuous entropy increase, it is determined to be a device fault state; if it is non-cooperative or transient fluctuation, it is determined to be an environmental noise state; if it is initially determined to be a risk, the fingerprint sequence volatility discriminator is activated. This discriminator, acting as the secondary arbitrator, further analyzes the volatility. When the volatility is low, i.e., V < 0.05, the analysis continues. At that time, the system finally confirms the risk status and outputs an early warning. When the volatility is high, i.e., V≥ If the condition is as described above, it is determined to be an external interference state. In addition, the system also includes three auxiliary units: an analysis parameter adaptive loading unit, which loads a threshold according to the scene complexity during initialization; a channel confidence evaluation unit, which evaluates the quality of the original image data and gates the fingerprint extraction unit; and a spatial registration self-verification unit, which verifies the spatial alignment of multimodal images and gates the cross-modal collaborative arbitration unit.

[0046] like Figure 2 As shown, the solid line represents the visible light modal fingerprint entropy change due to illumination, exhibiting periodic and instantaneous spike fluctuations caused by illumination. The dashed line represents the diurnal variation of the thermal infrared modal fingerprint entropy, which exhibits slow, periodic fluctuations caused by diurnal temperature variations. Since neither curve shows a continuous, non-periodic, cooperative growth trend, this state will be determined as an environmental noise state by the cross-modal cooperative arbitration unit. Figure 3 As shown, when the cross-modal collaborative arbitration unit monitors the input data stream in parallel and determines that the device is in a fault state, it sends the device fault determination result to the asymmetric entropy increase analysis unit. The asymmetric entropy increase analysis unit then identifies this as an asymmetric entropy increase event and classifies it as a secondary risk event. It then requests the fingerprint sequence volatility discriminator to perform volatility analysis. After calculating the statistical variance V of the fingerprint sequence, the fingerprint sequence volatility discriminator returns the volatility value V to the asymmetric entropy increase analysis unit. The asymmetric entropy increase analysis unit classifies the event based on the V value: when V is less than the low volatility threshold, it is classified as a source contamination event, and the system outputs a lens inspection and maintenance prompt; when V is greater than or equal to the low volatility threshold, it is classified as a scene foreign object intrusion event, and the system triggers a security alarm. The figure also shows that if a dual-modal collaborative change occurs, the cross-modal collaborative arbitration unit executes the normal risk determination process.

[0047] Example 4: To further illustrate the technical effect of the cross-modal collaborative arbitration mechanism in the present invention, the following comparative experiment was conducted: The same experimental platform and image sequence dataset as in Example 2 were used, but a control group system was configured. The only difference between this control group system and the test group system using the method of the present invention in Example 2 is that it does not employ a cross-modal collaborative arbitration unit, but instead uses a technique that analyzes each modality independently. Specifically, it monitors the temporal entropy curves of the visible light modality and the thermal infrared modality separately. When the temporal entropy curve of any modality meets the preset condition of continuous non-periodic growth, the system is determined to be in a risky state. The image fingerprint extraction algorithm, temporal information entropy calculation procedure, and historical fingerprint pool capacity used in this control group system are described. Frame and trend analysis sliding window Frames, and a slope threshold used to determine the sustained non-periodic growth of a single modality. units / frames and residual threshold All units were consistent with the test group system in Example 2; Scene 1 from Example 2, which included a simulated environmental noise sequence with drastic changes in afternoon light, was input into the control group system; near the 60th frame where the light changed rapidly, the characteristics of the visible light image underwent drastic and non-periodic changes, resulting in an increase segment in the visible light modal fingerprint entropy time series curve lasting approximately 15 minutes. Trend analysis of this segment showed its slope... The calculated value is units / frame, greater than And its fitting residuals with the standard periodic template The calculated value is units, greater than Meanwhile, the thermal infrared modal fingerprint entropy time series curve only shows a gentle diurnal variation trend during this period, and its slope is... The calculated value is units / frame, less than Since the judgment logic of the control group system is that an alarm is triggered when any mode meets the condition, at this point in time, the control group system, based on the independent analysis results of the visible light modal entropy curve, judged the image feature fluctuation caused by the change in illumination as a risk state and output an early warning signal.

[0048] Table 2: Example table of state determination for the control group system in handling environmental noise scenarios.

[0049]

[0050] The sequence from Scenario 3 in Example 2, i.e., the second half, in which simulated risk signals were injected, was input into the control group system. After the simulated risk injection began, for example, at frame 360, both the visible light and thermal infrared modal fingerprint entropies began to increase continuously and non-periodically, and both curves satisfied their respective... and Conditions: The control group system can detect the risk event and output a warning signal because it alarms when any modality meets the conditions. However, in the initial stage of the risk occurrence, the time point of its warning output is similar to that of the test group system using the method of the present invention in Example 2. The comparative test results show that the control group system, which lacks a cross-modal collaborative arbitration mechanism and only performs independent analysis on a single modality, can detect simulated risk events where both modalities change, but it cannot effectively distinguish the non-periodic changes in image features caused by strong single-modal environmental noise such as drastic changes in illumination. It is easy to misjudge such strong noise as a risk event and generate false alarms. In contrast, the test group system using the method of the present invention in Example 2 suppresses such false alarms caused by strong single-modal noise by requiring the fingerprint entropy of at least two modalities to show continuous non-periodic growth at the same time.

[0051] Example 5: This example illustrates an engineering calibration procedure for determining key analytical parameters in the system of the present invention, including the thresholds required for the operation of the cross-modal collaborative arbitration unit and the fingerprint sequence volatility discriminator. This procedure is executed during the initialization phase before the formal monitoring task begins, after the system is first deployed at a specific highway slope site. The initial conditions for the procedure are set as follows: the system hardware, including a visible light camera and a thermal infrared imager with a multimodal image acquisition unit, has been installed and fixed at a predetermined field of view; the spatial registration self-calibration unit has completed the initial anchor point calibration and the registration residual is lower than its threshold, indicating that the sensor physical alignment is effective; at the same time, the site environment is in a relatively stable state during the calibration period, with no obvious geological risk activities or large-scale human interference. The first step of the calibration procedure is initial data acquisition and scene complexity assessment. After the system is started, visible light image data and thermal infrared image data are continuously acquired for 7 days at a frequency of 1 frame per minute.

[0052] During this period, the image fingerprint extraction unit operates in parallel, applying the Local Binary Pattern (LBP) algorithm to each acquired image frame. Specifically, it employs a uniform LBP operator with a radius of 2 pixels and a neighborhood of 8 points, and divides the image into... After the sub-regions are processed, their histograms are concatenated to generate the corresponding first image fingerprint time series and second image fingerprint time series. The analysis parameter adaptive loading unit receives all image fingerprints generated within these 7 days to form an initial fingerprint pool. This unit calculates the average statistical variance of all fingerprint vectors in the initial fingerprint pool for both visible light and thermal infrared light. The calculation process involves first calculating the variance of each dimension within each fingerprint vector, and then averaging the internal variances of all fingerprints to obtain the visible light baseline variance. and thermal infrared baseline variance Based on these two baseline variance values, scene complexity is classified, and the variance thresholds for three complexity levels are set as follows: If and If it is, then it is classified as low complexity; if and If it is, then it is classified as medium complexity; if or If it is, then it is classified as high complexity; in the deployment scenario of this embodiment, the calculation is as follows: The scenario is classified as medium complexity.

[0053] The second step in the calibration procedure is to determine the slope threshold required for trend analysis. With residual threshold The temporal information entropy calculation unit uses the 7-day fingerprint sequence collected in the first step to calculate the corresponding first modality fingerprint entropy temporal curve. Time series curve of fingerprint entropy in the second modality K-Means clustering is used to calculate information entropy, and the number of clusters is... Based on historical fingerprint pool capacity Setting, when hour, The value is taken as Using Euclidean distance as the distance metric, two distances of length are obtained. After obtaining the entropy time series curves for each data point, a standard periodic template is generated. Specifically, the entropy data for 7 days is averaged hourly to obtain a standard daily variation curve for 24 hours. Then, for each point in the original entropy sequence... Calculate the residual sequence between it and the corresponding hourly template value. Next, throughout the entire residual sequence Above, a sliding window of 1440 frames, or 24 hours, is used. Calculate the linear fit slope within each window. and the root mean square value of the residuals within the window Collect all window calculations. Value set and After setting the value set, calculate its mean for each value. and standard deviation Finally, the threshold was set as the mean plus three standard deviations, i.e. , Perform this calculation separately for the visible light and thermal infrared modes to obtain the visible light mode threshold. Thermal infrared mode threshold The analysis parameter adaptive loading unit, based on the medium complexity classification results from the first step, searches for the corresponding threshold correction coefficient from the built-in parameter library and sets the multiplication factor for medium complexity scenarios. The final calibrated threshold is then loaded into the cross-modal collaborative arbitration unit.

[0054] The third step in the calibration procedure is to determine the low volatility threshold required for volatility screening. Determine the low volatility threshold. The engineering calibration procedure specifically includes: First, collecting and storing at least two types of standard fingerprint sequence samples, one type being fingerprint sequences of known internal slow risk events (such as simulated seepage), denoted as the sample set. Another type is the fingerprint sequence of known severe external disturbance events (such as near-field construction), denoted as the sample set. The second step is to target and Each sequence sample in the dataset is processed according to the aforementioned method for calculating volatility. The procedure (i.e., calculating the L2 norm sequence of the difference vectors and then calculating their variance) is used to calculate their respective volatility. Values, thus forming a set of low-volatility sample values. With a set of highly volatile sample values The third step is to calculate the sets respectively. mean and its standard deviation , and sets mean and its standard deviation The fourth step, to obtain a boundary that can effectively distinguish between the two types of events, is to use a low volatility threshold. It can be set between two distributions, for example, it can be set as , or set as The premise is to ensure Less than To ensure sufficient statistical separation between the two types of samples at this threshold, this step utilizes the 7-day stable image fingerprint sequence collected in the first step, and optionally, pre-recorded standard test fingerprint sequences containing known low-volatility events, such as video sequences simulating slow sand color gradation in a laboratory environment, and known high-volatility events, such as video sequences of close-range engineering vehicle operations. The fingerprint sequence volatility discriminator is configured to select 1-hour lengths from each of these fingerprint sequence segments from different sources and calculate the statistical variance of their first-order difference sequences. Collect all stable state segments and low-volatility event segments. Values ​​are used to form a low-volatility sample set; all high-volatility event segments are collected. Values ​​are used to form a high-volatility sample set; these two sample sets are analyzed. The values ​​are distributed, and their respective means and standard deviations are calculated; a threshold value that can distinguish between the two distributions is selected as the standard deviation. One method for determining this is to select a value between the mean of the low-volatility sample set plus three standard deviations and the mean of the high-volatility sample set minus three standard deviations; in this embodiment, this method is used to determine this value. for The analysis parameter adaptive loading unit searches for the corresponding parameter in the parameter library based on the medium complexity classification results from the first step. The baseline value or correction factor is used to set the baseline value corresponding to the complexity scenario. This value is then loaded into the fingerprint sequence volatility discriminator.

[0055] After executing the aforementioned standardized engineering calibration procedures, the system obtains a set of key analysis thresholds that match the baseline and complexity of the current deployment site's image features. These thresholds are derived from statistical analysis of the data collected on-site and are used by the cross-modal collaborative arbitration unit and the fingerprint sequence volatility discriminator to determine the status in subsequent monitoring tasks. In one application scenario, the system of this invention is deployed on a slope with no historical deformation records but potential shallow landslide risk during the rainy season. The system operates stably under continuous sunny weather. The multimodal image acquisition unit continuously collects visible light and thermal infrared image data. The two modal fingerprint entropy time-series curves output by the time-series information entropy calculation unit remain in the low-level stable range, and the cross-modal collaborative arbitration unit determines that the system status is stable. One afternoon, the weather suddenly changes. The environmental excitation response heterogeneity analysis unit identifies the occurrence of a global environmental excitation event of heavy rainfall by monitoring the sharp decrease in global brightness and contrast of the visible light image and the rapid and uniform decrease in global temperature of the thermal infrared image. At the moment the event is identified... This unit immediately divides the current visible light and thermal infrared image field of view spatially into The sub-region grid; the system continuously monitors the rainfall process, and at a point 15 minutes after the rainfall has stabilized. The environmental stimulus response heterogeneity analysis unit was reactivated to extract [data / materials]. Time and Visible light image fingerprint corresponding to each sub-region at any given time , With thermal infrared image fingerprint , ,in The sub-regions are indexed from 1 to 100; the cell then calculates the image fingerprint variation vector for each sub-region in both modalities. as well as The core step then is to calculate the statistical heterogeneity of these change vectors in spatial distribution; for example, to calculate the set of visible light change vectors separately. With the set of thermal infrared change vectors In the expression, the variance of the Euclidean distance between each vector and its mean vector is... and In this scenario, the calculation is as follows: , The system compares these two spatial statistical heterogeneity values ​​with a preset heterogeneity threshold, which is obtained by analyzing and calibrating the response of a known structurally homogeneous region under similar rainfall excitation during the system initialization phase. For example, it is set to... ; due to the calculation and If all values ​​exceed the threshold, the environmental excitation response heterogeneity analysis unit determines that the slope's image feature response exhibits spatial heterogeneity under heavy rainfall excitation. Based on this, the system outputs a prompt indicating potential structural heterogeneity risk. This prompt differs from the main risk warning triggered by cooperative entropy increase and aims to remind maintenance personnel to pay attention to the differentiated response behavior of the area under external environmental pressure.

[0056] Example 6: When the system was put back into operation after a maintenance cycle, a small stain was left on the edge of the visible light camera lens due to the negligence of the installers. Initially, this stain had little impact on the global image features, but it might slowly expand over time due to wind or minor vibrations. On the 5th day of system operation, the asymmetric entropy increase analysis unit monitored the input data stream of the cross-modal collaborative arbitration unit in parallel and noticed the visible light modal fingerprint entropy time series curve. It began to show a slow but continuous, non-cyclical growth trend, with a slope The calculated value is greater than the preset slope threshold. And residual Greater than the preset residual threshold Meanwhile, the time series curve of thermal infrared modal fingerprint entropy It continues to fluctuate steadily around the historical baseline. and None of them exceeded the corresponding threshold; the cross-modal collaborative arbitration unit did not detect the dual-modal collaborative entropy increase and therefore did not determine it as a risk state, but based on its built-in rules, it initially determined this single-modal continuous entropy increase event as a device failure state; after receiving the determination signal of this visible light mode entropy increase, the asymmetric entropy increase analysis unit did not directly output a device failure alarm, but instead initiated a secondary classification procedure; this unit reused the function of the fingerprint sequence volatility discriminator, retrieved the first image fingerprint time series sequence that caused the visible light entropy increase (e.g., data from the past 3 hours), and calculated the statistical variance of its first-order difference sequence. The calculation results are obtained The asymmetric entropy increase analysis unit compares this volatility value with the low volatility threshold applied to the system. (Assuming the scenario complexity is medium and this threshold is applied) comparisons are made; due to The unit determined that this asymmetric entropy increase event had low volatility characteristics. Based on the preset classification rules, namely, single-mode continuous aperiodic entropy increase with low volatility, the asymmetric entropy increase analysis unit ultimately classified this event as a source contamination event and output a maintenance prompt suggesting checking the visible light camera lens, rather than a geological risk warning or a general equipment failure alarm. In contrast, if, during another time period, personnel or large animals accidentally entered and continued to move within the monitoring area, it would also cause asymmetric continuous growth of the visible light modal fingerprint entropy, but due to the drastic changes in its image features, the calculated volatility would be lower. It will be greater than At this point, the asymmetric entropy increase analysis unit will classify it as a foreign object intrusion event in the scene, triggering a security-related alarm.

[0057] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A highway slope detection system based on a visual scheme, characterized by, The method comprises the following steps: a multi-modal image acquisition unit configured to acquire visual image data of at least a first modality and visual image data of a second modality of a highway slope; an image fingerprint extraction unit connected to the multi-modal image acquisition unit and configured to apply an image feature extraction algorithm to the visual image data of the first modality and the visual image data of the second modality respectively to generate respective first image fingerprint time series and second image fingerprint time series; a time series information entropy calculation unit configured to receive the first image fingerprint time series and the second image fingerprint time series and calculate information entropy values for them respectively to form a first modality fingerprint entropy time series curve and a second modality fingerprint entropy time series curve; a cross-modality collaborative arbitration unit configured to execute an arbitration rule set according to the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve, the arbitration rule set being used to distinguish and determine three mutually exclusive system states, and the arbitration rule set comprising the following rules: when it is determined that the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve simultaneously exhibit a sustained non-periodic growth trend, the system state is determined to be a risk state; when it is determined that at least one of the time series curves exhibits transient fluctuations or periodic fluctuations but does not meet the conditions defined for the risk state, the system state is determined to be an environmental noise state; and when it is determined that only a single curve of the first modality fingerprint entropy time series curve or the second modality fingerprint entropy time series curve exhibits a sustained non-periodic growth trend, the system state is determined to be a device failure state.

2. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The cross-modality collaborative arbitration unit is configured to determine a sustained non-periodic growth trend, specifically comprising: performing trend analysis on the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve respectively within a sliding time window; calculating the trend slopes of the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve respectively; and calculating the fitting residuals of the first modality fingerprint entropy time series curve and the second modality fingerprint entropy time series curve with a standard periodic template, wherein the standard periodic template is generated based on historical data of the respective time series curve in a stable state; when it is determined that the trend slopes of the two time series curves are both continuously higher than a preset slope threshold and the fitting residuals are both continuously higher than a preset residual threshold, it is determined that the two time series curves simultaneously exhibit a sustained non-periodic growth trend.

3. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The visual image data of the first modality is visible light image data, and the visual image data of the second modality is thermal infrared image data; the image feature extraction algorithm applied by the image fingerprint extraction unit is specifically configured to: divide the visual image data of the first modality and the visual image data of the second modality into a plurality of image sub-regions respectively; for each image sub-region, calculate its image feature histogram; and splice the image feature histograms of all image sub-regions corresponding to each modality to generate the first image fingerprint time series and the second image fingerprint time series.

4. The highway slope detection system based on visual scheme of claim 1, wherein, The time-series information entropy calculation unit is specifically configured to: construct a first historical fingerprint pool and a second historical fingerprint pool for the first image fingerprint time-series sequence and the second image fingerprint time-series sequence, respectively; add new image fingerprints in the first image fingerprint time-series sequence into the first historical fingerprint pool, and add new image fingerprints in the second image fingerprint time-series sequence into the second historical fingerprint pool; and calculate information entropy values of the first historical fingerprint pool and the second historical fingerprint pool, respectively, to form a first modal fingerprint entropy time-series curve and a second modal fingerprint entropy time-series curve.

5. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The cross-modal collaborative arbitration unit is further configured to: when determining that the system state is a risk state, suspend output of a risk warning, and activate a fingerprint sequence fluctuation discriminator; the fingerprint sequence fluctuation discriminator is configured to call and analyze the first image fingerprint time sequence and the second image fingerprint time sequence that lead to the determination of the risk state; and calculate the fluctuation V by the following steps: step a, calculating a first-order difference vector sequence of the first image fingerprint time sequence and the second image fingerprint time sequence respectively, step b, calculating the Euclidean norm of each difference vector in the first-order difference vector sequence respectively to form a respective scalar time sequence, step c, calculating the statistical variance of the scalar time sequence within a preset time window, and defining it as V; the cross-modal collaborative arbitration unit is further configured to execute a secondary arbitration rule: only when the fluctuation V is less than a preset low fluctuation threshold V , the risk state is finally confirmed and a warning signal is output, wherein is a preset low fluctuation threshold; when , the system state is determined to be an external interference state.

6. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The system further comprises an environmental stimulus response heterogeneity analysis unit, which is configured to operate independently of the time-series information entropy calculation unit and is used for: detecting, through image analysis, occurrence of a global environmental stimulus event in the visual image data of the first modal or the visual image data of the second modal, the global environmental stimulus event being a natural environmental change that causes a dramatic change in global statistical features of the image; when the global environmental stimulus event is detected, spatially segmenting the visual image data of the first modal or the visual image data of the second modal into a plurality of sub-regions; extracting and calculating, respectively, image fingerprints of the modal corresponding to each sub-region, a change vector before and after the global environmental stimulus event; calculating statistical variance or dispersion between the change vectors of the plurality of sub-regions to quantify spatial statistical heterogeneity; and when the statistical variance or dispersion is higher than a preset heterogeneity threshold, determining that there is a potential structural heterogeneity risk in the highway slope.

7. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The system further comprises a channel confidence assessment unit, which is configured to: before processing by the image fingerprint extraction unit, first calculate global image statistical features of the original image data of the visual image data of the first modal and the visual image data of the second modal, respectively, the global image statistical features including frame pixel variance or image contrast; determining whether each modal channel is in a data invalid state according to whether the global image statistical features are lower than a preset invalid threshold; and the cross-modal collaborative arbitration unit is further configured to: when it determines that all modal channels are not in the data invalid state, execute the arbitration rule set; and when it determines that only one modal channel is not in the data invalid state, switch its determination logic to a single-channel emergency warning mode, that is, determine the risk state only according to that the fingerprint entropy time-series curve of the valid modal channel presents a sustained aperiodic growth trend.

8. The highway slope detection system based on visual scheme according to claim 1, characterized in that, The system further comprises a spatial registration self-checking unit, which is configured to: at system initialization, store a set of static anchor points that can be stably identified in the visual image data of the first modal and the visual image data of the second modal and initial registration relationships thereof; periodically reuse the function of the image fingerprint extraction unit to re-detect current coordinates of the static anchor points in the currently acquired visual image data of the first modal and the visual image data of the second modal; and calculate registration residuals between the current coordinates and the initial registration relationships. and operation of the cross-modal collaborative arbitration unit is gated by the registration residual: the cross-modal collaborative arbitration unit is allowed to execute the arbitration rule set only if the registration residual is below a pre-set registration threshold.

9. The highway slope detection system based on visual scheme according to claim 5, characterized in that, The system further comprises an asymmetric entropy increase analysis unit configured for: monitoring in parallel the input of the cross-modal collaborative arbitration unit; receiving the decision result when deciding that the system state is the device failure state; and instead judges the asymmetric entropy increase event as a secondary risk event distinguished from the risk state; the asymmetric entropy increase analysis unit is further configured to reuse the function of the fingerprint sequence fluctuation discriminator to classify the secondary risk event: when judging that the statistical variance of the fingerprint time sequence of the modal channel causing the asymmetric entropy increase event is less than a low fluctuation threshold , it is classified as a source pollution event; and when judging that the statistical variance of the asymmetric entropy increase event is not less than a low fluctuation threshold , it is classified as a scene foreign matter intrusion event.

Images