A method and system for monitoring road low-visibility based on video analysis

Abstract

The application relates to the technical field of traffic management, and discloses a road low-visibility monitoring method and system based on video analysis, which comprises the following steps: quantizing the degradation characteristic of a reference object region dynamically identified in a target road monitoring video, generating a degradation value sequence of the multi-characteristics of the reference object region; extracting the space-time characteristics of the degradation value sequence, obtaining a steady-state characteristic vector of the reference object region, and simultaneously performing multiple linear regression calculation; warning and grading the road visibility in the target road monitoring video by using the obtained dynamic visibility index of the target road monitoring video, and generating a warning scheme. Through multi-frame trajectory correlation and streaming verification, four-dimensional characteristics of the reference object region are synchronously extracted, collinear characteristics are automatically removed, a high-precision dynamic visibility index is output, problems such as repeated calculation of signals and prediction errors are avoided, and the accuracy and stability of road low-visibility monitoring are improved.

Description

Technical Field

[0001] This invention relates to the field of traffic management technology, and in particular to a method and system for monitoring low visibility on roads based on video analysis. Background Technology

[0002] Current technical solutions rely on single visual features for visibility assessment, which cannot cope with complex environmental interference. Specifically, local feature degradation can cause overall monitoring failure, and single-point feature degradation can cause cascading collapse of the monitoring link. As a result, there is a lack of time-series dynamic filtering capability, which means that a single feature cannot respond to multi-dimensional degradation signals at the same time, and cannot maintain the output stability of the assessment link under complex interference.

[0003] Current methods treat image features as independent decision variables and use fixed weights for linear superposition, ignoring the nonlinear correlation between features. When features are strongly coupled, independent weight allocation leads to the repeated calculation of the same degraded signal, which increases the prediction bias. At the same time, multicollinearity is not effectively identified, which interferes with the stability of the scheme and cannot meet the requirements of high-precision road visibility monitoring. Summary of the Invention

[0004] This invention provides a method and system for monitoring low visibility on roads based on video analysis. Its main purpose is to solve the problems of fragile cascading and low accuracy in low visibility monitoring of roads based on video analysis.

[0005] To achieve the above objectives, the present invention provides a road low visibility monitoring method based on video analysis, comprising:

[0006] S1. Dynamically identify the inherent reference objects of the image features in the target road surveillance video to obtain the continuous reference object region in the target road surveillance video;

[0007] S2. Perform degradation feature quantization on the reference region to obtain a degradation value sequence of multiple features of the reference region;

[0008] S3. Perform spatiotemporal feature extraction on the degraded value sequence to obtain the steady-state feature vector of the reference region;

[0009] S4. Perform multiple linear regression calculation on the steady-state feature vector to obtain the dynamic visibility index of the target road monitoring video;

[0010] S5. Based on the dynamic visibility index, perform early warning classification on the road visibility in the target road monitoring video to obtain the early warning level of the road visibility.

[0011] In a preferred embodiment, the step of dynamically identifying inherent reference objects in the image features of the target road surveillance video to obtain continuous reference object regions in the target road surveillance video includes:

[0012] Using a reference object as the target, the surveillance video of the target road is detected to obtain an initial set of candidate boxes for the surveillance video of the target road;

[0013] Multi-frame trajectory association is performed on the initial candidate box set to obtain the motion trajectory data of the reference object;

[0014] The motion trajectory data of the reference object is verified by streaming to obtain a continuous reference object area in the target road surveillance video.

[0015] In a preferred embodiment, when the reference region is quantized to obtain a sequence of degradation values ​​for multiple features of the reference region, the process includes:

[0016] Single-frame degradation feature extraction is performed on the reference area to obtain the single-frame feature vector of the reference area;

[0017] The single-frame feature vector is quantized and iterated to obtain the frame sequence feature set of the reference area;

[0018] The frame sequence feature set is temporally integrated to obtain a sequence of degraded values ​​of multiple features in the reference area.

[0019] In a preferred embodiment, the single-frame feature vector includes:

[0020] The gradient magnitude of the grayscale image in the reference region is used as the quantified value of the edge sharpness of the target reference region;

[0021] LBP feature extraction is performed on the target region of the reference object to obtain the LBP encoded map of the target reference object region;

[0022] Histogram variance statistics are performed on the LBP encoded map to obtain the texture contrast quantization value of the target reference area;

[0023] Extract the channel mean of the reference area after format conversion, and quantize the channel mean into a color saturation quantization value;

[0024] The degradation rate of the current frame visual features and the baseline features in the reference area is used as the target identifiable distance estimate for the reference area.

[0025] In a preferred embodiment, when performing spatiotemporal feature extraction on the degradation value sequence to obtain the steady-state feature vector of the reference region, the process includes:

[0026] The mean of the degraded value sequence is used as the mean subsequence of the degraded value sequence;

[0027] The standard deviation of the features in the degraded value sequence under the same sliding window is taken as the feature fluctuation quantum sequence of the degraded value sequence;

[0028] Arrange the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain the steady-state characteristic vector of the reference region.

[0029] In a preferred embodiment, the process of arranging the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain a steady-state eigenvector includes:

[0030] The mean subsequence and the feature fluctuation quantum sequence are stored in time alignment to obtain a time-series label dataset of the degradation features of the reference region;

[0031] Spatiotemporal feature extraction is performed on the time-series label dataset to obtain a preliminary feature vector of the degradation features of the reference area;

[0032] The preliminary feature vectors are sequentially fused to obtain the steady-state feature vectors of the reference region.

[0033] In a preferred embodiment, when performing multiple linear regression calculation on the steady-state feature vector to obtain the dynamic visibility index of the target road surveillance video, the following steps are included:

[0034] The steady-state feature vector is trained with a parameter matrix to obtain the regression coefficient vector of the reference region;

[0035] The visibility fusion base value of the reference area is obtained by weighted summation of the regression coefficient vector;

[0036] The visibility fusion baseline value is activated and transformed to obtain the dynamic visibility index of the reference area.

[0037] In a preferred embodiment, when the steady-state feature vector is trained with a parameter matrix to obtain the regression coefficient vector of the reference region, the process includes:

[0038] The steady-state feature vector is concatenated based on the type of feature label to obtain the augmented matrix of the reference region;

[0039] Extract the covariance of the augmented matrix, and construct a multidimensional covariance matrix of the reference region based on the covariance;

[0040] The inverted covariance matrix is ​​multiplied element-wise with the covariance matrix to obtain the regression coefficient vector of the reference region.

[0041] In a preferred embodiment, when classifying the road visibility in the target road surveillance video based on the dynamic visibility index to obtain the warning level of the road visibility, the following steps are included:

[0042] The real-time visibility index is filtered for invalid values ​​to obtain the effective visibility sample set in the target road surveillance video.

[0043] The effective visibility sampling set is used for early warning matching to obtain the early warning level of the target road surveillance video;

[0044] Risk assessment is performed on the warning level to obtain signal modulation of the target road surveillance video.

[0045] To address the aforementioned problems, the present invention also provides a road low visibility monitoring system based on video analysis, the system comprising:

[0046] S1. The reference object extraction module is used to dynamically identify the inherent reference objects of the image features in the target road monitoring video to obtain the continuous reference object region in the target road monitoring video.

[0047] S2. The degradation value sequence generation module is used to quantize the degradation features of the reference area to obtain a degradation value sequence of multiple features of the reference area.

[0048] S3. The steady-state feature vector generation module is used to extract spatiotemporal features from the degraded value sequence to obtain the steady-state feature vector of the reference region.

[0049] S4. The dynamic visibility index module is used to perform multiple linear regression calculation on the steady-state feature vector to obtain the dynamic visibility index of the target road monitoring video.

[0050] S5. The warning level generation module is used to classify the road visibility in the target road monitoring video based on the dynamic visibility index to obtain the warning level of the road visibility.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. This invention breaks through the limitations of traditional single-dimensional perception, features images into multiple indicators, and constructs a four-dimensional degradation feature collaborative quantification system. By simultaneously fusing four complementary indicators, namely edge sharpness, texture contrast, color saturation, and identifiable distance, it reduces the misjudgment rate due to local feature degradation or single feature failure, avoids single-frame errors, realizes cross-modal feature cross-validation, and improves the stability of the monitoring scheme.

[0053] 2. This invention uses a covariance-driven dynamic regression architecture to analyze the nonlinear correlation between features, dynamically adjust the weights of four types of features, fit the linear relationship between each degenerate feature, automatically remove highly linearly correlated features, avoid prediction bias caused by ignoring feature correlation, and maximize feature accuracy. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a road low visibility monitoring method based on video analysis provided in an embodiment of the present invention.

[0055] Figure 2 This is a functional block diagram of a road low visibility monitoring system based on video analysis provided in an embodiment of the present invention;

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] This application provides a method for monitoring low visibility on roads based on video analysis. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0059] Reference Figure 1 The diagram shown is a flowchart illustrating a road low visibility monitoring method based on video analysis according to an embodiment of the present invention. In this embodiment, the reference information generation method based on artificial intelligence and smart home includes:

[0060] In an embodiment of the present invention, the step of dynamically identifying inherent reference objects in the image features of the target road surveillance video to obtain continuous reference object regions in the target road surveillance video is specifically used for:

[0061] Using a reference object as the target, the surveillance video of the target road is detected to obtain an initial set of candidate boxes for the surveillance video of the target road;

[0062] Multi-frame trajectory association is performed on the initial candidate box set to obtain the motion trajectory data of the reference object;

[0063] The motion trajectory data of the reference object is verified by streaming to obtain a continuous reference object area in the target road surveillance video.

[0064] Specifically, based on known reference points, the approximate location range of potential targets is quickly determined in each frame of the monitoring footage. Initial candidate bounding boxes are automatically drawn around these potential target locations, marking the areas of the image requiring further analysis.

[0065] Specifically, the location frames of reference objects captured at different time points are linked together to confirm the uniqueness of the reference objects they represent. By linking the positions at multiple time points, a continuous path of the reference object's movement over time is created, recording continuous motion information such as the direction and speed changes of the reference object's movement.

[0066] Specifically, the system monitors the emerging motion path in real time, determines whether the newly added position point at each moment is continuous with the previous position point, and whether the changes in movement direction and speed are natural. It identifies and eliminates obvious erroneous position points or route jumps caused by brief occlusion, image noise, or recognition errors, ensuring that the position area of ​​the reference object in each frame of the final output video is continuous without abrupt breaks or jumps.

[0067] Furthermore, it avoids indiscriminate detailed examination of the entire video frame, focusing subsequent analysis on local areas near the reference object where the target is more likely to appear, providing a clear set of area location information that requires in-depth examination for subsequent more refined identification or tracking steps.

[0068] Furthermore, the static position information of a single frame is transformed into a dynamic movement route over time, providing a basis for the scheme to continuously monitor and predict the future position of reference objects, helping to distinguish multiple similar reference objects or interference objects that may exist in the picture, and ensuring that only the route of the target reference object is tracked.

[0069] Furthermore, the reliability and stability of the reference object movement path and position information are enhanced, providing a set of verified, temporally continuous and spatially coherent reference object area information for all subsequent visibility analyses that rely on the reference object positions. This enables the entire scheme to eliminate recognition errors and image interference within a single frame or short period of time, and maintain stable operation over a long period of time.

[0070] In summary, by processing only key local areas in the image, the amount of data that needs to be analyzed is greatly reduced, making the entire surveillance video analysis process faster. Furthermore, since it only needs to be performed within the area marked by the initial candidate box, it avoids invalid operations on irrelevant backgrounds, improving the accuracy and speed of recognition. This makes it practically feasible to automate the analysis of long-term continuous surveillance videos covering a large area.

[0071] In summary, by clearly understanding the actual route, direction changes, and speed of the reference object within the monitoring area, key information such as the average speed, instantaneous speed, acceleration, and direction of travel of the reference object can be directly obtained from the formed movement route. Through the correlation and confirmation of multiple consecutive frames, possible transient interference or positioning errors in a single frame can be effectively eliminated, making target recognition more reliable.

[0072] In summary, more accurate target position, velocity, distance, and other parameters are obtained based on the consistent and stable position of the reference object, avoiding errors introduced by reference object position jitter or sudden changes. At the same time, it ensures that the reference object area is uninterrupted in the time dimension, so that the continuous tracking and analysis of visibility dependent on the reference object will not fail due to the interruption of reference object information, and the availability of the solution can be maintained even in complex real-world environments.

[0073] In an embodiment of the present invention, when the reference region is subjected to degradation feature quantization to obtain a degradation value sequence of multiple features of the reference region, the following steps are included:

[0074] Single-frame degradation feature extraction is performed on the reference area to obtain the single-frame feature vector of the reference area;

[0075] The single-frame feature vector is quantized and iterated to obtain the frame sequence feature set of the reference area;

[0076] The frame sequence feature set is temporally integrated to obtain a sequence of degraded values ​​of multiple features in the reference area.

[0077] Specifically, the specific characteristics of the reference area in the current single frame caused by various interferences are identified and recorded. The observed specific characteristics are transformed into a set of numerical descriptive information that can be understood and processed by subsequent steps, and a digital label reflecting the image quality status of the reference area in the current frame is generated for each frame.

[0078] Specifically, the image state features scattered across each frame are gathered together in chronological order of their appearance in the video. By comparing and correlating the feature values ​​of consecutive frames, patterns or trends in image state changes over time are identified, generating a continuous record of image state features covering the entire video timeline for reference areas.

[0079] Specifically, information from multiple features reflecting different aspects of image quality at each time period is aggregated into a more representative comprehensive value. The frame-by-frame feature sequence, which may contain redundancy or subtle fluctuations, is refined into a numerical sequence that better highlights the main trends of change, directly quantifying the overall degree of image quality degradation in the reference area at that moment.

[0080] Furthermore, the output sets of features are vectors of core input data for analyzing how the image quality of the reference area evolves with each frame. This provides a basis for subsequent visibility wind judgment of the image quality changes of the reference area between different frames, and helps subsequent steps to find the key video frame locations where the image quality changes significantly.

[0081] Furthermore, the state of a single frame is expanded into a complete view showing how the state of the image evolves over time. This provides a core data foundation for subsequent visibility analysis of the fluctuations, degradation, or recovery of image quality in the reference area throughout the video process, ensuring that the specific video frame locations where image quality significantly improves or declines are accurately identified.

[0082] Furthermore, the multi-dimensional image status time series is condensed into a one-dimensional image quality degradation level timeline, providing core quantitative evidence for the overall pattern, key turning points and severity of image quality degradation over time in the reference area for subsequent visibility analysis, and accelerating the identification of specific video time periods in which image quality significantly and continuously deteriorates.

[0083] In summary, by comparing the feature vectors of different frames, the specific video frames in which the image quality of the reference area begins to deteriorate or begin to recover can be clearly located, the turning point of quality change can be found, and the decision-making basis can be provided for subsequent analysis steps that need to be dynamically adjusted according to the image quality. The actual impact of various factors on the reference area in the monitoring image can be objectively recorded and reflected.

[0084] In summary, it visually presents the trend of image quality changes in the reference area from the beginning to the end of the video, clearly indicating the exact time points when key changes occur in the image state, as well as the periods when the image quality remains relatively stable and the periods when it fluctuates drastically. This effectively eliminates misjudgments caused by accidental interference and more accurately reflects the overall process of image quality changes.

[0085] In summary, this study provides an objective and continuous quantitative indicator sequence to measure the actual impact of interference factors on the availability of surveillance footage. It accurately identifies the specific time intervals during which the image quality begins to decline continuously, reaches its worst state, and begins to recover. This allows subsequent analysis and decision-making to focus on video segments with truly significant image quality problems, thus improving efficiency.

[0086] In this embodiment of the invention, the single-frame feature vector includes:

[0087] The gradient magnitude of the grayscale image in the reference region is used as the quantified value of the edge sharpness of the target reference region;

[0088] LBP feature extraction is performed on the target region of the reference object to obtain the LBP encoded map of the target reference object region;

[0089] Histogram variance statistics are performed on the LBP encoded map to obtain the texture contrast quantization value of the target reference area;

[0090] Extract the channel mean of the reference area after format conversion, and quantize the channel mean into a color saturation quantization value;

[0091] The degradation rate of the current frame visual features and the baseline features in the reference area is used as the target identifiable distance estimate for the reference area.

[0092] Specifically, the intensity of the brightness change at the edge is converted into a numerical value that directly represents the sharpness of the edge in that area. That is, the larger the gradient magnitude, the more drastic the brightness change at the edge, and the sharper the edge appears visually; the smaller the gradient magnitude, the gentler the brightness change at the edge, and the more blurred the edge appears visually.

[0093] Specifically, the brightness variation patterns of each pixel neighborhood within the target area are identified and recorded. The brightness relationship patterns of each pixel neighborhood are converted into specific digital codes, ultimately forming an image composed of these coded values. This provides a basic data layer that describes the microscopic texture structure of the target area surface for subsequent analysis.

[0094] Specifically, the degree of difference in the frequency of different texture patterns appearing throughout the region is monitored, and the observed distribution breadth and fluctuation range of texture patterns are comprehensively converted into a value that reflects the overall texture contrast strength of the region.

[0095] Specifically, by focusing on the data of a specific channel in the image that represents the purity of color, the average level of the vividness component in the entire reference area is found, and this average value is directly set as the core value that reflects the overall color saturation of the area.

[0096] Specifically, the system monitors the difference between the visual performance of the reference object region in the current frame and its optimal state, converts the ratio of this difference into a spatial distance value that reflects the decline in recognition ability, and outputs a value indicating how far the recognition system can still effectively recognize the reference object under the current image quality.

[0097] Furthermore, it provides a measurable baseline value for the entire visibility analysis, which is used to directly determine whether the edge contour of the reference area is clear and usable in a single frame. It is also one of the core input data for subsequent analysis of whether the image quality and clarity have degraded and the degree of degradation, and helps to provide a basis for comparing the differences in edge clarity between different frames or between different areas of the same frame.

[0098] Furthermore, LBP features focus on relative brightness relationships, giving them good stability against uniform changes in illumination intensity and improving the reliability of reference object recognition and matching. This provides core, local pattern-based feature data for subsequent identification, matching, or differentiation of different reference objects, or for analyzing the texture changes of the same reference object under different conditions.

[0099] Furthermore, this value is one of the important bases for judging whether the surface texture of the reference object is normal, whether it has changed significantly, or whether there is interference. It provides supplementary information for subsequent analysis of image quality, especially the loss of texture information caused by blurring or noise.

[0100] Furthermore, this value is one of the important bases for judging whether the color of the reference object is normal, whether it has faded or changed significantly, or whether it has been affected by interference. It provides a key and objective quantitative indicator for the subsequent estimation of the color characteristics of the reference object area, and also provides key information for changes in lighting conditions or image quality degradation.

[0101] Furthermore, it directly reflects the effective range of the reference object recognition system under the current environmental conditions, and is the most direct basis for judging whether the entire monitoring scheme can work normally at the expected distance under the current time and current image conditions. It also provides a decision-making basis based on the critical point of recognition capability for subsequent visibility analysis on whether to initiate countermeasures.

[0102] In summary, the edge sharpness of a reference area is crucial for its accurate identification and positioning, directly reflecting the usability quality of this key information. The more blurred the edges, the greater the identification and positioning error. By comparing this value of different reference objects or areas in the same image, the part with the worst sharpness in the image can be quickly identified, providing a feature basis for subsequent visibility assessment.

[0103] In general, different reference objects have unique local texture distributions. LBP encoding maps can effectively capture and characterize these unique texture fingerprints. At the same time, severe image blurring or noise can destroy the original clear texture structure, resulting in chaotic LBP encoding map patterns or abnormal feature value distributions, which can help reflect the degree of image quality degradation.

[0104] In general, severe image blurring reduces the diversity of local textures, causing originally rich texture patterns to become homogenized, thus lowering this value. Conversely, strong noise can introduce false texture patterns, potentially causing this value to rise abnormally. Therefore, this value indirectly reflects the impact of image quality degradation on texture information; a higher value indicates a greater variety and more uneven distribution of texture patterns within the area, resulting in visually richer, more complex, and more contrasting textures; a lower value indicates a simpler and more uniform distribution of texture patterns, resulting in visually flatter, blurrier, or more homogeneous textures.

[0105] In summary, this quantitative value directly indicates the intensity of the inherent color of the reference object. For reference objects that rely on color for identification and classification, a value that is too low means that the color-based judgment may be unreliable, requiring the activation of backup strategies or the issuance of warnings. At the same time, some errors in the video acquisition or processing process may cause the overall image to appear grayish or color-distorted, which will be reflected in the saturation value of the reference object area, indicating a decline in the quality of the image's color information.

[0106] In summary, transforming the actual impact of environmental interference on target recognition in a monitoring system into an intuitive numerical value with spatial distance implications ensures that the monitoring scheme will not make incorrect judgments beyond its actual effective recognition distance. This helps to dynamically identify specific areas and improve the reliability of monitoring results.

[0107] In an example of the present invention, when the degraded value sequence is subjected to spatiotemporal feature extraction to obtain the steady-state feature vector of the reference region, the following steps are included:

[0108] The mean of the degraded value sequence is used as the mean subsequence of the degraded value sequence;

[0109] The standard deviation of the features in the degraded value sequence under the same sliding window is taken as the feature fluctuation quantum sequence of the degraded value sequence;

[0110] Arrange the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain the steady-state characteristic vector of the reference region.

[0111] Specifically, the average value of all degradation values ​​representing the image quality status over the entire time period is obtained. This single average value is treated as a subsequence with only one element, used to represent the overall level of the original sequence.

[0112] Specifically, for each window of the degraded value sequence, the degree of fluctuation of the values ​​within that window is obtained, and the fluctuation amplitude values ​​of each window are arranged in chronological order to form a recording line that records the stability changes of the original degraded value sequence within a local time period.

[0113] Specifically, the two core aspects reflecting the state of image quality are brought together into the same time-series record, ensuring that the state information of these two dimensions is aligned and correlated at each corresponding time point, generating a time-series record that covers the entire video and describes the average degradation value and fluctuation value of the image quality of the reference area at each moment.

[0114] Furthermore, the core benchmark indicator used to measure the average state of image quality provides a highly generalized value for the entire visibility analysis scheme, representing the overall average level of image quality degradation of the reference area throughout the entire surveillance video.

[0115] Furthermore, it provides core data for subsequent visibility analysis of the stability evolution of image quality degradation during the monitoring video process, helping subsequent steps to quickly locate the specific video time periods where the image quality degradation value fluctuates drastically and abnormally.

[0116] Furthermore, by recognizing state patterns, detecting state anomalies, and predicting state evolution, integrated input data is provided for the subsequent comprehensive analysis of the image quality of the reference area throughout the entire monitoring video process. This allows the analysis of image quality status to simultaneously consider the relationship and influence between its average level and fluctuation.

[0117] In summary, the average value directly reflects the overall impact of environmental interference factors on image quality throughout the monitoring period. It serves as a core comprehensive indicator for measuring the overall image quality stability and availability of a monitoring system under specific time periods and environments. A higher average value indicates more severe overall degradation and lower system availability.

[0118] In general, high values ​​indicate that the degree of image quality degradation fluctuates greatly within the window, making the state unstable and difficult to predict; low values ​​indicate that the degree of degradation changes gradually within the window, making the state stable and predictable. Significant peaks in the values ​​of the fluctuating quantum sequence usually correspond precisely to the periods when sudden, intermittent interference events occur. Identifying the periods when image quality fluctuates drastically helps to concentrate maintenance resources and countermeasures during the most unstable periods, thus improving efficiency.

[0119] In summary, by combining two-dimensional information, it is no longer a single-dimensional numerical change, but a clear depiction of the movement trajectory of the state in the horizontal-stability two-dimensional space. Accurate identification of state patterns makes it easier to infer the reasons for state changes. Based on the understanding of the complete state evolution pattern, rather than just the current value, it can more accurately predict future state trends and make more reasonable operation and maintenance decisions accordingly.

[0120] In an embodiment of the present invention, the step of arranging the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain the steady-state feature vector includes:

[0121] The mean subsequence and the feature fluctuation quantum sequence are stored in time alignment to obtain a time-series label dataset of the degradation features of the reference region;

[0122] Spatiotemporal feature extraction is performed on the time-series label dataset to obtain a preliminary feature vector of the degradation features of the reference area;

[0123] The preliminary feature vectors are sequentially fused to obtain the steady-state feature vectors of the reference region.

[0124] Specifically, it ensures that the two values ​​representing the average degradation value and the fluctuation range value of the image quality are strictly corresponding at every specific point in time, generating a complete record containing two-dimensional status information for each segment of the monitoring video timeline, which is convenient for subsequent search, analysis and use.

[0125] Specifically, from the state records arranged by time, we can find the potential patterns of how the image quality state changes over time, analyze the relationship between state records in adjacent time periods, and transform and package the identified temporal evolution patterns and spatial correlation clues into feature values ​​that condense the key characteristics of state evolution.

[0126] Specifically, multiple features reflecting different aspects of state evolution are brought together and integrated to generate a compact feature set that can comprehensively summarize the essential characteristics of the state evolution of the reference area, focusing on the long-term stable evolution pattern of the state and weakening secondary instantaneous details.

[0127] Furthermore, a two-dimensional historical record library of image quality status of reference areas is created for the entire solution. The structured, timestamped tag data provided by the application scenarios for labeling, retrieving, associating, or predicting based on historical status data is the core input data source for any complex status analysis, pattern mining, model training, or effect backtracking that involves the time dimension in the subsequent solution.

[0128] Furthermore, the original complex time-series state data is refined into a set of features that better reflect the core characteristics of state changes, reducing dimensionality and increasing efficiency for subsequent complex analysis. At the same time, it serves as the core input data for subsequent intelligent analysis of the solution, such as automatic state pattern recognition, state evolution prediction, and state anomaly detection.

[0129] Furthermore, the steady-state feature vector is the highest-level input data for the subsequent understanding, classification, comparison, and decision-making of complex states in the scheme, generating the final and most representative feature set for the entire scheme regarding the core pattern of the long-term evolution of the image quality state of the reference area.

[0130] In summary, the system automatically discovers typical patterns of image quality changes over time, correlates these changes with externally recorded events on a timeline, identifies causal relationships, and forms a long-term, reusable knowledge base of reference area states. This provides data support for understanding the impact of different environmental conditions, device states, and time periods on image quality, continuously improving the system's intelligence level and operational efficiency.

[0131] In summary, based on feature vectors, the algorithm can automatically classify video clips into "stable and good periods", "gradual deterioration periods", and "sudden interference and turbulence periods". By utilizing the historical evolution patterns reflected by feature vectors, the model can predict the possible state of image quality in the short or medium term. When the feature vector corresponding to newly generated state data deviates significantly from the historical pattern, an alarm can be automatically triggered. The extracted spatiotemporal features help to conduct more accurate temporal correlation analysis with external events and infer the root cause of state changes.

[0132] In summary, a condensed identity is generated for each reference area, which centrally reflects how its image quality status changes over a long period of time. This unified and refined feature set allows for direct comparison of the long-term stability patterns of different monitoring points, identifying problematic or excellent points, and assessing environmental changes.

[0133] In an embodiment of the present invention, when the steady-state feature vector is subjected to multiple linear regression calculation to obtain the dynamic visibility index of the target road surveillance video, the following steps are included:

[0134] The steady-state feature vector is trained with a parameter matrix to obtain the regression coefficient vector of the reference region;

[0135] The visibility fusion base value of the reference area is obtained by weighted summation of the regression coefficient vector;

[0136] The visibility fusion baseline value is activated and transformed to obtain the dynamic visibility index of the reference area.

[0137] Specifically, we determine the degree of contribution and influence of each core state characteristic in the steady-state feature vector on the actual target effect, and establish a set of mathematical rules for predicting the target effect value based on the steady-state feature vector.

[0138] Specifically, the independent impact of each state characteristic on the final visibility is combined according to their respective weight ratios to calculate a single value that no longer reflects the image quality itself but directly represents the actual usable visual perception capability under that state, providing a core quantitative basis for the final decision.

[0139] Specifically, the original numerical distribution of the fusion base value is changed to fall within a more intuitive or easier-to-use range and enhance the discriminative power near the key threshold. The fusion base value is then nonlinearly mapped to make the generated index more closely match the true pattern of system performance changes.

[0140] Furthermore, the generated regression coefficient vector provides the core calculation basis for the subsequent intelligent prediction of the entire scheme, translating the abstract multidimensional image quality state evolution mode into specific and operable effect indicators, making the subsequent state-based decision-making more scientific and data-supported.

[0141] Furthermore, it quantifies the overall visibility effectiveness of the reference area under the current or predicted state, providing an objective and comparable single metric for the visibility effectiveness of different reference areas and different time points. This is the most critical input threshold for the subsequent automatic triggering of alarms, adjustment of system parameters, allocation of maintenance resources, or activation of emergency plans.

[0142] Furthermore, the fusion base value is converted into a standard range of easily comparable index values, which facilitates inter-system integration and historical data comparison. Through non-linear transformation, the differences in numerical changes in low-visibility areas are amplified, making the system more sensitive to subtle changes in threat monitoring effectiveness.

[0143] In summary, the study clarifies the positive weights of each state characteristic in maintaining good results and the absolute values ​​of their negative weights in harming those results. This directly links abstract state patterns to specific business values.

[0144] In general, the baseline value serves as an indicator of the effectiveness of the entire monitoring solution at a specific location and time period. The higher the baseline value, the better the solution's performance at that location and time period. It can also automatically adjust the sensitivity or confidence threshold of the video analysis algorithm based on the baseline value. When the baseline value drops sharply to a dangerous level due to sudden interference, the preset emergency procedures are automatically activated.

[0145] In summary, as a standard output interface, it facilitates integration with other systems, transmits a unified visibility risk signal, and in low visibility and high risk areas, a slight decrease in the fusion baseline value will be converted into a more significant exponential decrease, triggering alarms more sensitively and seizing the time for handling. In high visibility areas, i.e. safe areas, the exponential change is relatively gradual, avoiding unnecessary frequent alarms.

[0146] In an embodiment of the present invention, when the steady-state feature vector is trained to obtain the regression coefficient vector of the reference region, the following steps are included:

[0147] The steady-state feature vector is concatenated based on the type of feature label to obtain the augmented matrix of the reference region;

[0148] Extract the covariance of the augmented matrix, and construct a multidimensional covariance matrix of the reference region based on the covariance;

[0149] The inverted covariance matrix is ​​multiplied element-wise with the covariance matrix to obtain the regression coefficient vector of the reference region.

[0150] Specifically, based on the state attributes represented by the features, each feature value in the originally flat steady-state feature vector is classified, and a feature category dimension is explicitly added to the data structure, so that each feature value not only has its numerical value, but also has a clear semantic category affiliation.

[0151] Specifically, the changing trends of any two different categories of state features in the augmented matrix are accurately measured in historical data to capture the overall fluctuation pattern and construct a mathematical matrix that centrally stores the intensity and direction of the cooperative changes between all feature categories.

[0152] Specifically, information on how features change together and how they change independently after excluding the influence of other features is mathematically fused to obtain a set of regression weight coefficients for the net contribution of each feature to the target effect, taking into account the interdependencies of all other features.

[0153] Furthermore, a complete core state feature record library is created for each reference area, categorized semantically according to state attributes. This provides the core data structure for subsequent feature analysis, comparison, retrieval, and model input based on semantic categories, enabling different subsequent analysis modules to accurately extract their target-specific category feature sets.

[0154] Furthermore, generating a key data structure that describes the inherent coordinated change patterns among various core state features of the reference area is the foundation for subsequent multi-feature joint analysis, state pattern anomaly detection, data generation simulation, and advanced probabilistic inference.

[0155] Furthermore, the generated regression coefficient vector clarifies the complex interdependencies between state features, avoiding prediction bias or model instability caused by ignoring these dependencies. It is the core mathematical basis for the entire scheme to achieve high-precision, robust, and intelligent prediction.

[0156] In summary, by introducing the feature category label dimension, the augmented matrix elevates the steady-state feature vectors describing the core patterns of the reference region's state from a one-dimensional numerical list to a feature archive with a clear semantic structure. This solves the problems of poor feature interpretability and chaotic management, providing strong basic data structure support for refined analysis, interpretable modeling, modular system design, and efficient knowledge management, and laying the foundation for solution evolution.

[0157] In summary, the multidimensional covariance matrix addresses the problem of viewing individual state features in isolation. It represents the complex network relationships of the co-evolution of various core state features in a reference region, serving as a core data structure for understanding internal coupling mechanisms, achieving accurate anomaly detection, simulating real state evolution, assessing risk propagation paths, and constructing robust high-level models.

[0158] In summary, the regression coefficient vector obtained by fusing information from the covariance matrix and its inverse matrix through element-wise multiplication is the mathematical core of dependency-aware, high-precision, and highly robust prediction. It resolves the interference caused by feature multicollinearity, reveals the unique net contribution of features to the target effect, and improves the accuracy, stability, and interpretability of the prediction model. This makes intelligent prediction and decision-making based on this coefficient more reliable in complex and ever-changing traffic monitoring environments, and is a key technological guarantee for achieving advanced intelligent operation and maintenance.

[0159] In an example of the present invention, the step of classifying the road visibility in the target road surveillance video based on the dynamic visibility index to obtain the warning level of the road visibility includes:

[0160] The real-time visibility index is filtered for invalid values ​​to obtain the effective visibility sample set in the target road surveillance video.

[0161] The effective visibility sampling set is used for early warning matching to obtain the early warning level of the target road surveillance video;

[0162] Risk assessment is performed on the warning level to obtain signal modulation of the target road surveillance video.

[0163] Specifically, the system detects and marks unreliable values ​​in the real-time visibility index stream that are significantly outside the reasonable range, exhibit drastic jumps from previous and subsequent values, or are generated under specific conditions. The identified suspicious values ​​are removed from the data stream to prevent them from entering subsequent processing stages, generating a time series record containing only verified and valid index values ​​that reflect the actual visibility status.

[0164] Specifically, based on preset rules, the current effective visibility index value is determined to belong to which predefined risk level range, generating a standardized risk level label for the current monitoring scenario, clearly indicating the safety level of the current road monitoring environment in terms of visual perception capabilities.

[0165] Specifically, the warning level is converted into an operable road traffic management instruction. Based on a preset rule base, the best signal control scheme matching the current warning level is formulated or selected, generating a set of control instructions that can directly drive downstream traffic signal control equipment.

[0166] Furthermore, to prevent invalid or abnormal data from interfering with the normal operation of downstream modules, avoid causing misjudgments, false alarms or unstable states, and ensure that the final output visibility status report, alarms and decision recommendations are all based on valid data.

[0167] Furthermore, it generates a direct action trigger signal for the entire plan, directly quantifying the visibility risk level of the current effective coverage area of ​​the road surveillance video. This is the core decision input for the plan to automatically activate different levels of emergency plans, issue early warning information, adjust traffic control measures, or notify relevant personnel to intervene.

[0168] Furthermore, the preliminary analysis and early warning results are transformed into control actions to ensure safety, completing an automated closed loop from monitoring to early warning and finally regulation, serving as the final output of the entire intelligent monitoring solution.

[0169] In summary, invalid value filtering removes invalid and interfering information from the data stream in real time by effectively sampling data that reflects the visibility status of the real world. This ensures the purity and reliability of the data upon which all subsequent analysis, decision-making, and presentation stages rely, and is one of the most fundamental and important guarantees for reliable system output, accurate decision-making, and efficient operation and maintenance.

[0170] In summary, early warning matching is a key hub that transforms the single data stream of effective visibility sampling sets into instructions that drive the efficient operation of the traffic safety assurance system. Through standardized risk level classification, complex data is transformed into clear instructions, ensuring that the system can automatically initiate defense and response measures in low visibility conditions.

[0171] In summary, the system automatically implements tiered control based on the warning level, and makes fine adjustments based on real-time traffic flow data to maximize process efficiency. At the same time, it ensures that the same warning level triggers consistent and optimal control strategies at different times and on different road sections, eliminating differences and delays caused by human judgment, ensuring the standardization and reliability of the response, and enabling the road network to maintain basic and safe operation in low visibility and adverse environments.

[0172] Compared with the prior art, the present invention has the following beneficial effects:

[0173] 1. This solution breaks through the limitations of traditional single-dimensional perception by characterizing image features into multiple indicators and constructing a four-dimensional degradation feature collaborative quantification system. By simultaneously fusing four complementary indicators, namely edge sharpness, texture contrast, color saturation, and identifiable distance, it reduces the misjudgment rate due to local feature degradation or single feature failure, avoids single-frame errors, achieves cross-modal feature cross-validation, and improves the stability of the monitoring solution.

[0174] 2. This scheme uses a covariance-driven dynamic regression architecture to analyze the nonlinear correlation between features, dynamically adjust the weights of the four types of features, fit the linear relationship between each degenerate feature, automatically remove highly linearly correlated features, avoid prediction bias caused by ignoring feature correlation, and maximize feature accuracy.

[0175] like Figure 2 The diagram shown is a functional block diagram of road low visibility monitoring based on video analysis provided in an embodiment of the present invention.

[0176] The video analysis-based low visibility monitoring system 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the video analysis-based low visibility monitoring system 100 may include a reference object extraction module 101, a degradation value sequence generation module 102, a steady-state feature vector generation module 103, a steady-state feature vector generation module 104, and a warning level generation module 105. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0177] In this embodiment, the functions of each module / unit are as follows:

[0178] The reference object extraction module 101 is used to dynamically identify the inherent reference objects of the image features in the target road monitoring video to obtain continuous reference object regions in the target road monitoring video.

[0179] The degradation value sequence generation module 102 is used to quantize the degradation features of the reference area to obtain a degradation value sequence of multiple features of the reference area.

[0180] The steady-state feature vector generation module 103 is used to extract spatiotemporal features from the degraded value sequence to obtain the steady-state feature vector of the reference region.

[0181] The dynamic visibility index module 104 is used to perform multiple linear regression calculation on the steady-state feature vector to obtain the dynamic visibility index of the target road monitoring video.

[0182] The warning level generation module 105 is used to classify the road visibility in the target road monitoring video based on the dynamic visibility index to obtain the warning level of the road visibility.

[0183] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0184] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0186] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0187] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for monitoring low visibility on roads based on video analysis, characterized in that, The method includes: S1. Dynamically identify the inherent reference objects of the image features in the target road surveillance video to obtain the continuous reference object region in the target road surveillance video; S2. Quantize the degradation features of the reference region to obtain a sequence of degradation values ​​for multiple features of the reference region, including: Single-frame degradation features are extracted from the reference region to obtain a single-frame feature vector of the reference region. The single-frame feature vector uses the gradient magnitude of the grayscale image in the reference region as a quantified value of the edge sharpness of the target reference region. LBP feature extraction is performed on the target region of the reference object to obtain the LBP encoded map of the target reference object region. Histogram variance statistics are performed on the LBP encoded map to obtain the texture contrast quantization value of the target reference region. Extract the channel mean values ​​of the reference area after format conversion, and quantize the channel mean values ​​into color saturation quantization values. The degradation rate of the current frame visual features and the baseline features in the reference area is used as the target identifiable distance estimate of the reference area; The single-frame feature vector is quantized and iterated to obtain the frame sequence feature set of the reference area; Temporal integration of the frame sequence feature set yields a sequence of degraded values ​​for multiple features in the reference region; S3. Perform spatiotemporal feature extraction on the degraded value sequence to obtain the steady-state feature vector of the reference region, including: The mean of the degraded value sequence is used as the mean subsequence of the degraded value sequence; The standard deviation of the features in the degraded value sequence under the same sliding window is taken as the feature fluctuation quantum sequence of the degraded value sequence; Arrange the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain the steady-state characteristic vector of the reference region; S4. Perform multiple linear regression calculation on the steady-state feature vector to obtain the dynamic visibility index of the target road surveillance video, including: The steady-state feature vector is trained with a parameter matrix to obtain the regression coefficient vector of the reference region; The visibility fusion base value of the reference area is obtained by weighted summation of the regression coefficient vector; The visibility fusion baseline value is activated and transformed to obtain the dynamic visibility index of the reference area; S5. Based on the dynamic visibility index, perform early warning classification on the road visibility in the target road monitoring video to obtain the early warning level of the road visibility.

2. The method for monitoring low visibility on roads based on video analysis as described in claim 1, characterized in that, The process of dynamically identifying inherent reference objects in the image features of the target road surveillance video to obtain continuous reference object regions in the target road surveillance video includes: Using a reference object as the target, the surveillance video of the target road is detected to obtain an initial set of candidate boxes for the surveillance video of the target road; Multi-frame trajectory association is performed on the initial candidate box set to obtain the motion trajectory data of the reference object; The motion trajectory data of the reference object is verified by streaming to obtain a continuous reference object area in the target road surveillance video.

3. The method for monitoring low visibility on roads based on video analysis as described in claim 1, characterized in that, When performing the chronological arrangement of the mean subsequence and the characteristic fluctuation quantum sequence to obtain the steady-state eigenvector, the process includes: The mean subsequence and the feature fluctuation quantum sequence are stored in time alignment to obtain a time-series label dataset of the degradation features of the reference region; Spatiotemporal feature extraction is performed on the time-series label dataset to obtain a preliminary feature vector of the degradation features of the reference area; The preliminary feature vectors are sequentially fused to obtain the steady-state feature vectors of the reference region.

4. The method for monitoring low visibility on roads based on video analysis as described in claim 1, characterized in that, When training the parameter matrix of the steady-state feature vector to obtain the regression coefficient vector of the reference region, the process includes: The steady-state feature vector is concatenated based on the type of feature label to obtain the augmented matrix of the reference region; Extract the covariance of the augmented matrix, and construct a multidimensional covariance matrix of the reference region based on the covariance; The inverted covariance matrix is ​​multiplied element-wise with the covariance matrix to obtain the regression coefficient vector of the reference region.

5. The method for monitoring low visibility on roads based on video analysis as described in claim 1, characterized in that, When determining the warning level for road visibility in the target road surveillance video based on the dynamic visibility index, the following are included: The real-time visibility index is filtered for invalid values ​​to obtain the effective visibility sample set in the target road surveillance video. The effective visibility sampling set is used for early warning matching to obtain the early warning level of the target road surveillance video; Risk assessment is performed on the warning level to obtain signal modulation of the target road surveillance video.

6. A road low visibility monitoring system based on video analysis, used to implement the road low visibility monitoring method based on video analysis as described in any one of claims 1-5, characterized in that, The system includes the following modules: The reference object extraction module dynamically identifies inherent reference objects in the image features of the target road surveillance video to obtain continuous reference object regions in the target road surveillance video. The degradation value sequence generation module performs degradation feature quantization on the reference region to obtain a degradation value sequence of multiple features of the reference region, including: Single-frame degradation features are extracted from the reference region to obtain a single-frame feature vector of the reference region. The single-frame feature vector uses the gradient magnitude of the grayscale image in the reference region as a quantified value of the edge sharpness of the target reference region. LBP feature extraction is performed on the target region of the reference object to obtain the LBP encoded map of the target reference object region. Histogram variance statistics are performed on the LBP encoded map to obtain the texture contrast quantization value of the target reference region. Extract the channel mean values ​​of the reference area after format conversion, and quantize the channel mean values ​​into color saturation quantization values. The degradation rate of the current frame visual features and the baseline features in the reference area is used as the target identifiable distance estimate of the reference area; The single-frame feature vector is quantized and iterated to obtain the frame sequence feature set of the reference area; Temporal integration of the frame sequence feature set yields a sequence of degraded values ​​for multiple features in the reference region; The steady-state feature vector generation module performs spatiotemporal feature extraction on the degradation value sequence to obtain the steady-state feature vector of the reference region, including: The mean of the degraded value sequence is used as the mean subsequence of the degraded value sequence; The standard deviation of the features in the degraded value sequence under the same sliding window is taken as the feature fluctuation quantum sequence of the degraded value sequence; Arrange the mean subsequence and the characteristic fluctuation quantum sequence in chronological order to obtain the steady-state characteristic vector of the reference region; The steady-state feature vector generation module performs multiple linear regression calculations on the steady-state feature vectors to obtain the dynamic visibility index of the target road surveillance video, including: The steady-state feature vector is trained with a parameter matrix to obtain the regression coefficient vector of the reference region; The visibility fusion base value of the reference area is obtained by weighted summation of the regression coefficient vector; The visibility fusion baseline value is activated and transformed to obtain the dynamic visibility index of the reference area; The warning level generation module classifies the road visibility in the target road monitoring video based on the dynamic visibility index to obtain the warning level of the road visibility.

Images