Expressway abnormal event visual detection effect evaluation method and system

Abstract

This invention belongs to the field of transportation-related technology. To address the problems of incomplete evaluation and insufficient accuracy of existing detection algorithms, it proposes a method and system for evaluating the visual detection performance of highway anomaly events. By analyzing the characteristics of various highway anomaly events and clarifying quantitative judgment rules, a standardized highway anomaly event scene atlas is constructed. An orthogonal experimental design is used to control variables to form a basic real dataset. After expanding the basic real dataset, the authenticity and dynamic logical rationality of generated events are screened to construct a simulated highway anomaly event dataset. The highway video detection model is tested using the simulated highway anomaly event dataset, and a comprehensive test result is obtained by combining multi-dimensional evaluation indicators. This invention can comprehensively evaluate the performance of video detection systems and enables developers to make targeted improvements based on the evaluation results, thereby improving recognition efficiency.

Description

Technical Field

[0001] This invention belongs to the field of transportation-related technology, and in particular relates to a method and system for evaluating the effectiveness of visual detection of abnormal events on highways. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the continuous increase in the mileage of expressways and the expansion of the road network, the frequency of abnormal events on expressways has risen significantly. These abnormal events specifically include illegal parking in driving lanes, illegal parking in emergency lanes, littering, traffic congestion, and pedestrians mistakenly entering driving areas. These events not only disrupt the normal traffic flow on expressways but also easily lead to traffic accidents, posing a serious threat to road efficiency and traffic safety.

[0004] To promptly detect and handle such anomalies, video monitoring has become one of the core technological means in highway operation and management. Traditional manual video monitoring relies on staff to observe and judge massive amounts of surveillance footage in real time, which has two major problems: First, the monitoring workload is extremely large, and staff fatigue can easily lead to missed or false detections; second, the identification standards are greatly affected by subjective experience, and the judgment results of different monitoring personnel vary, making it difficult to guarantee the consistency and accuracy of monitoring results, and thus failing to meet the needs of high-efficiency monitoring of highways around the clock and over a wide area.

[0005] To address the limitations of manual monitoring, video analytics-based artificial intelligence (AI) detection technology, with its advantages of automation and real-time performance, has been widely applied in the field of highway anomaly detection. However, this technology still has significant shortcomings, mainly in the following aspects: 1. The algorithm market is highly diversified, with AI detection algorithms developed by different manufacturers showing significant differences in core performance indicators such as detection accuracy, response speed, and false detection rate, lacking a unified performance benchmark; 2. The actual application scenarios on highways are complex and varied, such as low-visibility environments at night, uneven lighting in tunnels, and detection scenarios involving small-volume spills (such as gravel and debris), which place extremely high demands on the robustness of AI algorithms, while existing technologies have not established specific evaluation standards for different scenarios; 3. The industry lacks a unified and systematic algorithm evaluation mechanism, making it impossible to scientifically and objectively assess the adaptability and effectiveness of various AI detection algorithms in different application scenarios, resulting in a lack of reliable basis for algorithm selection.

[0006] Currently, most highway video AI detection systems still heavily rely on the manual experience of technicians during algorithm selection and parameter tuning, lacking the ability to conduct automated evaluations across multiple scenarios and compare multiple algorithms. This directly leads to problems such as unstable detection results (e.g., a sharp drop in detection accuracy in specific scenarios) and low response efficiency to abnormal events after actual deployment of AI detection algorithms, making it difficult to fully leverage the supporting role of AI technology in intelligent highway management.

[0007] In summary, the current field of AI detection technology for highway video urgently needs to establish an AI detection effect evaluation system that covers multiple types of abnormal events, integrates multi-dimensional evaluation indicators, and adapts to complex application scenarios. This system will enable objective quantitative evaluation of AI detection algorithms, provide technical support for algorithm selection, optimization, and upgrading of highway intelligent monitoring systems, and ultimately improve the efficiency of handling abnormal events and the level of road operation safety. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, this invention provides a method and system for evaluating the visual detection effect of highway abnormal events, which comprehensively evaluates the performance of video detection models and enables developers to make targeted improvements based on the evaluation results, thereby improving recognition efficiency.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a method for evaluating the visual detection effect of abnormal events on highways, including:

[0011] For common abnormal events on highways, analyze the characteristics of various abnormal events and clarify the quantitative judgment rules, and construct a standardized atlas of abnormal event scenarios on highways;

[0012] Based on a standardized atlas of highway abnormal event scenarios, key influencing variables of highway abnormal events are analyzed. Orthogonal experimental design is used to control variables to form a basic real dataset. After expanding the basic real dataset, the authenticity and dynamic logic of the generated events are screened to construct a simulated dataset of highway abnormal events.

[0013] The highway video detection model was tested using the aforementioned highway abnormal event simulation dataset, and the comprehensive test results were obtained by combining multi-dimensional evaluation indicators.

[0014] Secondly, the present invention provides a visual detection effect evaluation system for highway abnormal events, comprising:

[0015] The analysis module is configured to: analyze the characteristics of various highway anomalies and clarify the quantitative judgment rules for common highway anomalies, and construct a standardized highway anomaly scenario atlas;

[0016] The construction module is configured as follows: based on a standardized highway abnormal event scenario atlas, key influencing variables of highway abnormal events are analyzed, orthogonal experimental design is used to control variables to form a basic real dataset, and after expanding the basic real dataset, the authenticity and dynamic logic of the generated events are screened to construct a highway abnormal event simulation dataset.

[0017] The evaluation module is configured to: test the highway video detection model using the highway abnormal event simulation dataset, and obtain a comprehensive test result by combining multi-dimensional evaluation indicators.

[0018] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0019] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0020] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0021] The above one or more technical solutions have the following beneficial effects:

[0022] In this invention, by analyzing the characteristics of various highway anomaly events and clarifying quantitative judgment rules, a standardized highway anomaly event scenario atlas is constructed. An orthogonal experimental design is used to control variables to form a basic real dataset. After expanding the basic real dataset, the authenticity and dynamic logical rationality of the generated events are screened to construct a highway anomaly event simulation dataset. The highway anomaly event simulation dataset is then used to test the highway video detection model, and a comprehensive test result is obtained by combining multi-dimensional evaluation indicators. This invention can comprehensively evaluate the performance of the video detection model and enables developers to make targeted improvements based on the evaluation results, thereby improving recognition efficiency.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 This is a flowchart of the method and system for evaluating the visual detection effect of abnormal events on highways provided in this embodiment of the invention.

[0026] Figure 2 This is a flowchart of the abnormal event scenario set construction process provided in the embodiments of the present invention;

[0027] Figure 3 This is a flowchart of the orthogonal experiment dataset construction process provided in this embodiment of the invention;

[0028] Figure 4 This is a schematic diagram of the detection effect indicators provided in the embodiments of the present invention;

[0029] Figure 5 This is a test flowchart of the video dataset-based detection system provided in an embodiment of the present invention;

[0030] Figure 6 This is a test flowchart of a real-time video stream detection system provided in an embodiment of the present invention;

[0031] Figure 7 This is a close-up photograph of a plastic bag taken during daytime in an orthogonal experiment for testing spilled materials, provided in an embodiment of the present invention.

[0032] Figure 8 These are real-life photos taken during the daytime, at a medium distance, of a plastic bag during an orthogonal experiment for testing projectiles provided in this embodiment of the invention.

[0033] Figure 9 These are real-time photos taken during the daytime, from a plastic bag, at a distance, in an orthogonal experiment for testing spilled materials, as provided in this embodiment of the invention.

[0034] Figure 10 These are close-up photos of a cardboard box taken during daytime in an orthogonal experiment for testing projectiles, as provided in this embodiment of the invention.

[0035] Figure 11 These are real-life photos taken during the daytime, at a medium distance, of a cardboard box during an orthogonal experiment for testing projectiles, as provided in this embodiment of the invention.

[0036] Figure 12 These are real-time photos taken at a distance during the daytime, from a cardboard box, in the orthogonal experiment for testing projectiles provided in this embodiment of the invention.

[0037] Figure 13 These are close-up photos of a cardboard box taken at night during an orthogonal experiment for testing projectiles, as provided in this embodiment of the invention.

[0038] Figure 14 This is a real-life photograph taken at night from a cardboard box at a medium distance during an orthogonal experiment for testing projectiles provided in an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0040] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0041] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0042] Example 1

[0043] This embodiment discloses a method for evaluating the visual detection effect of abnormal events on highways, including:

[0044] For common abnormal events on highways, analyze the characteristics of various abnormal events and clarify the quantitative judgment rules, and construct a standardized atlas of abnormal event scenarios on highways;

[0045] Based on a standardized atlas of highway abnormal event scenarios, key influencing variables of highway abnormal events are analyzed. Orthogonal experimental design is used to control variables to form a basic real dataset. After expanding the basic real dataset, the authenticity and dynamic logic of the generated events are screened to construct a simulated dataset of highway abnormal events.

[0046] The highway video detection model was tested using a simulated dataset of abnormal events on highways, and the comprehensive test results were obtained by combining multi-dimensional evaluation indicators.

[0047] The following is combined with Figure 1 This embodiment provides a detailed description of the visual detection performance evaluation method for highway anomaly events.

[0048] Step 1: Analyze the characteristics of various types of abnormal events on highways, clarify the quantitative judgment rules, and construct a standardized atlas of abnormal event scenarios on highways.

[0049] Specifically, such as Figure 2 As shown, abnormal events occurring in the highway network, such as abnormal parking (including emergency parking and regular parking), pedestrian intrusion, littering, and congestion, are collected, and different occurrence characteristics are analyzed to establish an abnormal event scenario set.

[0050] The main characteristics of an emergency stopping event are as follows: 1) A stationary non-engineering vehicle appears in the driving lane; 2) Although the non-engineering vehicle is stationary in the emergency lane, its body portion, as seen from the monitoring perspective, encroaches on the driving lane; 3) The non-engineering vehicle is stationary in key road sections such as ramps, merging / diverting zones, and tunnels. Meeting any of the above characteristics is sufficient to define an emergency stopping event.

[0051] The main characteristics of a general parking incident are: 1) a stationary non-engineering vehicle is present in the emergency lane; 2) the parking time exceeds a threshold (e.g., the threshold is 10 minutes, which can be adjusted according to actual business conditions); 3) the orthographic projection of the vehicle body does not encroach on the driving lane from the monitoring perspective; and 4) the parking incident does not occur in key road sections such as ramps, merging / diverting zones, and tunnels. Meeting all of the above characteristics defines a general parking incident.

[0052] The main characteristics of a pedestrian intrusion incident are: 1) the pedestrian appears at any location on the highway; 2) the pedestrian is not wearing a reflective vest or helmet; 3) no construction vehicles or abnormal parking events are present within the monitoring range. Meeting all of the above characteristics defines it as a general parking incident.

[0053] The main characteristics of a spill incident are: 1) the presence of objects such as cardboard boxes, tire covers, and tarpaulins on the highway surface that may affect driving safety; 2) the objects may be stationary or at a low speed due to wind or vehicle movement (a value of 25 km / h is recommended, but can be adjusted according to actual operational conditions). An incident can be defined as a spill incident if all of the above characteristics are met.

[0054] The main characteristics of a congestion event are: 1) vehicle speed is below a threshold (which can be half the design speed and adjusted according to actual business conditions); 2) the occupancy rate of one or more lanes is above a threshold (which can be 80% and adjusted according to actual business conditions). A congestion event is defined as one that meets all of the above characteristics.

[0055] By constructing the above set of abnormal event scenarios, we propose the characteristics of different scenarios in surveillance videos, which will facilitate the subsequent construction of an abnormal event dataset based on the scenario set.

[0056] Step 2: Based on the standardized highway abnormal event scenario atlas, analyze the key influencing variables of highway abnormal events, use orthogonal experimental design to control variables to form a basic real dataset, expand the basic real dataset, and screen the authenticity and dynamic logic of the generated events to construct a highway abnormal event simulation dataset.

[0057] like Figure 3As shown, the control variables for abnormal parking events include the lane where the parking event occurred (e.g., driving lane, emergency lane, ramp, etc.), the distance of the parking location from the monitoring point, vehicle type, and lighting conditions; the control variables for pedestrian intrusion events include the lane the pedestrian entered (e.g., driving lane, emergency lane, ramp, etc.), the distance of the pedestrian's entry location from the monitoring point, the number of pedestrians, whether the pedestrian moved, whether they were traffic management personnel (who usually wear reflective vests), and lighting conditions; the control variables for spilled material events include the type of spilled material (common spilled materials on highways, such as tire linings, cardboard boxes, etc.), the size of the spilled material, the lane where the spilled material occurred (e.g., driving lane, emergency lane, ramp, etc.), the distance of the spilled material's location from the monitoring point, and lighting conditions; and the control variables for congestion events include the time of congestion, the road segment where it occurred, the distance of the congestion from the monitoring point, and lighting conditions. Orthogonal experiments are designed based on the control variables to cover the combinations of control variables, and high-fidelity test data is collected according to the orthogonal experimental design to form a basic real dataset.

[0058] By designing and executing orthogonal experiments using the control variables involved in each of the above-defined anomalous events, and controlling different variables, multiple highly representative test scenarios were planned. This design ensures that the widest range of variable combinations is covered with the fewest number of experiments, providing a clear approach for subsequent physical data collection and artificial intelligence generation. Examples of orthogonal experimental designs for spill events are shown in Table 1.

[0059] Table 1:

[0060]

[0061] The basic real-world dataset includes data on common highway emergencies collected in step one, manually tested data collected through orthogonal experiments, and data generated by generative artificial intelligence techniques.

[0062] By leveraging generative artificial intelligence technology, large-scale, multi-dimensional scene enhancement and synthesis are carried out on a real road surveillance video test set and artificial test data constructed in a closed test site, achieving a significant improvement in data scale and scene coverage.

[0063] In terms of data augmentation, environmental conditions are transformed into existing video and image data through conditions generation methods such as diffusion models and generative adversarial networks. For example, abnormal event scenes during the day are moved to different lighting and weather conditions such as nighttime, rainy days, and snowy days, or vehicle attributes such as color and model are adjusted to simulate complex situations that are easy to cause recognition difficulties, such as spilled objects being in the shadow of the road or accident vehicles being similar in color to the road surface.

[0064] In terms of edge case generation, the focus is on constructing two types of data that are difficult to collect in the real world: one is "impossible scenarios," such as extreme disasters like partial bridge collapses or landslides burying roads; the other is adversarial scenarios, which specifically generate challenging samples that are prone to misjudgment or failure, addressing the weaknesses of current recognition models, in order to systematically improve the robustness of the algorithm. Ultimately, a large-scale generated dataset covering a vast array of diverse samples and a wide range of edge cases is formed, providing crucial data support for the comprehensive evaluation and reinforcement training of the abnormal event recognition model. Corresponding experimental images include... Figures 7-14 As shown.

[0065] In this embodiment, the expanded base real dataset is first screened for authenticity using a binary network based on a convolutional neural network (CNN).

[0066] Specifically, for the expanded base real dataset, image preprocessing is first performed, including normalization, size standardization, and data augmentation. Then, a binary classification network based on a convolutional neural network (CNN) is used for authenticity screening. This binary classification network takes the preprocessed images as input and converts them into a four-dimensional tensor format with dimensions (...). batch_size, channels, height, width ), batch_size, channels, height, width These represent the batch size, number of channels, height, and width, respectively. Local features (such as edges and textures) are extracted through convolutional layers, and feature dimensionality is reduced through pooling layers. Finally, the probability value (in the range [0,1]) of the output image being a real sample is determined by the Sigmoid activation function. The authenticity is judged with a threshold of 0.5. If the probability value is ≥0.5, it is considered real; otherwise, it is considered fake.

[0067] The training of convolutional neural networks uses a binary cross-entropy loss function and the Adam optimizer, and iteratively optimizes the model parameters by calculating the predicted values ​​through forward propagation and updating the weights through back propagation.

[0068] The dataset that has passed the authenticity screening is then fed into the spatiotemporal video understanding model for dynamic logical rationality screening. The spatiotemporal video understanding model uses an improved Transformer, which introduces a spatiotemporal graph attention network into the encoder to enhance the ability to model the dynamic logical relationships of the data.

[0069] In this embodiment, the improved Transformer is used to analyze the context of the video sequence, modeling and evaluating the dynamic development of events, temporal causal relationships, and the behavioral intentions of traffic participants (vehicles, etc.) (such as the rationality of trajectories), outputting a comprehensive, multi-dimensional evaluation vector that quantifies the rationality of the entire video segment in terms of dynamic logic. A causal reasoning network based on the Transformer (Spatiotemporal Transformer) architecture is used, with its encoder layer deeply customized.

[0070] Specifically, this embodiment does not use the standard 12-layer Transformer encoder, but instead inserts a Dynamic Spatio-Temporal Graph Attention Module (ST-GAT) serially after its 6th layer encoder block.

[0071] The operating mechanism of the dynamic spatiotemporal graph attention module is as follows:

[0072] Dynamic graph construction: First, the object-level features output by the 6th layer encoder, which already contain preliminary motion and appearance information (e.g., feature vectors of each traffic participant extracted through the RoI Align mechanism), are dynamically constructed into an interaction graph. In this interaction graph, nodes represent traffic participants (vehicles, etc.), and edges represent their potential interaction relationships (e.g., spatial proximity or correlation in motion trends).

[0073] Causal relationship modeling: Subsequently, the dynamic spatiotemporal graph attention module explicitly models the temporal causal relationships and behavioral intentions among participants using graph attention mechanisms. For example, it can learn the high attention weights and temporal associations between 'the brake lights of the car in front turn on' (cause) and 'this car begins to decelerate' (effect).

[0074] Feature fusion and back-injection: The output of the dynamic spatiotemporal graph attention module, which is a context vector rich in interaction logic, will be aggregated with the original global scene features from layer 6 through a newly designed cross-attention fusion layer. Finally, this fused enhanced feature will be input into the subsequent encoder layers 7 to 12 for further deep feature extraction.

[0075] The cross-attention fusion layer in this embodiment is designed to achieve adaptive and refined fusion of interactive logic features and global scene features. Its core is an interaction framework based on a scaled dot product attention mechanism, constructing a query (Q), key (K), and value (V) structure. The query (Q) originates from the context vector, a node feature rich in interactive logic, output by the dynamic spatiotemporal graph attention module. The key (K) and value (V) originate from the global scene features output by the original Transformer encoder in layer 6. Attention weights are obtained by calculating the similarity between Q and K, and these weights are then used to sum V, generating a new feature vector enhanced with scene context for each interactive logic. To improve the accuracy and efficiency of the fusion, this layer optionally introduces a spatial modulation mechanism. This involves generating a Gaussian weight mask based on the spatial location of traffic participants in the interaction graph and multiplying it with the original attention graph. This guides the model to prioritize local scene regions related to the current interactive logic, effectively suppressing interference from irrelevant background information. In the feature integration stage, a gated fusion mechanism is adopted. The enhanced interactive logic features and the original global features of the 6th layer are weighted and summed through a gate weight generated by the Sigmoid function. This allows the network to adaptively control the fusion ratio of new and old information. The final output is an enhanced feature that contains both macro scene information and deeply embedded micro interactive logic. This feature is then injected back into the subsequent encoders of the 7th to 12th layers for further processing. This forces the model to understand the video sequence from the level of "logical rationality" rather than just the pixel level, ensuring the accuracy and robustness of dynamic logic filtering.

[0076] By adjusting the architecture to "Backbone-Bypass-Fusion," the video understanding model in this embodiment is able to force the network to understand the interaction logic of "who did what to whom" at the mid-level semantic level, rather than simply learning surface pixel changes. This enables it to perform high-precision quantitative evaluation of the rationality of trajectories, the dynamic development of events, and the logic of temporal causal chains.

[0077] The above screening ensures that the generated abnormal events not only exist visually, but their occurrence process and dynamic interaction must also conform to the logical closed loop of the real world, thereby efficiently eliminating invalid samples that 'seem real but are logically flawed'.

[0078] Step 3: Test the highway video detection model using a simulated highway anomaly dataset, and obtain comprehensive test results by combining multi-dimensional evaluation metrics.

[0079] In this embodiment, as Figure 4As shown, the constructed detection performance evaluation index includes overall detection performance, nighttime detection capability, lane detection capability, event fusion capability, and resource consumption. Finally, a multi-dimensional radar map area penalty model is constructed to comprehensively evaluate the highway video detection model, forming a multi-dimensional algorithm evaluation index system.

[0080] For the constructed detection performance evaluation system, the overall detection performance of the algorithm under test is evaluated, using detection rate, accuracy, and repetition rate as evaluation indicators. The nighttime monitoring capability of the algorithm under test is evaluated across all event scenarios, using the average detection rate of the algorithm under test in nighttime environments as the nighttime detection capability evaluation indicator. The lane detection capability of the algorithm under test is evaluated in abnormal parking scenarios, using the accuracy rate of the algorithm under test in this scenario as the lane detection capability evaluation indicator. The event fusion capability of the algorithm under test is evaluated in abnormal parking and congestion scenarios, using the repetition rate of the algorithm under test in this scenario as the event fusion capability evaluation indicator. The resource consumption of the algorithm under test is evaluated across all events, using the average CPU, memory, video memory, and graphics card computing power utilization rate during the testing period as the resource consumption evaluation indicator. A comprehensive evaluation of the algorithm under test across the above dimensions is conducted by constructing a multi-dimensional radar chart area penalty model. The radar chart area represents the comprehensive score, and expert scoring is used to weight the dimensions. A penalty factor is designed to penalize the scores of algorithms under test with obvious defects, ultimately obtaining the comprehensive evaluation result of the model under test.

[0081] Specifically, accuracy refers to the percentage of the total number of events correctly detected by the detection and evaluation system, including the sum of correctly detected events and duplicate reported events, out of the total number of detected events. The total number of detected events consists of the correct number, the incorrect number, and the duplicate reported number. Detection rate refers to the percentage of correctly detected events out of the sum of unreported and correct events. Duplication rate refers to the percentage of duplicate reported events out of the sum of correct and duplicate reported events. The calculation formulas are as follows:

[0082]

[0083]

[0084]

[0085] For the extreme detection distance metric, the result is the extreme distance at which the algorithm under test can effectively detect events under good lighting conditions. For the nighttime detection performance metric, the result is the average detection rate of the algorithm under test in nighttime environments. For the lane detection metric, the result is the accuracy of the algorithm under test in judging emergency parking and normal parking. For the fusion reporting metric, the result is the repetition rate of the algorithm under test for events that need to be fused and reported. For system resource usage, the result is the average CPU, memory, GPU computing power, and VRAM usage during the test period. Note that this metric is only meaningful when all algorithms under test are used on computers with the same configuration; otherwise, this metric is discarded. A multi-dimensional radar chart area penalty model is constructed to achieve multi-dimensional comprehensive metric evaluation and calculate the algorithm's comprehensive score.

[0086] For the overall detection effect, the accuracy rate, detection rate and repeatability rate are directly taken as the scores of the first three indicators, D1, D2 and D3.

[0087] The index score for the limit detection distance The calculation method is as follows:

[0088]

[0089] in This represents the farthest distance that the model under test can detect under good lighting conditions. For the desired detection distance, it is recommended to use the farthest distance that can be distinguished by the naked eye, and .

[0090] The score of the nighttime detection effect index The calculation method is as follows:

[0091]

[0092] in, The detection rate in nighttime environments, This represents the total number of events occurring at night.

[0093] For lane detection capability index score The calculation method is as follows:

[0094]

[0095] in, To correctly detect the number of lanes with lane occupancy status in parking incidents. This represents the total number of parking incidents.

[0096] Scores for integrated reporting indicators The calculation method is as follows:

[0097]

[0098] in, The total repetition rate of events that need to be merged and processed. This represents the total number of parking incidents.

[0099] For system resource usage, the average CPU, memory, GPU computing power, and VRAM usage during the test period were taken. , , , .

[0100] Calculate the weighted values ​​of each indicator. .

[0101]

[0102] in, As an indicator The weights are determined by a scoring process involving transportation management personnel and industry experts; higher scores indicate greater importance, and the following conditions must be met. M represents the total number of indicators, which is 11 in this embodiment; The score is for the i-th indicator.

[0103] Calculate the actual weighted radar image area using the Gaussian area formula. .

[0104]

[0105]

[0106]

[0107] in, hour, Take 1.

[0108] Calculate the area of ​​an ideal radar image, that is, the area of ​​each dimension under ideal conditions. All scores are 100%, substituting into the above results in Calculated value, recorded as .

[0109] To assess the shortcomings of the comprehensive indicators, a penalty factor is designed to penalize the algorithm under test that has obvious shortcomings, so as to ensure the balance of all indicators.

[0110] 1) Calculate the weakness focus index for each dimension. :

[0111]

[0112] Among the settings The goal is to make the weaknesses more obvious with higher scores, and this needs to be combined with weighting. To be confirmed.

[0113] 2) Identify the most critical weakness :

[0114]

[0115] 3) Set penalty factors:

[0116]

[0117] Where k is the penalty intensity index, The larger the k value, the smaller the P value, and the more severe the penalty for key weaknesses. It can be adjusted as needed.

[0118] The numerical values ​​of the multidimensional radar chart area penalty model are calculated to obtain a comprehensive evaluation result.

[0119]

[0120] In this embodiment, the test schemes are divided into two categories according to the type of test video: one is a test scheme based on video datasets, and the other is a test scheme based on real-time video streams.

[0121] like Figure 5 As shown, the test scheme based on the video dataset is as follows: by constructing a video dataset and a database for storing detection results, we can ensure that there is enough dataset for testing, while ensuring that the test results can be stored in real time, and designing tables to record the test results;

[0122] At the start of the test, a dataset is provided to each algorithm under test to ensure that the test starts synchronously. The test results are written in real time, and the results are evaluated and analyzed according to the evaluation index requirements to output a test result report.

[0123] Real-time video-based testing solution: In view of the uncertainties in real-time video streams, firstly, the detection standards are clearly defined, and a unified standard for judging correct and repeated events is specified. At the same time, a database for storing detection results is established.

[0124] After the test begins, each algorithm under test is connected to the video stream in real time, and the test results are output to the database in real time. At the same time, the correctness of each algorithm is judged manually. Finally, based on the judgment results and the evaluation index requirements, key indicators are output and a test result report is generated.

[0125] like Figure 6 As shown, when the test data is real-time traffic flow data, using real-time traffic flow as the test data can better evaluate the system's dynamic handling capability for emergencies and its ability to identify unknown events. The main contents include:

[0126] Since real-time video data is used as test data, in order to avoid the detection unit's insufficient event classification and detection capabilities, which would prevent it from being unable to distinguish between driving lane parking events and emergency lane parking events, the types of events that need to be detected are clearly defined according to the different emergency response plans of the highway management department for various events, and a unified standard for judging "correct" and "repeated" events is stipulated.

[0127] A storage database for storing test results will be established simultaneously to provide a foundation for subsequent manual verification and indicator calculation;

[0128] All tests begin simultaneously. Operations personnel mark the reported events as "correct," "incorrect," or "duplicate," and the marking results are written into the corresponding data table specified above in real time to facilitate the real-time transmission, storage, and later verification of test results.

[0129] After the test is completed, evaluation indicators are automatically generated based on the test results of the algorithm under test, and the final test result report is generated based on the generated evaluation results.

[0130] Example 2

[0131] The purpose of this embodiment is to provide a visual detection performance evaluation system for highway abnormal events, including:

[0132] The analysis module is configured to: analyze the characteristics of various highway anomalies and clarify the quantitative judgment rules for common highway anomalies, and construct a standardized highway anomaly scenario atlas;

[0133] The construction module is configured as follows: based on a standardized highway abnormal event scenario atlas, key influencing variables of highway abnormal events are analyzed, orthogonal experimental design is used to control variables to form a basic real dataset, and after expanding the basic real dataset, the authenticity and dynamic logic of the generated events are screened to construct a highway abnormal event simulation dataset.

[0134] The evaluation module is configured to: test the highway video detection model using the highway abnormal event simulation dataset, and obtain a comprehensive test result by combining multi-dimensional evaluation indicators.

[0135] In further embodiments, the following is also provided:

[0136] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0137] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0138] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0139] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0140] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0141] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0142] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0143] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0144] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0145] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for evaluating the effect of visual detection of abnormal events on a highway, characterized in that, The method comprises the following steps: analyzing the characteristics of various types of highway abnormal events and determining the quantitative judgment rules to construct a standardized highway abnormal event scenario atlas; the common abnormal events on the highway are divided into four types of abnormal parking events, pedestrian intrusion events, littering events and congestion events; based on the standardized highway abnormal event scenario atlas, the key influence variables of the highway abnormal event are analyzed, the orthogonal test design is used to control the variables to form a basic real data set, the realness and dynamic logic rationality of the generated events are screened after the basic real data set is expanded, and a highway abnormal event simulation data set is constructed; wherein, for the expanded basic real data set, firstly, a binary network based on a convolutional neural network is used to screen the realness of the expanded basic real data set; an improved Transformer is used to screen the dynamic logic rationality of the data set that passes the realness screening; the improved Transformer introduces a dynamic spatio-temporal graph attention module in the encoder to enhance the modeling capability of the dynamic logic relationship of the data; the improved Transformer is specifically: the object-level features output by the original 6th layer encoder of the Transformer are dynamically constructed into an interaction graph; wherein, in the interaction graph, the nodes represent the traffic participants, and the edges represent the potential interaction relationship between the traffic participants; the dynamic spatio-temporal graph attention module uses the graph attention mechanism to explicitly model the temporal causal relationship and behavior intention between participants; a cross-attention fusion layer is introduced to aggregate and enhance the output of the dynamic spatio-temporal graph attention module and the output of the original 6th layer encoder of the Transformer, and the original 7th layer to 12th layer encoder of the Transformer is used to process the aggregated and enhanced features to obtain a multi-dimensional evaluation vector; the highway video detection model is tested by using the highway abnormal event simulation data set, and a comprehensive test result is obtained by combining multi-dimensional evaluation indexes; wherein, the multi-dimensional evaluation indexes include overall detection effect, night detection ability, lane detection ability, event fusion ability and resource occupation situation, and finally a multi-dimensional radar chart area penalty model is constructed to comprehensively evaluate the highway video detection model, forming a multi-dimensional algorithm evaluation index system; For the constructed detection effect evaluation system, the overall detection effect of the to-be-tested model is evaluated, and the detection rate, accuracy rate and repetition rate are used as evaluation indexes; the night monitoring capability of the to-be-tested model in all event scenarios is evaluated, and the average detection rate of the to-be-tested model in the night environment is used as the night detection capability evaluation index; the lane detection capability of the to-be-tested model in the abnormal parking scenario is evaluated, and the accuracy rate of the lane detection of the to-be-tested model in the scenario is used as the lane detection capability evaluation index; the event fusion capability of the to-be-tested model in the abnormal parking and congestion scenario is evaluated, and the repetition rate of the to-be-tested model in the scenario is used as the event fusion capability evaluation index; the resource occupation of the to-be-tested model in all events is evaluated, and the average CPU, memory, display memory and GPU average occupation rate during the test period are used as the resource occupation evaluation index; the to-be-tested model is comprehensively evaluated in the above dimensions, a multi-dimensional radar chart area penalty model is constructed, the comprehensive score is represented by the area of the radar chart, the dimensions are weighted by using the expert scoring method, and a penalty factor is designed to punish the score of the to-be-tested model with obvious defects, and finally the comprehensive evaluation result of the to-be-tested model is obtained; the penalty factor is designed to punish the score of the to-be-tested model with obvious defects, including: for each to-be-tested model, calculating the short board attention index of each dimension, finding the most critical short board according to the short board attention index of each dimension, and setting a penalty factor based on the most critical short board.

2. The method of claim 1, wherein the method further comprises: For common abnormal events on the expressway, the characteristics of various types of abnormal events on the expressway are analyzed and the quantitative determination rules are clarified, and a standardized expressway abnormal event scenario atlas is constructed, specifically: The abnormal parking is divided into emergency parking events and general parking events, and the non-engineering vehicle is determined as an emergency parking event if the non-engineering vehicle is stationary on the driving lane, the non-engineering vehicle is stationary on the emergency lane but the body orthographic projection invades the driving lane, and the non-engineering vehicle is stationary on any one of the key road sections; The pedestrian intrusion event is determined if the pedestrian appears at any position on the expressway, does not wear a reflective vest or safety helmet, and there is no engineering vehicle and abnormal parking event in the monitoring range; The object on the road surface is determined as a throwing object event if the object on the road surface is in a stationary or low-speed state; The congestion event is determined if the vehicle speed is lower than the set speed threshold and one or more lane occupancy rates are higher than the set percentage threshold.

3. The method of claim 2, wherein the method further comprises: The control variables of the abnormal parking event include lane, distance from the monitoring point, vehicle type and lighting conditions; the control variables of the pedestrian intrusion event include lane, distance from the monitoring point, number of people, whether moving, whether for traffic management personnel and lighting conditions; the control variables of the throwing object event include object type, object size, lane, distance from the monitoring point and lighting conditions; the control variables of the congestion event include time period, road section, distance from the monitoring point and lighting conditions, and an orthogonal experiment is designed based on the control variables to cover the control variable combinations, high-fidelity test data is collected according to the orthogonal experiment design, and a basic real data set is formed.

4. The system for evaluating the effect of visual detection of abnormal events on a highway, characterized in that including: The analysis module is configured to analyze characteristics of various expressway abnormal events and determine quantified judgment rules to construct a standardized expressway abnormal event scene atlas, wherein the expressway common abnormal events include four types of abnormal parking events, pedestrian intrusion events, littering events and congestion events. The construction module is configured to analyze key influence variables of expressway abnormal events based on the standardized expressway abnormal event scene atlas, form a basic real data set by using orthogonal test design to control variables, and construct an expressway abnormal event simulation data set by screening the realness and dynamic logic rationality of the generated events after expanding the basic real data set, wherein the expanded basic real data set is first screened for realness by using a binary network based on a convolutional neural network, and the data set that passes the realness screening is screened for dynamic logic rationality by using an improved Transformer, wherein the improved Transformer is a dynamic spatio-temporal graph attention module introduced in the encoder to enhance the modeling capability of the data dynamic logic relationship, and the improved Transformer is specifically: The Transformer original layer 6 encoder output is dynamically constructed into an interaction graph, wherein in the interaction graph, the nodes represent the traffic participants, and the edges represent the potential interaction relationship between the traffic participants; The dynamic spatio-temporal graph attention module uses the graph attention mechanism to explicitly model the time sequence causal relationship and behavior intention between the participants; A cross-attention fusion layer is introduced to aggregate and enhance the output of the dynamic spatio-temporal graph attention module and the output of the Transformer original layer 6 encoder, and the aggregated and enhanced features are processed by the Transformer original layer 7 to layer 12 encoder to obtain a multi-dimensional evaluation vector; The evaluation module is configured to test the expressway video detection model by using the expressway abnormal event simulation data set, obtain a comprehensive test result by combining multi-dimensional evaluation indexes, wherein the multi-dimensional evaluation indexes include overall detection effect, night detection capability, lane detection capability, event fusion capability and resource occupation situation, and finally construct a multi-dimensional radar plot area penalty model to comprehensively evaluate the expressway video detection model to form a multi-dimensional algorithm evaluation index system. The constructed detection effect evaluation system is used to evaluate the overall detection effect of the to-be-tested model, and the detection rate, accuracy rate and repetition rate are used as evaluation indexes; the night monitoring capability of the to-be-tested model in all event scenarios is evaluated, and the average detection rate of the to-be-tested model in the night environment is used as the night detection capability evaluation index; the lane detection capability of the to-be-tested model in the abnormal parking scenario is evaluated, and the accuracy rate of the lane detection of the to-be-tested model in the scenario is used as the lane detection capability evaluation index; the event fusion capability of the to-be-tested model in the abnormal parking and congestion scenario is evaluated, and the repetition rate of the event that needs to be fused and reported by the to-be-tested model in the scenario is used as the event fusion capability evaluation index; the resource occupation of the to-be-tested model in all events is evaluated, and the average CPU, memory, display memory and graphics card computing power average occupation rate during the test period are used as the resource occupation evaluation index; the to-be-tested model is comprehensively evaluated in the above dimensions, a multi-dimensional radar chart area penalty model is constructed, the comprehensive score is represented by the radar chart area, the dimensions are weighted by using the expert scoring method, and a penalty factor is designed to punish the score of the to-be-tested model with obvious defects, and finally the comprehensive evaluation result of the to-be-tested model is obtained; the penalty factor is designed to punish the score of the to-be-tested model with obvious defects, including: for each to-be-tested model, calculating the short board attention index of each dimension, finding the most critical short board according to the short board attention index of each dimension, and setting a penalty factor based on the most critical short board.

5. An electronic device, comprising: Computer program product, comprising a memory and a processor as well as computer instructions stored on the memory and running on the processor, which, when run by the processor, complete the method of any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, Computer program product, comprising a memory and a processor as well as computer instructions stored on the memory and running on the processor, which, when run by the processor, complete the method of any one of claims 1-3.

7. A computer program product, characterised in that, Computer program product, comprising a memory and a processor as well as computer instructions stored on the memory and running on the processor, which, when run by the processor, complete the method of any one of claims 1-3.

Images