Method, system and equipment for analyzing highway video cloud network monitoring events

Abstract

The present invention belongs to the fields of intelligent transportation, video image processing and artificial intelligence analysis, and provides an event analysis method, system and equipment based on highway video cloud network monitoring. It can identify and track the speed, steering angle, etc. of the vehicle in real time through the video stream collected by the roadside camera in combination with the pre-trained visual model. According to the historical correct detection rate and false alarm rate, the system dynamically generates the traffic flow certificate ratio component, the stay frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold, which are used for the judgment of congestion, accidents and illegal intrusion events respectively. It also introduces a video bit rate and packet loss rate monitoring mechanism to mark and report video quality anomalies. This method has adaptive adjustment capabilities and is suitable for long-term operation scenarios after the model is cold started. It can effectively improve the accuracy of traffic event recognition and the robustness of the system.

Description

Technical Field

[0001] The present invention belongs to the fields of intelligent transportation, video image processing and artificial intelligence analysis, and specifically relates to a method, system and equipment for analyzing highway video cloud network monitoring events. Background Art

[0002] With the rapid development of intelligent urban transportation systems, road video surveillance data has become a crucial data source for traffic status analysis and event identification. Especially on highways, urban expressways, and congestion-prone areas, using video streams to detect traffic incidents has become a key technology for improving road safety and traffic efficiency.

[0003] Existing computer vision-based video event recognition technology primarily relies on deep learning models to process video data captured by cameras, identifying events such as traffic jams, traffic accidents, and illegal vehicle intrusions. Such systems typically include video acquisition, edge access, image recognition, trajectory tracking, behavior modeling, and event output. This includes, but is not limited to, data from video surveillance, log analysis, and real-time bitrate detection. Comprehensive statistical analysis of various monitored events can be performed, including total event counts, false alarms, true alarms, the percentage of each type of event, and rankings of route events. A unified data interface can be designed to collect event data from various subsystems. Using big data analytics and machine learning algorithms, events can be automatically identified and classified, distinguishing true from false alarms, and calculating various statistical indicators. A visual dashboard can be constructed to display real-time statistical results, trend charts, rankings, and other information, supporting rapid decision-making by managers.

[0004] However, in actual deployment, existing technologies have difficulty in decoupling model training and deployment. Existing technologies mostly rely on pre-trained visual models, which are fine-tuned using existing labeled data in the initial deployment phase to improve accuracy. However, after the system enters the large-scale operation stage, it is limited by computing resources and network transmission bandwidth, and the model parameters cannot be continuously updated, resulting in model performance degradation over time, especially when specific road sections, time periods or traffic conditions change, which is prone to false alarms or missed alarms. For example, the patent document with announcement number CN114738627B provides an intelligent traffic monitoring system and monitoring method, in which event judgment parameters are rigid and lack self-adaptation capabilities. For example, the vehicle speed threshold, angle deviation judgment standard, stay time threshold, etc. are mostly set as fixed empirical values, without considering the current recognition accuracy of the model, real-time traffic conditions, or changes in environmental factors, which can easily cause the rules to fail. For example, the same threshold performs very differently during the day and at night, and at high speeds and in congested conditions. However, general systems lack the ability to recognize the quality of their own input data. For example, the patent document with announcement number CN118247761B describes an on-board monitoring system for intelligent traffic inspection vehicles. Because traditional systems generally ignore the impact of transmission quality indicators such as video bit rate and network packet loss on recognition results, once the video quality degrades, even if the recognition result is abnormal, it is difficult for the system to identify whether it is a misjudgment due to inaccurate viewing, thereby affecting the credibility of the judgment result. Current technology is limited by the inability to continuously train models. The existing system lacks a mechanism that does not change the main model structure but can dynamically optimize based on historical judgment results and real-time recognition performance, often causing the system to fall into a dilemma where it cannot adjust parameters or retrain. Therefore, in order to achieve intelligent fusion and real-time statistics of cross-system data, the accuracy and visualization level of event monitoring should be comprehensively improved to provide decision support for intelligent monitoring and operation and maintenance management. Summary of the Invention

[0005] The purpose of the present invention is to propose a method, system and equipment for analyzing highway video cloud network monitoring events to solve one or more technical problems existing in the prior art and at least provide a beneficial option or create conditions.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for analyzing highway video cloud network monitoring events is provided, the method comprising the following steps:

[0007] The video stream collected by the roadside camera is connected to the analysis system through the edge gateway. The video frame is accompanied by the camera number, geographic coordinates and timestamp;

[0008] The analysis system analyzes each frame of image, identifies the target object and calculates its moving speed and turning angle, and analyzes and outputs the historical correct detection rate and historical false alarm rate;

[0009] Based on the data of the video stream collected by the roadside camera, the traffic flow certificate ratio component, the dwell frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold are dynamically generated: the traffic flow certificate ratio component is generated by combining the road speed limit and the historical free flow speed with the adjustment certificate ratio primer, wherein the historical detection certificate ratio is calculated using the historical correct detection rate and the historical false alarm rate, and the adjustment certificate ratio primer is formed by the historical detection certificate ratio; the dwell frame threshold is obtained by multiplying the shortest duration required for congestion judgment by the video sampling rate and rounding up; the corner mutation certificate ratio component is obtained by adding the mean of the statistical steering angle change to the corner mutation benchmark and the historical detection certificate ratio; the accident judgment time threshold is obtained by adding the congestion judgment duration and the pre-set accident delay compensation time; the restricted area stay time threshold is obtained by dividing the physical length of the prohibited area by the real-time average vehicle speed of the same road section;

[0010] Apply target detection and tracking algorithms to each frame of the image, assign a unique identifier to each target object, and calculate speed and angle changes in real time. For each identified target, obtain its speed and low-speed frame number, and determine whether a traffic jam has occurred by combining the traffic flow certificate ratio component and the stay frame threshold. Obtain the single-frame steering angle change and speed change of each identified target, and determine whether an accident has occurred by combining the angle mutation certificate ratio component and the traffic flow certificate ratio component. When the center of the bounding box of each identified target enters the set prohibited area and the continuous residence time of the restricted area reaches or exceeds the restricted area residence time threshold, an illegal intrusion event is determined.

[0011] Aggregate multiple recognition results of the same identifier, the same event type, and adjacent or overlapping in time to form a complete event record, which includes the event type, target identifier, camera number, start and end time, and geographic coordinates.

[0012] Furthermore, the analysis system adopts a real-time detection network based on deep learning and a multi-target association tracking algorithm to achieve real-time calculation and annotation of the positioning and recognition of the target in each frame and its speed and angle changes.

[0013] Furthermore, the stay frame threshold is dynamically calculated based on the shortest congestion determination time and the actual sampling frame rate of the video stream and rounded up; wherein, the average flow rate is calculated based on the speed of each vehicle in the video stream data collected by the roadside camera, and vehicles with a speed lower than 10% of the average flow rate in the video stream data are judged as congested vehicles, and the average value of the stay time of congested vehicles is taken as the shortest congestion determination time, and frames in the video stream data with a speed lower than 10% of the average flow rate are taken as low-speed frames, and the number of low-speed frames therein is taken as the number of low-speed frames.

[0014] Furthermore, the accident judgment time threshold is obtained by adding the shortest congestion judgment time and the accident delay compensation time, wherein the accident delay compensation time is the average value of the time from the occurrence of stopping to resuming movement of each vehicle in the video stream collected by the roadside camera.

[0015] Furthermore, the restricted area stay time threshold is calculated based on the ratio of the physical length of the restricted area to the real-time average driving speed of the road section, wherein the physical length of the restricted area is the length obtained by proportionally scaling or actual measurement in physical space based on the prohibited area set in the video stream.

[0016] Furthermore, the traffic flow certificate ratio component is generated by combining the adjusted certificate ratio with the road speed limit and the historical free flow speed, and specifically includes:

[0017] Obtain the historical correct detection rate, historical false alarm rate, target detection rate, and target false alarm rate; wherein the historical correct detection rate is the frequency of detecting real congestion events within a period of time; the historical false alarm rate is the frequency of misjudging normal traffic as congestion within the same period of time;

[0018] The ratio of the historical correct detection rate to the historical false alarm rate is the historical detection-verification ratio, the natural logarithm function of the historical detection-verification ratio is the historical verification ratio, and the ratio of the historical verification ratio to 100 is the adjusted verification ratio.

[0019] The target false alarm rate is obtained by dividing the historical false alarm rate by the historical detection rate, and the target detection rate is obtained by subtracting the target false alarm rate from the total.

[0020] Selecting a sampling time window, obtaining a correct detection rate and a false alarm rate within the sampling time window, and if the correct detection rate is less than the target detection rate, updating the specific value of the adjusted certificate ratio primer by multiplying the ratio obtained by dividing the historical correct detection rate by the target detection rate;

[0021] The road speed limit of the road section to be detected is obtained, and the arithmetic average of the vehicle flow speeds of the road section to be detected in the past period of time, which may be a plurality of different sampling time windows in the past, is obtained as a historical free flow speed. The minimum value of the road speed limit and the historical free flow speed is obtained and combined with the updated adjustment ratio to generate a driving speed threshold.

[0022] Furthermore, the corner mutation verification ratio component is obtained by adding the mean of the statistical steering angle change to the historical detection verification ratio and the corner mutation benchmark, and specifically includes:

[0023] The maximum value in the rotation angle difference sequence within the sampling time window is the rotation angle difference peak value;

[0024] Calculate the ratio of the turn sequence standard deviation to the turn sequence mean as the sequence turn coordinate, multiply the sequence turn coordinate by the turn difference peak to obtain the turn mutation benchmark, and combine the turn sequence mean plus the historical detection certificate ratio with the turn mutation benchmark to obtain the turn mutation certificate ratio component;

[0025] The historical detection certificate ratio is combined with the corner mutation benchmark in a multiplicative manner.

[0026] Furthermore, it also includes, when the video stream collected by the roadside camera is connected to the analysis system through the edge gateway, the average bit rate and network packet loss rate of the video stream are monitored in real time to obtain the real-time packet loss rate, and the video quality abnormality is marked as a special event and uploaded through the acceptable bit rate certificate ratio lower limit and the packet loss rate certificate ratio upper limit; wherein, the positive alarm rate of multiple different sampling time windows is continuously monitored and the average of the positive alarm rate is obtained, the acceptable bit rate certificate ratio lower limit is automatically taken as the product of the average bit rate and the average of the positive alarm rate based on historical statistics, the packet loss rate certificate ratio upper limit is the product of the average packet loss rate and the historical detection certificate ratio, and the average packet loss rate is the arithmetic mean of the real-time packet loss rate.

[0027] The present invention also provides a highway video cloud network monitoring event analysis system, the highway video cloud network monitoring event analysis system comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the highway video cloud network monitoring event analysis method, the highway video cloud network monitoring event analysis system can be run on computing devices such as desktop computers, laptop computers, PDAs, and cloud data centers, and the executable system may include, but is not limited to, a processor, a memory, and a server cluster, and the processor executes the computer program to run in the following system units:

[0028] A data acquisition unit, which is used to connect the video stream collected by the roadside camera to the analysis system through the edge gateway, and attach the camera number, geographic coordinates and timestamp data to each frame in the video stream. The analysis system analyzes and identifies the movement speed and angle of the vehicle in each frame, and analyzes and outputs the historical correct detection rate and historical false alarm rate;

[0029] A variable operation unit is used to dynamically generate a traffic flow certificate ratio component, a stay frame threshold, a corner mutation certificate ratio component, an accident judgment time threshold, and a restricted area stay time threshold based on the video stream data collected by the roadside camera;

[0030] The target recognition unit applies a real-time target detection and tracking algorithm to each frame, assigns a unique identifier to each pedestrian or vehicle, and calculates its instantaneous speed and the change in steering angle between adjacent frames;

[0031] The target judgment unit is used to determine, for each marked target, that a traffic congestion event has occurred when its speed is continuously lower than the traffic flow certificate ratio component and the number of low-speed frames reaches or exceeds the retention frame threshold; to determine that an accident event has occurred when its single-frame steering angle change exceeds the sudden change angle certificate ratio component and its speed subsequently remains lower than the traffic flow certificate ratio component within the accident judgment time threshold; and to determine that an illegal intrusion event has occurred when the center of the bounding box of each marked target enters a set prohibited area and its continuous residence time reaches or exceeds the restricted area residence time threshold;

[0032] The judgment transmission unit is used to aggregate multiple judgment results of the same identification, the same event type, and which are adjacent or overlapping in time, and output them as a complete event record. The event record includes the event type, target identification, camera number, start and end time, and geographic coordinates.

[0033] Correspondingly, the present invention also provides an electronic device, a readable storage medium, and a computer program product:

[0034] An electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for analyzing highway video cloud network monitoring events and the steps therein.

[0035] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for analyzing highway video cloud network monitoring events and the steps thereof.

[0036] A computer program product includes a computer program, which, when executed by a processor, implements the method for analyzing highway video cloud network monitoring events and the steps thereof.

[0037] The beneficial effects of the present invention are as follows: the present invention relates to an event analysis method, system and equipment based on highway video cloud network monitoring, which can perform real-time identification and tracking of vehicle speed, steering angle, etc. through the video stream collected by the roadside camera in combination with the pre-trained visual model. According to the historical correct detection rate and false alarm rate, the system dynamically generates the traffic flow certificate ratio component, the stay frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold, which are used for the judgment of congestion, accidents and illegal intrusion events respectively. A video bit rate and packet loss rate monitoring mechanism is also introduced to mark and report video quality anomalies. The method has adaptive adjustment capabilities and is suitable for long-term operation scenarios after the model is cold started. It can effectively improve the accuracy of traffic event recognition and the robustness of the system.

[0038] The present invention implements an external control mechanism for the behavioral accuracy after the model is cold-started. By introducing an adjustment ratio primer, combined with the road speed limit and the historical free flow speed, a traffic flow ratio component is adaptively generated for dynamic discrimination of events such as traffic congestion and accidents. This design overcomes the performance degradation problem caused by the inability to update model parameters after the cold-start phase, and realizes self-learning and self-optimization of the external judgment strategy under the premise of a fixed main model. The present invention also realizes accurate identification of abnormal vehicle deviation behavior by constructing a corner mutation ratio component, combined with the mean, standard deviation and peak data of the angle change in the real-time sampling window, and effectively improves the response capability to sudden events such as traffic accidents. It can effectively distinguish between normal lane change behavior and unexpected steering behavior, and enhance the system's robustness in discrimination under actual complex road conditions. In a highway video cloud networking environment, it can simultaneously identify multiple types of traffic abnormalities such as congestion, accidents, and illegal intrusions, and has the ability to aggregate repeated judgment results into complete event records, which facilitates unified system scheduling and record management, and is suitable for large-scale deployment and deep integration in intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other features of the present invention will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. In the drawings of the present invention, the same reference numerals represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present invention. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings:

[0040] Figure 1 Shown is a flow chart of an event analysis method based on highway video cloud network monitoring;

[0041] Figure 2 Shown is the system structure diagram of the highway video cloud network monitoring event analysis system. DETAILED DESCRIPTION

[0042] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict.

[0043] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0044] like Figure 1 The flowchart of the method for analyzing highway video cloud network monitoring events according to the present invention is shown below. Figure 1 To illustrate the method, system and equipment for analyzing highway video cloud network monitoring events according to the implementation manner of the present invention.

[0045] The present invention proposes a method for analyzing highway video cloud network monitoring events, which specifically includes the following steps:

[0046] The video stream collected by the roadside camera is connected to the analysis system through the edge gateway. Each frame in the video stream is accompanied by the camera number, geographic coordinates and timestamp data. The analysis system analyzes and identifies the movement speed and angle of the vehicle in each frame, and outputs the historical correct detection rate and historical false alarm rate.

[0047] Based on the video stream data collected by the roadside camera, the traffic flow certificate ratio component, the stay frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold are dynamically generated;

[0048] Apply real-time object detection and tracking algorithms to each frame, assign a unique identifier to each pedestrian or vehicle, and calculate its instantaneous speed and steering angle change between adjacent frames;

[0049] For each marked target, a traffic jam is determined to have occurred when its speed is continuously lower than the traffic flow certificate ratio and the number of low-speed frames reaches or exceeds the retention frame threshold. An accident is determined to have occurred when the steering angle change in a single frame exceeds the sudden change certificate ratio and the speed subsequently remains lower than the traffic flow certificate ratio within the accident determination time threshold. An illegal intrusion is determined to have occurred when the center of the bounding box of each marked target enters the set prohibited area and the duration of its stay reaches or exceeds the restricted area retention time threshold.

[0050] Multiple identification results of the same identification, the same event type, and being adjacent or overlapping in time are aggregated and output as a complete event record, which includes the event type, target identification, camera number, start and end time, and geographic coordinates.

[0051] Among them, the traffic flow certificate ratio component is generated by combining the adjustment certificate ratio primer with the road speed limit and the historical free flow speed; the stay frame threshold is obtained by multiplying the shortest duration required for congestion judgment by the video sampling rate and rounding up; the corner mutation certificate ratio component is obtained by adding the mean of the statistical steering angle change plus the corner mutation benchmark and the historical detection certificate ratio; the accident judgment time threshold is obtained by adding the congestion judgment duration and the pre-set accident delay compensation duration; the restricted area stay time threshold is obtained by dividing the physical length of the prohibited area by the real-time average vehicle speed of the same road section.

[0052] In some embodiments, streaming video access can be achieved through the H264 video stream collected by the roadside camera through the edge gateway access system, and the gateway pushes the video frame to the cloud or regional node. Each frame carries metadata such as camera number, GPS coordinates, timestamp, etc. Each frame of the image is used to identify all pedestrians and vehicles, including but not limited to YOLO-V5 and / or other real-time detection models, and a unique ID is assigned to each target through tracking algorithms such as DeepSORT. At the same time, the instantaneous speed of the target based on the position difference between adjacent frames and the steering angle difference between the target in the current frame and the previous frame are calculated. In particular, a pre-trained visual large model is embedded in the analysis system, including but not limited to the visual large model of YOLO, etc. The pre-trained visual large model can be used to identify and classify event behaviors based on the data of the video stream collected by the input roadside camera. The analysis system should have a cold start phase at the beginning of its operation. During the cold start phase, the pre-trained large visual model identifies and classifies event behaviors based on the training data of the existing video stream, and calculates the accuracy rate obtained from the identification and classification of event behaviors as the historical correct detection rate, and the error rate obtained as the historical false alarm rate, and fine-tunes the model, that is, the training and update of part of the neural network. The training data of the existing video stream can be training data in the form of a training set and a verification set, in which the answers to the identification and classification of event behaviors used as standards for verification are annotated, for example, the numerical calculation of recall and precision is selected in the process of calculating the F1-score of the model. Thereby, it is convenient for the pre-trained large visual model to obtain the historical correct detection rate and the historical false alarm rate during the cold start phase based on the data of the video stream collected by the input roadside camera. After the cold start phase, the normal operation phase begins. According to existing technologies, if the pre-trained large visual model still needs to be fine-tuned during the normal operation phase, the computing power cost will be very high. It is difficult to support the data of so many video streams on the road. The model parameter training and updating are always necessary, and the data cluster is under great pressure. However, if the parameters are not updated, the recognition efficiency and accuracy are not enough, which often puts technical personnel in this field in a dilemma.

[0053] After entering the normal operation stage, due to the huge amount of highway video data and limited computing power, the model parameters cannot be updated continuously, which may lead to a decrease in recognition accuracy. The system needs to rely on cold start period statistics as subsequent judgment criteria, but once these static data deviate from the actual situation, the robustness of the model will drop sharply.

[0054] The displacement of the vehicle in the video stream is calculated along with the video playback time and its displacement, and is proportionally scaled to the actual physical distance according to the lens shooting ratio to obtain the actual speed and actual moving distance of the vehicle in the video stream, thereby obtaining the instantaneous speed of the vehicle in each frame of the video stream.

[0055] The identifiers are information labeling vehicle targets within each frame of the video stream, and the event types are textual descriptions of events detected for vehicle targets within each frame of the video stream. The event type, target identifier, associated camera number, start and end times, and geographic coordinates represent data types that can be stored in a database. The generated "congestion," "accident," and "intrusion" events should be understood as event recognition classifications for road video streams. Any equivalent alternatives fall within the scope of protection of the embodiments described herein.

[0056] Furthermore, the analysis system adopts a real-time detection network based on deep learning and a multi-target association tracking algorithm to achieve real-time calculation and annotation of the positioning and recognition of the target in each frame and its speed and angle changes.

[0057] In some embodiments, the number of frames in the video stream captured by a roadside camera is n, where the chronological order of the frames is t. For two frames, target recognition and tracking can be performed using machine vision algorithms and models. The target in each frame is detected in each frame. The coordinates of frame t are labeled (x1=250px, y1=400px), and frame t+1 = (x2=255px, y2=404px). The pixel scaling factor is 0.1 m / px, and the real-time inter-frame interval is 1 / 5fps = 0.2s. The instantaneous bitrate and network packet loss rate of each video stream are recorded.

[0058] The dynamic displacement between the two coordinates is then calculated to be approximately 0.64 m. The instantaneous velocity can be calculated as 3.2 m / s by dividing 0.64 m by 0.2 s. The angular shift between the coordinates in the two frames is recorded. The lane axis can be adjusted based on the camera angle, obtaining its direction in real time. The difference between the two frames' axis directions is the turning angle, which is then used to determine the steering angle change for a single frame, which is then used for subsequent dynamic analysis.

[0059] The analysis system may also employ a deep learning-based real-time detection network, YOLOv5, and a multi-target association tracking algorithm, such as MHT and / or PDA, to perform image recognition on each vehicle image in each frame. This may include, but is not limited to, using a ground-truth bounding box algorithm to identify the bounding box of each vehicle image within the frame. The geometric center of each vehicle image's bounding box is its bounding box center. For example, at a certain time in frame t, the target bounding box center moves from (x=250px, y=400px) to (x=255px, y=402px) in frame t+1. With a camera field of view conversion factor of 0.1 m / px, the distance change is approximately 0.54 m. With an inter-frame time of 1 / 6 s, the instantaneous velocity is approximately 0.54 / (1 / 6), which is 3.24 m / s. The steering angle is calculated as 12° based on the change in the bounding box's principal axis direction.

[0060] In some embodiments, the calculation method for the true alarm rate and false alarm rate can be obtained by importing data from an external database and / or performing recognition and analysis on video streams collected by roadside cameras. Data detection and event records are obtained from a time sampling window, including 7 events marked as "congestion", 3 events as "accident", and 2 events as "trespass". After verification, 8 true alarms and 4 false alarms are detected. The dynamic calculation shows that the true alarm rate is 8 divided by (7 + 3 + 2), which is approximately 66.7%, and the false alarm rate is approximately 33.3%.

[0061] Furthermore, the stay frame threshold is dynamically calculated based on the shortest congestion determination time and the actual sampling frame rate of the video stream and rounded up;

[0062] Among them, the average flow speed is calculated based on the speed of each vehicle in the video stream data collected by the roadside camera, and the vehicles in the video stream data whose speed is lower than 10% of the average flow speed are judged as congested vehicles. The average value of the stay time of congested vehicles is used as the shortest congestion judgment time, and the frames in the video stream data whose speed is lower than 10% of the average flow speed are used as low-speed frames, and the number of low-speed frames is used as the number of low-speed frames.

[0063] In some embodiments, the minimum congestion determination time can be automatically calculated through traffic monitoring to obtain 2.5 seconds. If network congestion causes the actual sampling frame rate of the video stream, the actual frame rate can be reduced from the default frame rate of 6 fps to, for example, 5 fps, to ensure consistent time determination accuracy under different frame rates. The dwell frame threshold is the result of dynamically multiplying the minimum congestion determination time by the actual sampling frame rate of the video stream. When 2.5 seconds is multiplied by 5 fps, the dwell frame threshold is 13 frames. If the frame rate returns to 6 fps, the dwell frame threshold is 15 frames.

[0064] Furthermore, the accident judgment time threshold is obtained by adding the shortest congestion judgment time and the accident delay compensation time, wherein the accident delay compensation time is the average value of the time from the occurrence of stopping to resuming movement of each vehicle in the video stream collected by the roadside camera.

[0065] In some embodiments, the accident delay compensation duration can be automatically obtained based on an average stagnation of 4 seconds in the video stream data collected by the roadside camera.

[0066] Furthermore, the restricted area stay time threshold is calculated based on the ratio of the physical length of the restricted area to the real-time average driving speed of the road section, wherein the physical length of the restricted area is the length obtained by proportionally scaling or actual measurement in physical space based on the prohibited area set in the video stream.

[0067] In some embodiments, the physical length of the restricted area can be statistically measured by map measurement and is preferably set to 15m.

[0068] The real-time average speed for the road section is the real-time average speed for the same road section in the video. The real-time average speed can be calculated based on real-time traffic statistics, and preferably, the real-time average speed can be 12 m / s. The restricted zone dwell time threshold is the physical length of the restricted zone (15 m) divided by the real-time average speed of the road section (12 m / s), which is approximately 1.25 seconds. If a sudden increase in traffic volume is detected in the video stream, causing the average speed to drop to 10 m / s, the restricted zone dwell time threshold is approximately 1.5 seconds. This adaptively determines the illegal stay determination criteria for different speed limits and different road sections.

[0069] Furthermore, the traffic flow certificate ratio component is generated by combining the adjusted certificate ratio with the road speed limit and the historical free flow speed, and specifically includes:

[0070] Obtain the historical correct detection rate, historical false alarm rate, target detection rate, and target false alarm rate, where:

[0071] The historical correct detection rate is the frequency of detecting real congestion events in the past period of time;

[0072] The historical false alarm rate is the frequency of misjudging normal traffic as congestion during the same period in the past;

[0073] The ratio of the historical correct detection rate to the historical false alarm rate is the historical detection-verification ratio, the natural logarithm function of the historical detection-verification ratio is the historical verification ratio, and the ratio of the historical verification ratio to 100 is the adjusted verification ratio.

[0074] The target false alarm rate is obtained by dividing the historical false alarm rate by the historical detection rate, and the target detection rate is obtained by subtracting the target false alarm rate from the total.

[0075] Selecting a sampling time window, obtaining a correct detection rate and a false alarm rate within the sampling time window, and if the correct detection rate is less than the target detection rate, updating the specific value of the adjusted certificate ratio primer by multiplying the ratio obtained by dividing the historical correct detection rate by the target detection rate;

[0076] The road speed limit of the road section to be detected is obtained, and the arithmetic average of the vehicle flow speeds of the road section to be detected in the past period of time, which may be a plurality of different sampling time windows in the past, is obtained as a historical free flow speed. The minimum value of the road speed limit and the historical free flow speed is obtained and combined with the updated adjustment ratio to generate a driving speed threshold.

[0077] In Example 2, the road speed limit of 90 km / h is converted to 25 m / s, the historical free flow speed is 18 m / s, and the safety factor is 2%. The smaller of the two, 18 m / s, is multiplied by 2% to obtain a speed threshold of 0.36 m / s. However, this implementation still has some problems, so a preferred embodiment is proposed.

[0078] In a preferred embodiment, the safety factor determines the mapping ratio of the traffic flow certificate ratio component to the road speed limit or free flow speed. If it is too high, congestion may be missed, and if it is too low, false alarms may be caused.

[0079] To this end, the historical correct detection rate, historical false alarm rate, target detection rate and target false alarm rate are obtained. The historical correct detection rate is the frequency of detecting real congestion events in the past period of time;

[0080] The historical false alarm rate is the frequency of misjudging normal traffic as congestion during the same period in the past;

[0081] The past period of time may be a collection of multiple different sampling time windows in the past, and the duration of a sampling time window is about 5 seconds to 10 seconds.

[0082] For example, the recent statistical real congestion detection rate is 85% of the historical correct detection rate, and the recent statistical real congestion detection false alarm rate is 15% of the historical false alarm rate.

[0083] The target detection rate and the target false alarm rate correspond to the historical correct detection rate and the historical false alarm rate, the target detection rate is higher than the historical correct detection rate, and the target false alarm rate is lower than the historical false alarm rate.

[0084] Specifically, the ratio of the historical correct detection rate to the historical false alarm rate is the historical verification ratio, the natural logarithm function of the historical verification ratio is the historical validation ratio, and the ratio of the historical validation ratio to 100 is the adjusted validation ratio. For example, if Ln (the historical verification ratio) is calculated to be approximately 1.7, then the historical validation ratio equal to 100 is 1.7%.

[0085] Then, divide the historical false alarm rate by the historical verification-to-verification ratio to obtain the target false alarm rate. The target detection rate is calculated by subtracting the target false alarm rate from one. For example, if the historical verification-to-verification ratio is 1.7 and the historical false alarm rate is 15%, the target false alarm rate obtained by dividing the historical false alarm rate by the historical verification-to-verification ratio can be rounded to 9%, resulting in a target detection rate of 1-9% = 91%.

[0086] Furthermore, after the initial calculation, the traffic flow ratio component is dynamically adjusted based on the historical road speed limit and off-peak average speed for the road section corresponding to each camera. A recent sampling time window is selected, which may not be within the past period, and the true correct detection rate and true false alarm rate within the recent sampling time window are obtained. If the true correct detection rate is less than the target detection rate, the specific value of the adjusted ratio is updated by multiplying the ratio of the historical correct detection rate divided by the target detection rate. For example, when the true false alarm rate within the recent sampling time window increases relative to the previous sampling time window, the specific value of the adjusted ratio is updated to 1.6% by multiplying the ratio of 0.93, which is the historical correct detection rate of 85% divided by the target detection rate of 91%. Preferably, the ratio is updated at all times by multiplying the ratio by the smaller of the two. The road speed limit of the road section to be detected is obtained, and the arithmetic average of the vehicle flow speeds of the road section to be detected in the past period of time, which may be a plurality of different sampling time windows in the past, is obtained as a historical free flow speed. The minimum value of the road speed limit and the historical free flow speed is obtained and combined with the updated adjustment ratio to generate a driving speed threshold.

[0087] The present invention uses statistical indicators such as historical correct detection rates and historical false alarm rates to calculate the historical detection-to-certification ratio. This ratio is then derived using the natural logarithm of this ratio, and combined with a percentage constant to form an adjusted-to-certification ratio primer. Ultimately, the adjusted-to-certification ratio primer is combined with the smaller of the road speed limit or the historical free-flow speed to generate the traffic flow-to-certification ratio component used for event recognition. This ratio is dynamically adjusted based on the true alarm / false alarm rate within the actual sampling window.

[0088] Traditional traffic event judgment models based on speed thresholds often rely on manual experience to set static parameters. They are unable to adapt to changes in model accuracy under different road sections, different time periods, and different traffic conditions. They are unable to automatically adjust to fluctuations in recognition errors, are prone to insufficient generalization or overfitting problems, and lack a compensation mechanism for the degradation of model accuracy during continuous operation.

[0089] The technical solution described in the present invention introduces an adjustment ratio primer as a quantitative regulator for model accuracy control, and combines historical discrimination performance to construct an adjustable speed threshold through the calculation of the positive alarm rate / false alarm rate. This design takes into account both the modeling capability of the dynamic volatility of the data set itself and the closed-loop feedback control of the recognition accuracy index. The concept of information entropy can be introduced through the natural logarithm function to effectively smooth the fluctuation of accuracy response. This mechanism is equivalent to giving the event discrimination threshold a self-learning and self-adaptation capability, which enables the system to self-adjust the performance of different accuracy stages. It significantly improves the judgment accuracy under different traffic conditions, congestion, high speed, and abnormalities, and reduces the proportion of false alarms and missed alarms of the system in long-term operation. It reduces the frequency of manual intervention in parameter setting, improves the level of intelligent autonomy of the system, provides a quantifiable and traceable error correction path, and enables the system to have dynamic strategy adjustment capabilities.

[0090] The method described in this invention can alleviate the problem of model sluggishness after parameter freezing. Using the historical detection-to-evidence ratio obtained from a cold start as input, combined with real-time feedback to update and adjust the ratio, the decision threshold can be dynamically adjusted without retraining the main model. This creates a lightweight behavioral parameter relearning mechanism. Using the positive and false alarm rate data from the sampling window, it is equivalent to modifying the model's behavioral decisions without changing the model, resulting in pseudo-fine-tuning of the outer decision logic. This can also improve the model's long-term stability and adaptability, enable data-driven sustainable parameter migration, avoid performance solidification after a cold start, and resolve the engineering bottleneck of a dilemma.

[0091] Furthermore, the corner mutation verification ratio component is obtained by adding the mean of the statistical steering angle change to the historical detection verification ratio and the corner mutation benchmark, and specifically includes:

[0092] The maximum value in the rotation angle difference sequence within the sampling time window is the rotation angle difference peak value;

[0093] Calculate the ratio of the turn sequence standard deviation to the turn sequence mean as the sequence turn coordinate, multiply the sequence turn coordinate by the turn difference peak to obtain the turn mutation benchmark, and combine the turn sequence mean plus the historical detection certificate ratio with the turn mutation benchmark to obtain the turn mutation certificate ratio component;

[0094] The historical detection certificate ratio is combined with the corner mutation benchmark in a multiplicative manner.

[0095] In the fourth embodiment, a 5-second sampling time window is used as a sequence of angle difference values ​​within the 5-second sampling time window. The OpenCV module and the ultralytics module are used to identify the difference between the vehicle's moving angle in each frame and its angle in the previous frame. The sequence includes 8 frames, including: 4°, 6°, 3°, 7°, 5°, 8°, 2°, and 6°.

[0096] The angle sequence mean of the angle difference sequence calculated within the 5s sampling time window is approximately equal to 5.1°, and the angle sequence standard deviation is approximately equal to 2.1°;

[0097] In some embodiments, the turn mutation ratio component is obtained by adding the mean of the turn sequence to twice the standard deviation of the turn sequence. However, such an implementation still has some problems, and a preferred embodiment is proposed for this purpose.

[0098] In a preferred embodiment,

[0099] The maximum value of 8° in the rotation angle difference sequence within the 5s sampling time window is obtained as the rotation angle difference peak value;

[0100] The ratio of the turn angle sequence standard deviation (2.1°) to the turn angle sequence mean (5.1°) was calculated as 0.41, which was used as the sequence turn angle coordinate. The turn angle difference peak value (8°) was multiplied by the sequence turn angle coordinate (0.41) to obtain the turn angle mutation benchmark (3.28°). The turn angle sequence mean (5.1°) was added to the historical detection-certified ratio and the turn angle mutation benchmark to obtain the turn angle mutation ratio component (10.7). The historical detection-certified ratio (1.7) and the turn angle mutation benchmark (3.28°) were multiplied to obtain 5.6°.

[0101] The present invention calculates the sequential rotation angle coordinates by taking the mean and standard deviation of the rotation angle variation within a sampling time window. This is then multiplied by the peak angle to construct a sudden rotation angle benchmark. This benchmark is then combined with the historical detection and verification ratio to generate a sudden rotation angle verification ratio component, which is used to identify abnormal deflection events such as accidents.

[0102] In traditional traffic event recognition systems based on target trajectory analysis, spontaneous lane changes or circumventions often lead to misjudgments, especially when the camera angle is poor or the data is jittery. These systems are generally unable to effectively identify abnormal deviations or sudden changes in deflection. They lack the ability to statistically model angle fluctuations and cannot quantitatively determine critical deflections.

[0103] The technical solution of the present invention constructs a dynamic corner deviation index system, and establishes a benchmark for judging abnormal deviations by utilizing the ratio of peak value to standard deviation. This benchmark is combined with the historical judgment capability index to form a risk-adjusted angle mutation threshold. Through the above logic, accurate identification of abnormal angle changes is achieved, and the robustness of the system in handling irregular trajectory behaviors is enhanced. It can significantly reduce misjudgments in normal deviation behaviors such as bends and lane changes. It can also effectively identify nonlinear deflection behaviors caused by accidents or sharp turns, and improve the system's recognition sensitivity to trajectory incoherence or precursors of abnormal behavior.

[0104] In order to solve the problem that the system's perspective is fixed after a cold start and cannot adapt to the angular characteristics of a new road section, the technical solution described in the present invention provides a self-learning angular behavior calibration mechanism. Based on the dynamic collection of corner sequences for each road section, an angle fluctuation feature space is formed, which can adapt to changes in terrain and streamlines without updating the model structure. Local statistics such as the angle mean, standard deviation, and peak value are used to construct a dynamic safety belt to enhance the response capability to sudden behaviors such as accidents, and to make up for the model's blind spots in learning non-standard behaviors caused by insufficient coverage of the cold start data set. This can give the system behavioral cognitive boundaries, similar to using mathematical methods to define the behavioral boundaries of an existing model, such as whether this is a normal deflection or an accident, so that policy-level adjustments can be made without retraining.

[0105] Furthermore, it may preferably also include, when the video stream collected by the roadside camera is accessed to the analysis system through the edge gateway, real-time monitoring of the average bit rate and network packet loss rate of the video stream is performed to obtain the real-time packet loss rate, and the video quality abnormality is marked as a special event and uploaded through the acceptable bit rate certificate ratio lower limit and the packet loss rate certificate ratio upper limit; wherein, the positive alarm rate of multiple different sampling time windows is continuously monitored and the average of the positive alarm rate is obtained, the acceptable bit rate certificate ratio lower limit is automatically taken as the product of the average bit rate and the average of the positive alarm rate based on historical statistics, the packet loss rate certificate ratio upper limit is the product of the average packet loss rate and the historical detection certificate ratio, and the average packet loss rate is the arithmetic mean of the real-time packet loss rate.

[0106] In some embodiments, a 10-second sampling window is used, and the sampling bit rates within the 10-second sampling window are: 1600, 1400, 1500, 1300, and 1700 kbps; the real-time packet loss rates are: 2%, 6%, and 4%. During the process of the video stream accessing the analysis system through the edge gateway, i.e., during the video access stage, quality testing is performed to obtain and record the instantaneous bit rate and network packet loss rate of each video stream. Preferably, the system's ping command can be called through a subprocess library, and the output of the ping command can be parsed based on the output result to calculate the packet loss rate.

[0107] Continuously monitor the positive reporting rates of multiple different sampling time windows and obtain the average of the positive reporting rates. The lower limit of the acceptable code rate certification ratio can be automatically taken as the product of the average code rate and the average of the positive reporting rates based on historical statistics;

[0108] The upper limit of the packet loss rate ratio can be set as the product of the average packet loss rate and the historical detection ratio, and the average packet loss rate is the arithmetic mean of the real-time packet loss rate.

[0109] In one embodiment, the calculated average of the positive reporting rate is 75%, and the product of the average bit rate of 1500kbps and the average positive reporting rate of 75% is 1687.5kbps; the arithmetic mean of the real-time packet loss rate of 4% and the historical detection certificate ratio of 1.7 are combined to obtain 6.8%.

[0110] When the sampling bit rate within the sampling time window is lower than the lower limit of the acceptable bit rate ratio, it will be triggered and marked as an "insufficient bit rate" event; if the packet loss rate rises to exceed the upper limit of the packet loss rate ratio, it will trigger a "poor connectivity" event; the video data within the sampling time window including the "insufficient bit rate" event and the "poor connectivity" event will be uploaded and saved.

[0111] This invention incorporates real-time video bitrate and packet loss rate monitoring into the roadside camera video access analysis system. Lower and upper bounds for bitrate and packet loss are established by combining the average bitrate with the mean positive rate and the average packet loss rate with historical comparisons. If video quality parameters exceed these limits, an abnormality such as insufficient bitrate or poor connectivity is detected and reported.

[0112] Existing event recognition systems generally ignore low bitrates or high packet loss, which can lead to blurred targets and recognition failures. These systems cannot distinguish between recognition failure and signal loss. Furthermore, network quality fluctuations are uncontrollable and often ignored as black box variables. Sometimes, video quality issues are not even classified as events and cannot be included in analysis models, resulting in a lack of systematic response strategies.

[0113] The technical solution described in this invention, for the first time, incorporates video signal quality anomalies as part of the recognition event itself, using acceptable lower bitrate limits and upper packet loss rates as independent judgment thresholds. This logic establishes an input credibility model for the recognition system, linking data quality with behavioral judgment results. This is similar to giving the system self-awareness, allowing it to determine whether it sees clearly enough. This mechanism ensures data validity at the input source and is a key component of the recognition system's full-link quality control closed loop. It not only significantly improves the system's fault tolerance for low-quality video sources, but also enables automatic marking and removal of unreliable recognition segments, enhancing the overall credibility of event judgments. Introducing a signal source quality awareness dimension into the traffic video analysis system enhances system integrity and provides meta-event monitoring capabilities, identifying not only traffic events but also technical faults that affect event recognition. This allows for timely detection and diagnosis of the root causes of model output distortion, building a complete data-model-decision closed loop. Consequently, upon detecting bitrate or packet loss anomalies, quality issues are automatically reported, enabling the system to recognize its own capability failures, avoiding blind decisions or false positive / false negative events, and preventing performance misperceptions during cold start periods from extending into operational life.

[0114] The highway video cloud network monitoring event analysis system runs on any computing device such as a desktop computer, a laptop computer, a PDA or a cloud data center. The computing device includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps in the highway video cloud network monitoring event analysis method are implemented. The executable system may include, but is not limited to, a processor, a memory, and a server cluster.

[0115] The embodiment of the present invention provides a highway video cloud network monitoring event analysis system, such as Figure 2 As shown, the highway video cloud network monitoring event analysis system of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned highway video cloud network monitoring event analysis method embodiment are implemented. The processor executes the computer program to run in the following system units:

[0116] A data acquisition unit, which is used to connect the video stream collected by the roadside camera to the analysis system through the edge gateway, and attach the camera number, geographic coordinates and timestamp data to each frame in the video stream. The analysis system analyzes and identifies the movement speed and angle of the vehicle in each frame, and analyzes and outputs the historical correct detection rate and historical false alarm rate;

[0117] A variable operation unit is used to dynamically generate a traffic flow certificate ratio component, a stay frame threshold, a corner mutation certificate ratio component, an accident judgment time threshold, and a restricted area stay time threshold based on the video stream data collected by the roadside camera;

[0118] The target recognition unit applies a real-time target detection and tracking algorithm to each frame, assigns a unique identifier to each pedestrian or vehicle, and calculates its instantaneous speed and steering angle change between adjacent frames;

[0119] The target judgment unit is used to determine, for each marked target, that a traffic congestion event has occurred when its speed is continuously lower than the traffic flow certificate ratio component and the number of low-speed frames reaches or exceeds the retention frame threshold; to determine that an accident event has occurred when its single-frame steering angle change exceeds the sudden change angle certificate ratio component and its speed subsequently remains lower than the traffic flow certificate ratio component within the accident judgment time threshold; and to determine that an illegal intrusion event has occurred when the center of the bounding box of each marked target enters a set prohibited area and its continuous residence time reaches or exceeds the restricted area residence time threshold;

[0120] The judgment transmission unit is used to aggregate multiple judgment results of the same identification, the same event type, and which are adjacent or overlapping in time, and output them as a complete event record. The event record includes the event type, target identification, camera number, start and end time, and geographic coordinates.

[0121] Among them, in order to better unify the numerical linear relationship and probabilistic connection between physical quantities of different units of measurement, different physical quantities can be dimensionally processed.

[0122] Among them, preferably, all undefined variables in the present invention, if not clearly defined, can be manually set thresholds.

[0123] The highway video cloud network monitoring event analysis system can be run on computing devices such as desktop computers, laptops, PDAs, and cloud data centers. The highway video cloud network monitoring event analysis system includes, but is not limited to, a processor and a memory. Those skilled in the art will understand that the examples described are merely examples of the highway video cloud network monitoring event analysis method, system, and device, and do not constitute a limitation on the highway video cloud network monitoring event analysis method, system, and device. The system may include more or fewer components than the examples, or a combination of certain components, or different components. For example, the highway video cloud network monitoring event analysis system may also include input and output devices, network access devices, buses, and the like.

[0124] The present invention also provides an electronic device, a readable storage medium, and a computer program product:

[0125] An electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for analyzing highway video cloud network monitoring events and the steps therein.

[0126] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for analyzing highway video cloud network monitoring events and the steps thereof.

[0127] A computer program product includes a computer program, which, when executed by a processor, implements the method for analyzing highway video cloud network monitoring events and the steps thereof.

[0128] The term "electronic device" herein is intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The term "electronic device" may also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the inventions described and / or claimed herein.

[0129] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0135] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete component gate circuits or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the highway video cloud network-based monitoring event analysis system, and utilizes various interfaces and lines to connect the various sub-areas of the entire highway video cloud network-based monitoring event analysis system.

[0136] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the highway video cloud network monitoring event analysis method, system, and device by running or executing the computer programs and / or modules stored in the memory and accessing the data stored in the memory. The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0137] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0138] The present invention relates to an event analysis method, system and equipment based on highway video cloud network monitoring. The system can identify and track the speed, steering angle, etc. of the vehicle in real time through the video stream collected by the roadside camera in combination with the pre-trained visual model. According to the historical correct detection rate and false alarm rate, the system dynamically generates the traffic flow certificate ratio component, the stay frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold, which are used for the judgment of congestion, accidents and illegal intrusion events respectively. A video bit rate and packet loss rate monitoring mechanism is also introduced to mark and report video quality anomalies. The method has adaptive adjustment capabilities and is suitable for long-term operation scenarios after the model is cold started. It can effectively improve the accuracy of traffic event recognition and the robustness of the system. The present invention uses observable physical quantities and real-time statistical data to drive key parameters, and has a high degree of adaptability and scenario versatility.

[0139] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. Based on the highway video cloud network monitoring event analysis method, the video stream collected by the roadside camera is connected to the analysis system through the edge gateway. The analysis system analyzes each frame of the image, identifies the target object and calculates its moving speed and turning angle, and analyzes and outputs the historical correct detection rate and historical false alarm rate. The characteristics are: The method comprises: Based on the data of the video stream collected by the roadside camera, the traffic flow certificate ratio component, the dwell frame threshold, the corner mutation certificate ratio component, the accident judgment time threshold and the restricted area stay time threshold are dynamically generated: the traffic flow certificate ratio component is generated by combining the road speed limit and the historical free flow speed with the adjustment certificate ratio primer, wherein the historical detection certificate ratio is calculated using the historical correct detection rate and the historical false alarm rate, and the adjustment certificate ratio primer is formed by the historical detection certificate ratio; the dwell frame threshold is obtained by multiplying the shortest duration required for congestion judgment by the video sampling rate and rounding up; the corner mutation certificate ratio component is obtained by adding the mean of the statistical steering angle change to the corner mutation benchmark and the historical detection certificate ratio; the accident judgment time threshold is obtained by adding the congestion judgment duration and the pre-set accident delay compensation time; the restricted area stay time threshold is obtained by dividing the physical length of the prohibited area by the real-time average vehicle speed of the same road section; Apply target detection and tracking algorithms to each frame of the image, assign a unique identifier to each target object, and calculate speed and angle changes in real time. For each identified target, obtain its speed and low-speed frame number, and determine whether a traffic jam has occurred by combining the traffic flow certificate ratio component and the stay frame threshold. Obtain the single-frame steering angle change and speed change of each identified target, and determine whether an accident has occurred by combining the angle mutation certificate ratio component and the traffic flow certificate ratio component. When the center of the bounding box of each identified target enters the set prohibited area and the continuous residence time of the restricted area reaches or exceeds the restricted area residence time threshold, an illegal intrusion event is determined. Aggregate multiple recognition results of the same identifier, the same event type, and adjacent or overlapping in time to form a complete event record; The traffic flow certificate ratio component is generated by combining the adjusted certificate ratio with the road speed limit and the historical free flow speed, and specifically includes: Obtain the historical correct detection rate, historical false alarm rate, target detection rate, and target false alarm rate; wherein the historical correct detection rate is the frequency of detecting real congestion events within a period of time; the historical false alarm rate is the frequency of misjudging normal traffic as congestion within the same period of time; The ratio of the historical correct detection rate to the historical false alarm rate is the historical detection-verification ratio, the natural logarithm function of the historical detection-verification ratio is the historical verification ratio, and the ratio of the historical verification ratio to 100 is the adjusted verification ratio. The target false alarm rate is obtained by dividing the historical false alarm rate by the historical detection rate, and the target detection rate is obtained by subtracting the target false alarm rate from the total. Selecting a sampling time window, obtaining a correct detection rate and a false alarm rate within the sampling time window, and if the correct detection rate is less than the target detection rate, updating the specific value of the adjusted certificate ratio primer by multiplying the ratio obtained by dividing the historical correct detection rate by the target detection rate; Obtaining a road speed limit for the road section to be detected, and obtaining an arithmetic average of vehicle flow speeds of the road section to be detected over a past period of time, which may be a plurality of different sampling time windows in the past, as a historical free flow speed; obtaining a minimum value between the road speed limit and the historical free flow speed, and combining the minimum value with the updated adjustment ratio to generate a driving speed threshold; The angle mutation certificate ratio component is obtained by adding the mean of the statistical steering angle change to the historical detection certificate ratio and the angle mutation benchmark, and specifically includes: The maximum value in the rotation angle difference sequence within the sampling time window is the rotation angle difference peak value; The ratio of the turn sequence standard deviation to the turn sequence mean is calculated as the sequence turn coordinate, the turn mutation benchmark is obtained by multiplying the sequence turn coordinate with the turn difference peak value, and the turn mutation ratio component is obtained by combining the turn sequence mean, the historical detection certificate ratio, and the turn mutation benchmark; The historical detection certificate ratio is combined with the corner mutation benchmark in a multiplicative manner.

2. The method according to claim 1, characterized in that The analysis system adopts a real-time detection network based on deep learning and a multi-target association tracking algorithm to realize the positioning and recognition of the target in each frame and the real-time calculation and annotation of its speed and angle changes.

3. The method according to claim 1, characterized in that The stay frame threshold is dynamically calculated based on the shortest congestion determination time and the actual sampling frame rate of the video stream and rounded up; wherein, the average flow rate is calculated based on the speed of each vehicle in the video stream data collected by the roadside camera, and vehicles with a speed lower than 10% of the average flow rate in the video stream data are judged as congested vehicles, and the average value of the stay time of congested vehicles is used as the shortest congestion determination time, and frames in the video stream data with a speed lower than 10% of the average flow rate are regarded as low-speed frames, and the number of low-speed frames is used as the number of low-speed frames.

4. The method according to claim 3, characterized in that The accident judgment time threshold is obtained by adding the shortest congestion judgment time and the accident delay compensation time, wherein the accident delay compensation time is the average value of the time from the occurrence of stopping to resuming movement of each vehicle in the video stream collected by the roadside camera.

5. The method according to claim 3, characterized in that: The restricted area stay time threshold is calculated based on the ratio of the physical length of the restricted area to the real-time average driving speed of the road section, wherein the physical length of the restricted area is the length obtained by proportionally scaling or actual measurement in physical space based on the prohibited area set in the video stream.

6. The method according to claim 1, characterized in that It also includes, when the video stream collected by the roadside camera is connected to the analysis system through the edge gateway, real-time monitoring of the average bit rate and network packet loss rate of the video stream is carried out to obtain the real-time packet loss rate, and the video quality abnormality is marked as a special event and uploaded through the acceptable bit rate certificate ratio lower limit and the packet loss rate certificate ratio upper limit; wherein, the positive alarm rate of multiple different sampling time windows is continuously monitored and the average of the positive alarm rate is obtained, the acceptable bit rate certificate ratio lower limit is automatically taken as the product of the average bit rate and the average of the positive alarm rate based on historical statistics, the packet loss rate certificate ratio upper limit is the product of the average packet loss rate and the historical detection certificate ratio, and the average packet loss rate is the arithmetic mean of the real-time packet loss rate.

7. Highway video cloud network monitoring event analysis system, characterized by: The highway video cloud network monitoring event analysis system runs on any computing device such as a desktop computer, a laptop computer or a cloud data center. The computing device includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps in the highway video cloud network monitoring event analysis method as described in any one of claims 1 to 6 are implemented.

8. An electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

Images