Security and protection video monitoring method based on mixed feature recognition

Abstract

The invention relates to the technical field of security and protection monitoring, in particular to a security and protection video monitoring method based on mixed feature recognition. The method comprises the following steps: firstly, constructing a historical traffic congestion data set and a real-time traffic congestion data set, wherein the data sets comprise vehicle appearance features, vehicle motion states, pedestrian states and environment state multi-dimensional features; and constructing a first traffic congestion probability prediction model and a second traffic congestion probability prediction model according to historical data. Secondly, analyzing real-time vehicle movement and pedestrian state data to obtain a first prediction probability of traffic congestion; and analyzing the influence of the vehicle appearance characteristics and the environmental state data on the vehicle speed, and further influencing a second prediction probability of traffic congestion. And finally, the comprehensive prediction probability of traffic congestion is calculated through multi-feature fusion, and light and sound are used for early warning based on the probability to prompt that the traffic congestion is about to occur or possibly occurs, so that the prediction precision of the traffic congestion is effectively improved, and intelligent support is provided for traffic management and dispersion.

Description

Technical Field

[0001] The present invention relates to the technical field of security monitoring, and in particular to a security video monitoring method based on hybrid feature recognition. Background Art

[0002] Traffic congestion has become a common management problem faced by large cities. The surge in traffic flow during peak hours makes it difficult for traffic management departments to quickly and effectively respond to sudden traffic congestion, resulting in reduced traffic efficiency, increased vehicle delays, and even traffic accidents. In the prior art, traditional traffic monitoring systems mainly rely on fixed sensors, traffic lights, and manual observation to monitor traffic flow. However, these methods have significant limitations when dealing with complex and changing traffic conditions.

[0003] Traditional monitoring systems are not good at real-time and dynamic monitoring of traffic flow, and are usually unable to provide real-time warnings. Intervention measures are often taken only after traffic congestion has formed, resulting in delayed responses. The existing system's monitoring methods are relatively simple, usually only evaluating traffic conditions by counting the number of vehicles or monitoring vehicle speeds, ignoring the comprehensive impact of vehicle shape, driving trajectory and environmental factors on traffic flow. This single-dimensional analysis method makes it difficult to accurately capture early signals of congestion, resulting in one-sided monitoring information.

[0004] Therefore, a security video surveillance method based on hybrid feature recognition is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a security video monitoring method based on hybrid feature recognition. The present invention relates to the field of security monitoring technology, and specifically to a security video monitoring method based on hybrid feature recognition. First, a historical traffic congestion data set and a real-time traffic congestion data set are constructed, and the data set includes multi-dimensional features of vehicle appearance features, vehicle motion status, pedestrian status and environmental status; and a first and a second traffic congestion probability prediction model are constructed according to historical data. By analyzing the real-time vehicle motion and pedestrian status data, a first prediction probability of traffic congestion is obtained; at the same time, the influence of vehicle appearance features and environmental status data on vehicle speed is particularly analyzed, thereby affecting the second prediction probability of traffic congestion. Finally, the comprehensive prediction probability of traffic congestion is calculated by multi-feature fusion, and warnings are issued through lights and sounds to indicate impending or possible traffic congestion. Through spatiotemporal correlation analysis, machine learning models and multi-feature fusion, the prediction accuracy of traffic congestion is effectively improved, providing intelligent support for traffic management and diversion.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A security video monitoring method based on hybrid feature recognition, comprising:

[0008] Collect historical video data and real-time video data of historical traffic flow from surveillance cameras on sections of roads where traffic congestion often occurs;

[0009] Acquire a historical traffic congestion data set when traffic congestion occurs based on the historical video data; the historical traffic congestion data set includes historical vehicle appearance feature data, historical vehicle motion state data, historical pedestrian state data and historical environmental state data;

[0010] Acquire a real-time traffic congestion data set according to the real-time video data; the real-time traffic congestion data set includes real-time vehicle appearance feature data, real-time vehicle motion state data, real-time pedestrian state data and real-time environment state data;

[0011] Constructing a first traffic congestion probability prediction model based on the historical vehicle motion state data and the historical pedestrian state data;

[0012] Identify the real-time vehicle motion state data and the real-time pedestrian state data by using the first traffic congestion probability prediction model to obtain a first traffic congestion prediction probability;

[0013] constructing a second traffic congestion probability prediction model based on the historical vehicle appearance feature data and the historical environmental state data;

[0014] Analyzing the real-time vehicle appearance feature data and the real-time environmental status data by using the second traffic congestion probability prediction model to obtain a second traffic congestion prediction probability;

[0015] According to the first predicted probability of traffic congestion and the second predicted probability of traffic congestion, a comprehensive predicted probability of traffic congestion is calculated; and an early warning is issued according to a threshold value of the comprehensive predicted probability of traffic congestion.

[0016] Preferably, the historical vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color;

[0017] The historical vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior;

[0018] The historical pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density;

[0019] The historical environmental status data includes weather conditions, light intensity, and signal light status;

[0020] The real-time vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color;

[0021] The real-time vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior;

[0022] The real-time pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density;

[0023] The real-time environmental status data includes weather conditions, light intensity, and signal light status.

[0024] Preferably, the historical traffic congestion data set is preprocessed to remove abnormal data and missing values, and the data is standardized;

[0025] Extracting features from the historical traffic congestion data set to obtain a historical vehicle appearance feature vector group, a historical vehicle motion state vector group, a historical pedestrian state vector group, and a historical environment state vector group;

[0026] Comprehensively analyzing the historical vehicle motion state data and the historical pedestrian state data to obtain vehicle-pedestrian interaction characteristics;

[0027] A comprehensive analysis is performed on the historical vehicle appearance feature vector group and the historical environment state vector group to obtain vehicle-pedestrian interaction features.

[0028] Preferably, the first traffic congestion probability prediction model includes a spatiotemporal correlation evaluation unit, a state evaluation unit and a first congestion fusion unit;

[0029] The spatiotemporal correlation evaluation unit analyzes the interaction between vehicles and pedestrians in the spatial and temporal dimensions through the real-time vehicle motion state vector group and the real-time pedestrian state vector group, detects the distance between pedestrians and vehicles, and obtains the first congestion probability of the driving direction;

[0030] The state evaluation unit analyzes the density of vehicles and pedestrians through the real-time vehicle motion state vector group and the real-time pedestrian state vector group to obtain the second congestion probability of the driving direction;

[0031] The first congestion fusion unit determines a first predicted probability of traffic congestion according to the first congestion probability of the driving direction and the second congestion probability of the driving direction.

[0032] Preferably, the specific calculation formula of the first predicted probability of traffic congestion is:

[0033]

[0034] Among them, P 1 is the first predicted probability of traffic congestion, ω 1 is the weight for evaluating spatiotemporal correlation, α 1is the average distance weight between pedestrians and vehicles, d is the average distance between pedestrians and vehicles, α 2 is the speed difference weight between pedestrians and vehicles, v is the speed difference between pedestrians and vehicles, ω 2 is the state evaluation weight, β 1 is the vehicle density weight, ρ c is the vehicle density, β 2 is the pedestrian density weight, ρ p is the pedestrian density.

[0035] Preferably, the second traffic congestion probability prediction model is trained by a historical vehicle appearance feature vector group and a historical environment state vector group;

[0036] The step of training the second traffic congestion probability prediction model by using the historical vehicle appearance feature vector group and the historical environment state vector group comprises:

[0037] Dataset division: the historical vehicle appearance feature vector group and the historical environment state vector group are divided into a training set and a test set in a ratio of 7:3;

[0038] Training the model: inputting the training set into the second traffic congestion probability prediction model until the model training stop condition is met to obtain an initial second traffic congestion probability prediction model; the model training stop condition is that the loss of the second traffic congestion probability prediction model reaches convergence;

[0039] Testing model: inputting the test set into the second traffic congestion probability prediction model for testing, optimizing the second traffic congestion probability prediction model according to the test result, and obtaining the optimal second traffic congestion probability prediction model;

[0040] The second traffic congestion probability prediction model includes a preprocessing layer, a feature analysis layer and a traffic congestion second prediction probability output layer;

[0041] The preprocessing layer is used to perform preprocessing operations on the real-time vehicle appearance feature data and the real-time environment status data;

[0042] The feature analysis layer includes an LSTM network structure, which is used to analyze the probability of speed for traffic congestion on the preprocessed vector group;

[0043] The traffic congestion second prediction probability output layer is used to output the traffic congestion second prediction probability.

[0044] Preferably, the feature analysis layer analysis process is:

[0045] Based on the real-time vehicle appearance feature vector group and the real-time environment state vector group, the influence of the vehicle appearance feature on the speed and the influence of the environment state on the speed are obtained;

[0046] Comprehensively analyzing the real-time vehicle appearance feature vector group and the real-time environment state vector group to obtain the influence of the interaction feature on the speed;

[0047] Combining the influence of vehicle appearance features on speed, the influence of environmental conditions on speed and the influence of interaction features on speed, the probability of speed affecting traffic congestion is obtained.

[0048] Preferably, the specific calculation formula of the second predicted probability of traffic congestion is:

[0049]

[0050] Among them, P 2 is the second predicted probability of traffic congestion, λ 1 is the vehicle appearance feature weight, γ 1 is the vehicle size weight, S is the vehicle size information, γ 2 is the vehicle weight, W is the vehicle weight, γ 3 is the vehicle shape weight, O is the vehicle shape, λ 2 is the environmental state weight, κ 1 is the weather condition weight, E is the weather condition, κ 2 is the light intensity weight, L is the light intensity, κ 3 is the traffic light state weight, T is the traffic light state, λ 3 is the interaction feature weight, P con is the probability of speed versus congestion.

[0051] Preferably, according to the first prediction probability of traffic congestion and the second prediction probability of traffic congestion, a comprehensive prediction probability of traffic congestion is generated by multi-feature fusion; the formula for multi-feature fusion is:

[0052] P=ε 1 P 1 +ε 2 P 2 ;

[0053] Among them, P is the comprehensive prediction probability of traffic congestion, ε 1 is the first predicted probability weight of traffic congestion, P 1 is the first predicted probability of traffic congestion, ε 2 is the second predicted probability weight of traffic congestion, P 2 The second predicted probability for traffic congestion.

[0054] Preferably, an early warning is given according to the comprehensive predicted probability threshold of traffic congestion, and warnings are given in the form of lights and sounds according to the comprehensive predicted probability of traffic congestion, including: warning information is that congestion is about to occur, congestion may occur, and congestion will not occur, and red light, yellow light, and green light are used for warning respectively.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. The present invention realizes accurate prediction of traffic congestion by constructing a first traffic congestion probability prediction model. By introducing a spatiotemporal correlation evaluation unit and a state evaluation unit, the interaction between vehicles and pedestrians is deeply analyzed in spatial and temporal dimensions, especially the distance change between pedestrians and vehicles and between vehicles, to quantify the congestion risk in the driving direction. In addition, the state evaluation unit captures traffic mobility under different density conditions based on data analysis of vehicle and pedestrian density, and provides a more comprehensive assessment of congestion risk. The first congestion probability of the driving direction is combined with the second congestion probability through the first congestion fusion unit, and multi-source data is integrated to improve the accuracy and real-time performance of congestion warning.

[0057] 2. The present invention improves the accuracy of traffic congestion prediction by constructing a second traffic congestion probability prediction model. The feature analysis layer evaluates the impact of vehicle appearance features on vehicle speed by real-time analysis of vehicle appearance features and environmental conditions, and further improves the perception of dynamic changes in traffic by combining environmental conditions. By analyzing the interactive features of vehicle appearance features and environmental conditions, the impact of various factors on speed is quantified. This process combines the comprehensive effects of multi-dimensional features to calculate the probability of speed for traffic congestion, thereby enhancing the adaptability to dynamic changes in traffic flow.

[0058] 3. The present invention generates a comprehensive prediction probability of traffic congestion by fusing multiple features, thereby improving the accuracy and real-time performance of traffic congestion prediction. By fusing the first prediction probability of traffic congestion with the second prediction probability, the multi-dimensional data of vehicle motion status, vehicle appearance characteristics, pedestrian status, and environmental conditions are comprehensively considered, thereby improving the adaptability to complex traffic conditions. In addition, the early warning mechanism based on the comprehensive prediction probability of traffic congestion provides warning information of impending, possible, or non-occurring congestion through lights and sounds, thereby improving the intelligent level of traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic flow chart of a security video monitoring method based on hybrid feature recognition provided by an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of the structure of a first traffic congestion probability prediction model provided by an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of the structure of a second traffic congestion probability prediction model provided by an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of a second prediction probability flow chart of traffic congestion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] In order to more accurately predict traffic congestion, the present invention provides a security video monitoring method based on hybrid feature recognition. Figure 1 :

[0065] Collect historical video data and real-time video data of historical traffic flow from surveillance cameras on sections of roads where traffic congestion often occurs;

[0066] In this embodiment, when collecting historical traffic flow data and real-time video data of surveillance cameras on congested sections of roads, the cameras are installed in accordance with conventional installation methods, and are installed at major roads, intersections, and sections where congestion often occurs during peak hours. The collection of historical video data is mainly used to analyze the trend of traffic flow changes over a period of time, including the number of vehicles entering and exiting, vehicle speed, and queue length information, so as to identify traffic bottlenecks and high-frequency congestion periods. The collection of real-time video data is used to monitor the traffic conditions of the current section in real time, provide real-time flow information, facilitate traffic scheduling and management, and avoid further congestion.

[0067] Acquire a historical traffic congestion data set when traffic congestion occurs based on the historical video data; the historical traffic congestion data set includes historical vehicle appearance feature data, historical vehicle motion state data, historical pedestrian state data and historical environmental state data;

[0068] Acquire a real-time traffic congestion data set according to the real-time video data; the real-time traffic congestion data set includes real-time vehicle appearance feature data, real-time vehicle motion state data, real-time pedestrian state data and real-time environment state data;

[0069] Furthermore, the historical vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color;

[0070] The historical vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior;

[0071] The historical pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density;

[0072] The historical environmental status data includes weather conditions, light intensity, and signal light status;

[0073] The weather conditions include sunny days, rainy days, snowy days, thunderstorms and extreme weather;

[0074] The signal light status includes the waiting time and whether the signal light is damaged;

[0075] The real-time vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color;

[0076] The vehicle size includes the length, width and height of the vehicle; the vehicle shape is the streamline shape of the vehicle;

[0077] The real-time vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior;

[0078] The real-time pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density;

[0079] The real-time environmental status data includes weather conditions, light intensity, and signal light status.

[0080] In this embodiment, by collecting and analyzing historical and real-time vehicle and pedestrian status data, combined with environmental status information, comprehensive monitoring and accurate analysis of traffic flow is achieved. The characteristics of different types of vehicles and their potential impact on traffic are identified through vehicle appearance feature data; dynamic information about traffic fluidity is provided based on vehicle motion status data, providing a reliable basis for traffic flow assessment. Pedestrian data including motion trajectory and density helps predict pedestrian behavior and its interference with traffic. Finally, environmental status data further optimizes the traffic flow prediction model. By combining historical data with real-time monitoring data, dynamic control and prediction functions are achieved, which improves the scientificity and accuracy of traffic management.

[0081] Further, the historical traffic congestion data set and the real-time traffic congestion data set are preprocessed to remove abnormal data and missing values, and the data are standardized;

[0082] Performing feature extraction on the historical traffic congestion dataset and the real-time traffic congestion dataset to obtain a historical vehicle appearance feature vector group, a historical vehicle motion state vector group, a historical pedestrian state vector group, a historical environment state vector group, a real-time vehicle appearance feature vector group, a real-time vehicle motion state vector group, a real-time pedestrian state vector group, and a real-time environment state vector group;

[0083] Comprehensively analyzing the historical vehicle motion state data and the historical pedestrian state data to obtain vehicle-pedestrian interaction characteristics;

[0084] A comprehensive analysis is performed on the historical vehicle appearance feature vector group and the historical environment state vector group to obtain vehicle-pedestrian interaction features.

[0085] In this embodiment, the abnormal data in the historical traffic congestion data set include extreme speeds, abnormal vehicle sizes, and pedestrian density beyond a reasonable range; these data may be caused by sensor failure or data entry errors. Missing values ​​are filled by the mean filling method. After cleaning, the data needs to be standardized. Standardization Since different features have different dimensions, such as the dimensions of vehicle speed (km / h) and vehicle weight (kg), standardization normalizes different features to be within the range of [0,1].

[0086] During the feature extraction process, the information in the preprocessed historical traffic congestion data set is used to extract historical vehicle appearance features, historical vehicle motion state features, historical pedestrian state features, and historical environmental state features. The vehicle appearance feature vector group includes the size, weight, shape, and color of the vehicle. By analyzing these features, different types of vehicles can be distinguished, such as small cars and trucks. The vehicle motion state vector group includes speed, acceleration, driving trajectory, driving time, and braking behavior, which helps analyze the dynamic performance of vehicles in traffic flow. The pedestrian state vector group involves the number, motion trajectory, speed, and density of pedestrians, reflecting the characteristics of pedestrian flow. The environmental state vector group covers weather conditions, light intensity, and signal light status, describing the impact of the external environment on traffic conditions. By extracting the feature vector group, the vehicle motion state can be combined with the pedestrian state to analyze the interaction between vehicles and pedestrians in traffic flow. This method not only improves the accuracy of feature extraction, but also provides an important basis for traffic flow prediction and congestion warning, and optimizes the performance of traffic management and decision support systems.

[0087] Based on the historical vehicle motion state data and the historical pedestrian state data, a first traffic congestion probability prediction model is constructed, referring to Figure 2 ;

[0088] Further, the first traffic congestion probability prediction model includes a spatiotemporal correlation evaluation unit, a state evaluation unit and a first congestion fusion unit;

[0089] The spatiotemporal correlation evaluation unit analyzes the interaction between vehicles and pedestrians in the spatial and temporal dimensions, detects the distance between pedestrians and vehicles, and obtains the first congestion probability in the driving direction; by evaluating the relative distance between vehicles and pedestrians, the moving direction, speed and position changes of pedestrians and vehicles are considered. In the driving direction, when the distance between vehicles and pedestrians is too close or they cross each other, the spatiotemporal correlation evaluation unit will output the first congestion probability in the driving direction.

[0090] The state evaluation unit obtains the second congestion probability of the driving direction by analyzing the density of vehicles and pedestrians; vehicle density refers to the number of vehicles in a unit area, and pedestrian density refers to the number of pedestrians in a unit area. High-density areas often lead to poor traffic flow. Therefore, the unit outputs the second congestion probability of the driving direction by calculating the density of vehicles and pedestrians.

[0091] The first congestion fusion unit fuses the first congestion probability calculated by the spatiotemporal correlation evaluation unit and the second congestion probability calculated by the state evaluation unit, and obtains a first predicted probability of traffic congestion by comprehensively considering factors such as the distance between vehicles and pedestrians, the driving direction, and the density. The predicted probability can provide effective traffic condition warning for the traffic management department, and help to achieve early detection and response to congestion.

[0092] Furthermore, the specific calculation formula of the first predicted probability of traffic congestion is:

[0093]

[0094] Among them, P 1 is the first predicted probability of traffic congestion, ω 1 is the weight for evaluating spatiotemporal correlation, α 1 is the average distance weight between pedestrians and vehicles, d is the average distance between pedestrians and vehicles, α 2 is the speed difference weight between pedestrians and vehicles, v is the speed difference between pedestrians and vehicles, ω 2 is the state evaluation weight, β 1 is the vehicle density weight, ρ c is the vehicle density, β 2 is the pedestrian density weight, ρ p is the pedestrian density.

[0095] In this embodiment, by introducing a specific calculation formula for the first prediction probability of traffic congestion, the possibility of traffic congestion is quantified and evaluated. The various weights and variables in the formula take into account the spatiotemporal correlation factors and the state evaluation factors, respectively, so that the model has high adaptability under different traffic conditions. By integrating the key parameters into a unified calculation formula, not only can the weights be dynamically adjusted according to real-time data, where the weights are obtained through training based on the historical vehicle motion state characteristics and historical pedestrian state characteristics, but also adaptive optimization can be performed for specific problems in different traffic environments, such as high-density pedestrian flow or fast-moving vehicles, which significantly improves the accuracy and response speed of traffic congestion prediction.

[0096] Based on the historical vehicle appearance feature data and the historical environmental status data, a second traffic congestion probability prediction model is constructed, referring to Figure 3 ;

[0097] Furthermore, the second traffic congestion probability prediction model is trained by a historical vehicle appearance feature vector group and a historical environment state vector group;

[0098] The step of training the second traffic congestion probability prediction model by using the historical vehicle appearance feature vector group and the historical environment state vector group comprises:

[0099] Dataset division: the historical vehicle appearance feature vector group and the historical environment state vector group are divided into a training set and a test set in a ratio of 7:3;

[0100] Training the model: inputting the training set into the second traffic congestion probability prediction model until the model training stop condition is met to obtain an initial second traffic congestion probability prediction model; the model training stop condition is that the loss of the second traffic congestion probability prediction model reaches convergence;

[0101] Testing model: inputting the test set into the second traffic congestion probability prediction model for testing, optimizing the second traffic congestion probability prediction model according to the test result, and obtaining the optimal second traffic congestion probability prediction model;

[0102] The second traffic congestion probability prediction model includes a preprocessing layer, a feature analysis layer and a traffic congestion second prediction probability output layer;

[0103] The preprocessing layer is used to perform preprocessing operations on the real-time vehicle appearance feature data and the real-time environment status data;

[0104] The feature analysis layer includes an LSTM network structure, which is used to analyze the probability of speed for traffic congestion on the preprocessed vector group;

[0105] The traffic congestion second prediction probability output layer is used to output the traffic congestion second prediction probability.

[0106] In this embodiment, by constructing a second traffic congestion probability prediction model, the historical vehicle appearance feature vector group and the historical environmental state vector group are divided into a training set and a test set in a ratio of 7:3, which effectively avoids the overfitting problem of the model. During the model training process, the LSTM network structure is used for feature analysis, so that the model can process complex time series data, thereby capturing the role of vehicle appearance features and environmental status in traffic congestion. The trained model is further optimized through the test set to ensure that the final output of the second traffic congestion prediction probability has a high accuracy and provides a more accurate traffic congestion warning.

[0107] Furthermore, the feature analysis layer analysis process is:

[0108] Based on the real-time vehicle appearance feature vector group and the real-time environment state vector group, the influence of the vehicle appearance feature on the speed and the influence of the environment state on the speed are obtained; the influence of the vehicle appearance feature on the speed refers to factors such as the size, weight, and shape of the vehicle. For example, larger vehicles such as trucks are usually slower, while small vehicles may be more maneuverable under the same conditions and travel at a relatively faster speed.

[0109] The real-time vehicle appearance feature data and the real-time environmental status data are comprehensively analyzed to obtain the impact of the interactive features on the speed; the impact of the environmental status on the speed is analyzed by analyzing factors such as real-time weather conditions, light intensity, and signal light status. For example, in bad weather such as rain, snow, strong winds, or insufficient light, the speed of the vehicle is usually restricted. Changes in traffic lights will also affect the mobility and speed of vehicles, thereby increasing the probability of traffic congestion.

[0110] Combining the influence of vehicle appearance features on speed, the influence of environmental conditions on speed and the influence of interaction features on speed, the probability of speed affecting traffic congestion is obtained.

[0111] In this embodiment, based on real-time vehicle appearance feature data and real-time environmental status data, the impact of vehicle appearance and environmental factors on speed are evaluated respectively, such as the restrictions on its driving speed imposed by vehicle size and weight, and the adjustment of speed imposed by environmental factors such as weather and light. The vehicle appearance features and environmental status are comprehensively analyzed, and the nonlinear impact of the interaction on speed is further analyzed. Combined with the comprehensive effects of various aspects on speed, the impact of speed changes on traffic flow is dynamically predicted. When the speed is reduced, the system can timely warn of the potential occurrence of traffic congestion, thereby providing a scientific basis for traffic management and helping to achieve congestion relief and optimal scheduling of traffic flow.

[0112] Furthermore, the specific calculation formula of the second predicted probability of traffic congestion is:

[0113]

[0114] Among them, P 2 is the second predicted probability of traffic congestion, λ 1 is the vehicle appearance feature weight, γ 1 is the vehicle size weight, S is the vehicle size information, γ 2 is the vehicle weight, W is the vehicle weight, γ 3 is the vehicle shape weight, O is the vehicle shape, λ 2 is the environmental state weight, κ 1 is the weather condition weight, E is the weather condition, κ 2 is the light intensity weight, L is the light intensity, κ 3 is the traffic light state weight, T is the traffic light state, λ 3 is the interaction feature weight, P con is the probability of speed versus congestion.

[0115] By introducing the specific calculation formula for the second prediction probability of traffic congestion, different factors are weighted by vehicle appearance feature weights and environmental state weights, and the weights are obtained through training of historical vehicle appearance features and historical environmental state features. Appearance features such as vehicle size, weight and shape, as well as weather conditions, light intensity and signal light status environmental conditions. Each weight is trained to obtain the optimal value through model training and can adapt to different traffic scenarios and conditions. Through the calculation method of each feature, not only the accuracy of traffic congestion prediction is improved, but also dynamic adjustment can be made according to real-time data, thereby effectively reducing the uncertainty in traffic management, providing strong support for preventing traffic congestion, and helping to optimize traffic flow and improve road utilization.

[0116] Furthermore, according to the first prediction probability of traffic congestion and the second prediction probability of traffic congestion, a comprehensive prediction probability of traffic congestion is generated by multi-feature fusion; the formula for multi-feature fusion is:

[0117] P=ε 1 P 1 +ε 2 P 2 ;

[0118] Among them, P is the comprehensive prediction probability of traffic congestion, ε 1 is the first predicted probability weight of traffic congestion, P 1 is the first predicted probability of traffic congestion, ε 2 is the second predicted probability weight of traffic congestion, P 2 The second predicted probability for traffic congestion.

[0119] The present invention introduces a multi-feature fusion formula of the first prediction probability and the second prediction probability of traffic congestion to generate a comprehensive prediction probability of traffic congestion. The fusion formula integrates multi-dimensional information by weighted summation of the two types of prediction probabilities. This method can not only improve the accuracy of congestion prediction, but also make dynamic adjustments based on real-time traffic data, thereby alleviating traffic congestion, optimizing road utilization efficiency, and improving traffic circulation.

[0120] Furthermore, an early warning is given according to the comprehensive predicted probability threshold of traffic congestion, and warnings are given in the form of lights and sounds according to the comprehensive predicted probability of traffic congestion, including: warning information is that congestion is about to occur, congestion may occur, and congestion will not occur, and red light, yellow light, and green light are used for warning respectively.

[0121] This early warning mechanism effectively enhances the initiative and accuracy of traffic management, allowing managers to take measures in advance to alleviate potential traffic pressure. In addition, the combination of light and sound ensures the visualization and efficient communication of warning information, reduces the management difficulty caused by information lag, and thus improves the efficiency and safety of road traffic circulation.

[0122] The present invention combines historical and real-time data to predict and warn traffic congestion, thereby improving traffic management efficiency. By collecting multidimensional data such as vehicle appearance features, vehicle movement status, pedestrian status and environmental status in historical videos and real-time videos, a first traffic congestion probability prediction model and a second traffic congestion probability prediction model are constructed. The multi-feature fusion of the two models generates a comprehensive prediction probability of traffic congestion, so that the potential congestion risk can be assessed more accurately. The model introduces spatiotemporal correlation evaluation, state evaluation and multi-level weight calculation methods, which effectively improves the accuracy and robustness of congestion prediction. At the same time, the LSTM network is used for feature analysis of historical data, which can capture complex time series changes, thereby improving the ability to predict future traffic conditions. Based on the threshold warning mechanism of the comprehensive prediction probability of traffic congestion, combined with the light and sound warning method, traffic management personnel can obtain congestion warnings in time, take countermeasures, and avoid congestion escalation. This method provides an intelligent and dynamic management solution for complex urban traffic scenes, improving traffic circulation efficiency.

[0123] Embodiment 2

[0124] This embodiment includes all the contents of the first embodiment, and provides a security video monitoring method based on hybrid feature recognition.

[0125] The second traffic congestion probability prediction model is trained by a historical vehicle appearance feature vector group and a historical environment state vector group;

[0126] The second traffic congestion probability prediction model includes a preprocessing layer, a feature analysis layer and a traffic congestion second prediction probability output layer;

[0127] The preprocessing layer is used to perform preprocessing operations on the real-time vehicle appearance feature data and the real-time environment status data;

[0128] The feature analysis layer includes an LSTM network structure, which is used to analyze the probability of speed for traffic congestion on the preprocessed vector group;

[0129] The traffic congestion second prediction probability output layer is used to output the traffic congestion second prediction probability.

[0130] Furthermore, the feature analysis layer analysis process is to combine Figure 4 :

[0131] Based on the real-time vehicle appearance feature vector group and the real-time environment state vector group, the influence of the vehicle appearance feature on the speed and the influence of the environment state on the speed are obtained;

[0132] Comprehensively analyzing the real-time vehicle appearance feature data and the real-time environmental status data to obtain the influence of the interaction feature on the speed;

[0133] Combining the influence of vehicle appearance features on speed, the influence of environmental conditions on speed and the influence of interactive features on speed, the probability of speed for traffic congestion is obtained. The influence of the interactive features on speed can be specifically expressed as:

[0134]

[0135] Among them, v is the vehicle speed under the interactive feature, v 0 is the vehicle speed under normal weather conditions, α 01 is the weight of the appearance feature on wind resistance, C d is the drag coefficient, A is the frontal area of ​​the vehicle, W is the vehicle weight, α 02 is the environmental impact weight, β 0 is the intensity of wind speed, and F is the additional influence of crosswind environmental factors.

[0136] The probability of speed for traffic congestion is specifically expressed as:

[0137]

[0138] Among them, P con is the probability of speed to congestion, v is the vehicle speed under the interaction feature, and v 0 is the vehicle speed under normal weather conditions, and σ is the standard deviation of traffic speed, which is used to reflect the fluctuation range of vehicle speed.

[0139] When v ≈ v 0When the vehicle is traveling close to the ideal speed and the traffic is smooth, P con Close to zero;

[0140] When v is less than v 0 When the vehicle speed drops significantly, the traffic density increases, and P con Rapid increase;

[0141] When v→0, the vehicle speed is close to zero, which means that the traffic is almost stopped. con Close to 1, it means complete congestion.

[0142] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A security video monitoring method based on hybrid feature recognition, characterized in that: include: Collect historical video data and real-time video data to monitor historical traffic flow on sections where traffic congestion often occurs; Acquire a historical traffic congestion data set when traffic congestion occurs based on the historical video data; the historical traffic congestion data set includes historical vehicle appearance feature data, historical vehicle motion state data, historical pedestrian state data and historical environmental state data; Acquire a real-time traffic congestion data set according to the real-time video data; the real-time traffic congestion data set includes real-time vehicle appearance feature data, real-time vehicle motion state data, real-time pedestrian state data and real-time environment state data; Constructing a first traffic congestion probability prediction model based on the historical vehicle motion state data and the historical pedestrian state data; Identify the real-time vehicle motion state data and the real-time pedestrian state data by using the first traffic congestion probability prediction model to obtain a first traffic congestion prediction probability; constructing a second traffic congestion probability prediction model based on the historical vehicle appearance feature data and the historical environmental state data; Analyzing the real-time vehicle appearance feature data and the real-time environmental status data by using the second traffic congestion probability prediction model to obtain a second traffic congestion prediction probability; According to the first predicted probability of traffic congestion and the second predicted probability of traffic congestion, a comprehensive predicted probability of traffic congestion is calculated; and an early warning is issued according to a threshold value of the comprehensive predicted probability of traffic congestion.

2. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: The historical vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color; The historical vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior; The historical pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density; The historical environmental status data includes weather conditions, light intensity, and signal light status; The real-time vehicle appearance feature data includes vehicle size, vehicle weight, vehicle shape and vehicle color; The real-time vehicle motion state data includes vehicle speed, acceleration, driving trajectory, driving time and braking behavior; The real-time pedestrian status data includes the number of pedestrians, pedestrian movement trajectories, pedestrian speeds, and pedestrian density; The real-time environmental status data includes weather conditions, light intensity, and signal light status.

3. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: Preprocessing the historical traffic congestion data set, removing abnormal data and missing values, and standardizing the data; Extracting features from the historical traffic congestion data set to obtain a historical vehicle appearance feature vector group, a historical vehicle motion state vector group, a historical pedestrian state vector group, and a historical environment state vector group; Comprehensively analyzing the historical vehicle motion state data and the historical pedestrian state data to obtain vehicle-pedestrian interaction characteristics; A comprehensive analysis is performed on the historical vehicle appearance feature vector group and the historical environment state vector group to obtain vehicle-pedestrian interaction features.

4. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: The first traffic congestion probability prediction model includes a spatiotemporal correlation evaluation unit, a state evaluation unit and a first congestion fusion unit; The spatiotemporal correlation evaluation unit analyzes the interaction between vehicles and pedestrians in the spatial and temporal dimensions through the real-time vehicle motion state vector group and the real-time pedestrian state vector group, detects the distance between pedestrians and vehicles, and obtains the first congestion probability of the driving direction; The state evaluation unit analyzes the density of vehicles and pedestrians through the real-time vehicle motion state vector group and the real-time pedestrian state vector group to obtain the second congestion probability of the driving direction; The first congestion fusion unit determines a first predicted probability of traffic congestion according to the first congestion probability of the driving direction and the second congestion probability of the driving direction.

5. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: The specific calculation formula of the first predicted probability of traffic congestion is: Among them, P1 is the first predicted probability of traffic congestion, ω1 is the spatiotemporal association evaluation weight, α1 is the average distance weight between pedestrians and vehicles, d is the average distance between pedestrians and vehicles, α2 is the speed difference weight between pedestrians and vehicles, v is the speed difference between pedestrians and vehicles, ω2 is the state evaluation weight, β1 is the vehicle density weight, ρ c is the vehicle density, β2 is the pedestrian density weight, ρ p is the pedestrian density.

6. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: The second traffic congestion probability prediction model is trained by a historical vehicle appearance feature vector group and a historical environment state vector group; The step of training the second traffic congestion probability prediction model by using the historical vehicle appearance feature vector group and the historical environment state vector group comprises: Dataset division: the historical vehicle appearance feature vector group and the historical environment state vector group are divided into a training set and a test set in a ratio of 7:3; Training the model: inputting the training set into the second traffic congestion probability prediction model until the model training stop condition is met to obtain an initial second traffic congestion probability prediction model; the model training stop condition is that the loss of the second traffic congestion probability prediction model reaches convergence; Testing model: inputting the test set into the second traffic congestion probability prediction model for testing, optimizing the second traffic congestion probability prediction model according to the test result, and obtaining the optimal second traffic congestion probability prediction model; The second traffic congestion probability prediction model includes a preprocessing layer, a feature analysis layer and a traffic congestion second prediction probability output layer; The preprocessing layer is used to perform preprocessing operations on the real-time vehicle appearance feature data and the real-time environmental status data; the feature analysis layer includes an LSTM network structure, which is used to analyze the probability of speed for traffic congestion on the preprocessed vector group; the traffic congestion second prediction probability output layer is used to output the traffic congestion second prediction probability.

7. The security video monitoring method based on hybrid feature recognition according to claim 6 is characterized in that: The feature analysis layer analysis process is as follows: Based on the real-time vehicle appearance feature vector group and the real-time environment state vector group, the influence of the vehicle appearance feature on the speed and the influence of the environment state on the speed are obtained; Comprehensively analyze the real-time vehicle appearance feature vector group and the real-time environment state vector group to obtain the impact of the interactive features on the speed; Combining the influence of vehicle appearance features on speed, the influence of environmental conditions on speed and the influence of interaction features on speed, the probability of speed affecting traffic congestion is obtained.

8. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: The specific calculation formula of the second predicted probability of traffic congestion is: Among them, P2 is the second prediction probability of traffic congestion, λ1 is the weight of vehicle appearance features, γ1 is the weight of vehicle size, S is the vehicle size information, γ2 is the weight of vehicle weight, W is the vehicle weight, γ3 is the weight of vehicle shape, O is the vehicle shape, λ2 is the weight of environmental state, κ1 is the weight of weather conditions, E is the weather conditions, κ2 is the weight of light intensity, L is the light intensity, κ3 is the weight of signal light state, T is the signal light state, λ3 is the weight of interaction features, P con is the speed versus congestion probability.

9. The security video monitoring method based on hybrid feature recognition according to claim 1, characterized in that: According to the first prediction probability of traffic congestion and the second prediction probability of traffic congestion, a comprehensive prediction probability of traffic congestion is generated by multi-feature fusion; the formula for multi-feature fusion is: P = ε1P1 + ε2P2; Among them, P is the comprehensive prediction probability of traffic congestion, ε1 is the first prediction probability weight of traffic congestion, P1 is the first prediction probability of traffic congestion, ε2 is the second prediction probability weight of traffic congestion, and P2 is the second prediction probability of traffic congestion.

10. The security video monitoring method based on hybrid feature recognition according to claim 1, characterized in that: Providing an early warning based on the comprehensive predicted probability threshold of traffic congestion, and providing warnings in the form of lights and sounds based on the comprehensive predicted probability of traffic congestion include: warning information is that congestion is about to occur, congestion may occur, and congestion will not occur, using red lights, yellow lights, and green lights for warnings respectively.

Images