Expressway video detection method and system based on large model

Abstract

The invention discloses an expressway video detection method and system based on a large model, relates to the technical field of intelligent traffic, and aims to solve the technical problems that the traditional detection technology is low in recognition accuracy, high in manual monitoring cost and easy to make mistakes in a complex scene, and cannot timely and accurately detect traffic abnormity. Comprising the following steps: S1, acquiring expressway traffic monitoring camera data to construct an expressway traffic monitoring video data set; s2, performing fine tuning based on the highway traffic monitoring video data set and the pre-training model to obtain a video classification model; s3, vehicles, roads, marks and the like in the traffic scene are detected, and the type of traffic abnormity is judged according to a detection result; s4, performing early warning processing based on the abnormal traffic scene; according to the method, the accuracy of traffic anomaly detection is greatly improved, the problem of low recognition accuracy in a complex traffic scene in the prior art is effectively solved, and a more reliable decision basis is provided for a traffic management department.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation, and more specifically, to a highway video detection method and system based on a large model. Background Art

[0002] Currently, in the field of highway traffic management, accurately and timely detecting traffic abnormal conditions is crucial for ensuring road safety and traffic flow. Most traditional highway video detection technologies are based on simple image recognition algorithms or manual monitoring, and there are many limitations. For example, the detection method based on simple image recognition algorithms has a low recognition accuracy for targets such as vehicles and road signs in complex traffic scenarios, such as bad weather (heavy rain, fog, etc.) and peak traffic flow periods, and is prone to misjudgment and missed judgment. Manual monitoring requires a large amount of human and time costs, and it is easy for humans to get tired after long-term observation of videos, which will also reduce the accuracy and timeliness of detection.

[0003] With the continuous increase in highway traffic flow and the increasing complexity of traffic scenarios, the existing detection technologies are difficult to meet the requirements of quickly and accurately detecting traffic abnormal conditions, and cannot provide timely and effective decision-making support for traffic management departments. As a result, when abnormal conditions such as traffic jams and traffic accidents occur, corresponding measures cannot be taken quickly, which in turn affects the overall operation efficiency and safety of highways. Therefore, there is an urgent need for a more advanced and efficient highway video detection method and system to solve these problems. In view of this, we propose a highway video detection method and system based on a large model. Summary of the Invention

[0004] The purpose of the present invention is to provide a highway video detection method and system based on a large model to solve the technical problems that traditional detection technologies have low recognition accuracy in complex scenarios, high manual monitoring costs and are prone to errors, and cannot detect traffic abnormalities quickly and accurately.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: A highway video detection method based on a large model, including the following steps:

[0006] S1: Obtain highway traffic monitoring camera data to construct a highway traffic monitoring video dataset;

[0007] S2: Fine-tune a video classification model based on the highway traffic monitoring video dataset and a pre-trained model. The specific steps are as follows:

[0008] S201: Randomly select N frames of images from the highway traffic monitoring video dataset to obtain N images;

[0009] S202: Use the pre-trained model to crop the size of each image to H×W, and input N images into the model to extract features;

[0010] S203: Input the features of each image into the classifier to obtain the probability of the type to which the image belongs;

[0011] S204: Use the cross-entropy loss function to perform backpropagation on the classifier to maximize the probability of the image on the video;

[0012] S205: Repeat steps S201 - S204 until the probability meets the preset threshold, and save the model parameters;

[0013] S3: Detect vehicles, roads, signs, etc. in the traffic scene, and judge the type of traffic anomaly according to the detection results;

[0014] S4: Perform early warning processing based on the abnormal traffic scene, and give a warning about the traffic anomaly based on the type of the abnormal traffic scene;

[0015] Among them, the type of the abnormal traffic scene includes the type of traffic anomaly, the time and location where the traffic anomaly exists, and the recommended information for eliminating the traffic anomaly.

[0016] Through constructing a highway traffic monitoring video dataset, fine-tuning the pre-trained model based on this to obtain a video classification model, and combining a series of operations such as object detection, modeling, and anomaly evaluation, the present invention can accurately detect targets such as vehicles, roads, signs, etc. in the traffic scene, and accurately judge the type of traffic anomaly. Compared with the traditional detection technology, the accuracy of traffic anomaly detection is greatly improved, effectively solving the problem of low recognition accuracy of the existing technology in complex traffic scenes, and providing a more reliable decision-making basis for traffic management departments.

[0017] Preferably, the specific method for extracting features in step S202 is as follows:

[0018] S202a: Form a sequence of N images;

[0019] S202b: Extract the inter-frame information thereof;

[0020] S202c: Extract the features of each frame image in the sequence;

[0021] S202d: After fully connecting the features of the inter-frame information, concatenate them with the features of each frame image to obtain the features of the image.

[0022] Preferably, for the detection of vehicles, roads, signs, etc. in the traffic scene in step S3 and judging the type of traffic anomaly according to the detection results, the specific steps are as follows:

[0023] S301: Obtain the target features in the traffic scenario, where the target features include the category, location, and spatial distribution of the target;

[0024] S302: Based on the target features, model the vehicles, roads, and signs in the traffic scenario to obtain a traffic scenario model;

[0025] Among them, the traffic scenario model includes traffic scenario features and traffic scenario probabilities;

[0026] The traffic scenario features are based on a target detection model. Input the images in the traffic scenario into the target detection model to obtain the category, location, and spatial distribution of the vehicles, roads, and signs in the traffic scenario;

[0027] The traffic scenario probabilities are determined based on a classifier. Input each image into the image classifier to obtain the probability of the scene to which the image belongs;

[0028] S303: Obtain traffic anomaly types and a traffic anomaly model based on prior knowledge;

[0029] Among them, the traffic anomaly model includes traffic anomaly types, anomaly features corresponding to the traffic anomaly types, and the space of the traffic anomaly types in the traffic scenario;

[0030] The determination of the anomaly features corresponding to the traffic anomaly types is achieved by inputting the features of the traffic anomaly types existing in the traffic scenario into an anomaly feature model to obtain the anomaly features;

[0031] The anomaly feature model includes two fully connected layers and an activation function layer. Input the features of the traffic anomaly types into the first fully connected layer for full connection to obtain second features, then input the second features into the second fully connected layer for full connection to obtain third features, and input the third features into the activation function layer for activation to obtain the anomaly features;

[0032] S304: Based on the traffic scenario model, detect each frame of the highway traffic monitoring video dataset to obtain the corresponding traffic scenario;

[0033] S305: Based on the traffic anomaly model, obtain the abnormal traffic scenarios existing in the traffic scenario.

[0034] Preferably, the specific method for obtaining the abnormal traffic scenarios existing in the traffic scenario in step S305 is as follows:

[0035] S305a: Construct an anomaly evaluation model for evaluating the probability of the existence of traffic anomaly types in each traffic area. The anomaly evaluation model is S = αP + β, where α is the weight, β is the deviation, and P is the probability of the existence of a certain traffic anomaly type in each traffic area;

[0036] S305b: Obtain the second weight for each traffic anomaly type, evaluate the probability of each traffic anomaly type existing in each traffic area based on the anomaly evaluation model, and obtain the score of each traffic anomaly type existing in each traffic area. Let the second weight of the j-th traffic anomaly type be w j , then the score of the j-th traffic anomaly type in the i-th traffic area is S ij = w j (αP ij + β);

[0037] S305c: Screen the traffic anomaly type corresponding to the highest score in the scores as the traffic anomaly type of the abnormal traffic scenario. That is, for the i-th traffic area, the traffic anomaly type of the abnormal traffic scenario is argmax j S ij ;

[0038] S305d: Determine that the current frame traffic scenario has an abnormal traffic scenario based on the fact that the score of the traffic anomaly type of the current frame traffic scenario in the traffic area is greater than the abnormal score threshold, and the abnormal score threshold is 0.01.

[0039] Preferably, the loss function in the model training process adopts the linear weighted sum of the cross-entropy loss function and the adversarial loss function;

[0040] The cross-entropy loss function is: In the formula, C is the number of categories, y i is the true label, y ∈ {0, 1}, y is the annotation in the true traffic scenario, and p i is the predicted probability;

[0041] The adversarial loss function is: In the formula, n is the number of traffic scenarios with abnormal traffic scenarios in the model training process, represents the probability of predicting that there is an abnormal traffic scenario in the traffic scenario.

[0042] Preferably, the method for determining the traffic anomaly type in the traffic scenario is: divide the traffic scenario into grid-like segments to obtain multiple traffic areas, and determine the probability of each traffic anomaly type existing in each traffic area based on the abnormal features and traffic scenario features corresponding to each traffic anomaly type.

[0043] Preferably, the method for determining the traffic scenario model in step S302 is:

[0044] S302a: Determine the score of the abnormal traffic scenario existing in the traffic scenario based on the scores of the abnormal traffic scenarios existing in each traffic area;

[0045] Among them, the score of the abnormal traffic scenario in the traffic scenario is: the score of the traffic anomaly type is obtained by traversing and adding the scores of each traffic anomaly type existing in each traffic area.

[0046] Let the total number of traffic areas be X, the probability of a certain traffic anomaly type existing in each traffic area be P, and the score of the abnormal traffic scenario in the traffic scenario be: In the formula, M is the number of traffic anomaly types.

[0047] S302b: Determine whether the score of the abnormal traffic scenario in the traffic scenario exceeds the preset score threshold. If so, the traffic scenario is an abnormal traffic scenario.

[0048] A traffic anomaly detection and alarm system based on a large model includes a data acquisition module, a data input module, a data output module, and a data processing module. The data acquisition module is connected to the data input module, the data input module is connected to the data processing module, and the data processing module is connected to the data output module;

[0049] The data acquisition module is used to obtain highway traffic monitoring camera data and save it to the database.

[0050] The data input module is used to load the traffic monitoring camera data saved by the data acquisition module into the large model, and based on the large model, predict and process the traffic monitoring camera data to obtain the detection result.

[0051] Among them, the large model includes a video detection model, a traffic scenario model, a traffic anomaly model, and an anomaly evaluation model;

[0052] The video detection model is used to detect the traffic monitoring camera data to obtain the traffic scenario.

[0053] The traffic scenario model is used to determine the traffic scenario.

[0054] The traffic anomaly model is used to analyze whether there is an anomaly in the traffic scenario and what kind of anomaly exists.

[0055] The anomaly evaluation model is used to judge whether there is an anomaly in each traffic area in the traffic scenario and the score of the traffic area with an anomaly, and determine whether there is an anomaly in the traffic area based on the score.

[0056] The data output module is used to visualize the detection result and push it to the database and traffic personnel.

[0057] Among them, the database is used to store the traffic monitoring camera data collected by the data acquisition module and the detection result obtained by the data output module, and the traffic personnel are traffic police.

[0058] The data processing module is used to screen the detection information and traffic monitoring camera data;

[0059] Among them, the detection information includes the traffic monitoring camera number, traffic scene and detection time.

[0060] Preferably, the screening of the traffic monitoring camera data is carried out in the following way: based on the score of the abnormal traffic scene in the traffic scene, it is determined whether the traffic monitoring camera data is an abnormal traffic scene.

[0061] Preferably, the screening of the detection information and traffic monitoring camera data is specifically carried out in the following way:

[0062] Based on the detection result of the traffic scene corresponding to the traffic monitoring camera data, it is judged whether there is an abnormal traffic scene;

[0063] Among them, the traffic scene model is pre-constructed and constructed based on prior knowledge;

[0064] If there is an abnormal traffic scene, the traffic scene, detection result and traffic monitoring camera data are screened out and pushed to the traffic personnel;

[0065] The content of visualizing the detection result is to visualize the abnormal situation of the entire traffic scene, including traffic abnormal features, traffic abnormal types, duration of traffic abnormality, and location;

[0066] The specific content of pushing to the database and traffic personnel is: pushing the 10 seconds before the abnormal judgment, the 10 seconds during the abnormality, and the 10 seconds after the abnormal judgment ends to the database and traffic personnel;

[0067] When traffic personnel analyze the abnormal traffic monitoring camera data in the database, if it does not constitute a traffic abnormality, the traffic personnel can return that the traffic monitoring camera data is normal;

[0068] If it constitutes a traffic abnormality, the traffic personnel can mark the traffic monitoring camera data as abnormal.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] 1. By constructing a highway traffic monitoring video dataset, fine-tuning the pre-trained model based on this to obtain a video classification model, and combining a series of operations such as object detection, modeling, and anomaly evaluation, the present invention can accurately detect targets such as vehicles, roads, and signs in the traffic scene, and accurately judge the types of traffic anomalies. Compared with traditional detection technologies, the accuracy of traffic anomaly detection is greatly improved, effectively solving the problem of low recognition accuracy of the prior art in complex traffic scenes, and providing a more reliable decision-making basis for traffic management departments.

[0071] 2. During the detection process of the present invention, the linear weighted sum of the cross-entropy loss function and the adversarial loss function is used as the loss function for model training. This approach enables the model to better learn the features of normal and abnormal traffic scenarios during training, further enhancing the adaptability and detection accuracy of the model to complex traffic scenarios, thus more accurately identifying traffic anomalies, further solving the problem of improving the detection accuracy in the main beneficial effects, and reducing the occurrence of misjudgment and missed judgment situations.

[0072] 3. The traffic anomaly detection and alarm system in the present invention is provided with data acquisition, input, processing, and output modules. It can not only achieve efficient data processing and visualization of detection results, but also push abnormal situations to the database and traffic personnel in a timely manner, and provide data for a specific time period before and after the occurrence of the anomaly when pushing. This design facilitates traffic personnel to comprehensively understand abnormal situations and take effective measures to deal with traffic anomalies in a timely manner, further improving the efficiency of traffic management, deepening the main beneficial effects from the practical application level, and better ensuring the safe and smooth operation of expressways. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is the flowchart of the method of the present invention;

[0074] Figure 2 is the schematic structural diagram of the system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0075] Example 1: As Figure 1 shown, a highway video detection method based on a large model according to the present invention includes the following steps:

[0076] S1: Obtain highway traffic monitoring camera data to construct a highway traffic monitoring video dataset;

[0077] S2: Fine-tune a video classification model based on the highway traffic monitoring video dataset and a pre-trained model. The specific steps are as follows:

[0078] S201: Randomly select N frames of images from the highway traffic monitoring video dataset to obtain N images;

[0079] S202: Use the pre-trained model to crop the size of each image to H×W, and input the N images into the model to extract features;

[0080] The specific method for extracting features in step S202 is as follows:

[0081] S202a: Combine the N images into a sequence;

[0082] S202b: Extract the inter-frame information thereof;

[0083] S202c: Extract the features of each frame image in the sequence;

[0084] S202d: After fully connecting the features of the inter-frame information, cascade them with the features of each frame image to obtain the features of the image;

[0085] S203: Input the features of each image into the classifier to obtain the probability of the type to which the image belongs;

[0086] S204: Use the cross-entropy loss function to perform backpropagation on the classifier to maximize the probability of the image in the video;

[0087] S205: Repeat steps S201 - S204 until the probability meets the preset threshold, and save the model parameters;

[0088] S3: Detect vehicles, roads, signs, etc. in the traffic scene, and judge the type of traffic anomaly according to the detection results;

[0089] Among them, in step S3, detecting vehicles, roads, signs, etc. in the traffic scene, and judging the type of traffic anomaly according to the detection results, the specific steps are as follows:

[0090] S301: Obtain the target features in the traffic scene, where the target features include the category, location, and spatial distribution of the target;

[0091] S302: Based on the target features, model the vehicles, roads, and signs in the traffic scene to obtain a traffic scene model;

[0092] Among them, the traffic scene model includes traffic scene features and traffic scene probabilities;

[0093] The traffic scene features are based on the target detection model. Input the images in the traffic scene into the target detection model to obtain the category, location, and spatial distribution of vehicles, roads, and signs in the traffic scene;

[0094] The traffic scene probabilities are determined based on the classifier. Input each image into the image classifier to obtain the probability of the scene to which the image belongs;

[0095] S303: Obtain the traffic anomaly type and traffic anomaly model based on prior knowledge;

[0096] Among them, the traffic anomaly model includes the traffic anomaly type, the abnormal features corresponding to the traffic anomaly type, and the space of the traffic anomaly type in the traffic scene;

[0097] The determination of the abnormal features corresponding to the traffic anomaly type is obtained by inputting the features of the traffic anomaly type existing in the traffic scene into the abnormal feature model to obtain the abnormal features;

[0098] The abnormal feature model includes two fully connected layers and an activation function layer. The features of the traffic abnormal type are input into the first fully connected layer for full connection to obtain the second feature. The second feature is then input into the second fully connected layer for full connection to obtain the third feature. The third feature is input into the activation function layer for activation to obtain the abnormal feature;

[0099] S304: Based on the traffic scene model, each frame of the highway traffic monitoring video dataset is detected to obtain the corresponding traffic scene;

[0100] S305: Based on the traffic anomaly model, obtain the abnormal traffic scenes existing in the traffic scene,

[0101] Among them, the specific method for obtaining the abnormal traffic scenes existing in the traffic scene in step S305 is as follows:

[0102] S305a: Construct an abnormal evaluation model for evaluating the probability of the existence of traffic abnormal types in each traffic area. The abnormal evaluation model is S = αP + β, where α is the weight, β is the deviation, and P is the probability of the existence of a certain traffic abnormal type in each traffic area;

[0103] S305b: Obtain the second weight of each traffic abnormal type, and evaluate the probability of the existence of each traffic abnormal type in each traffic area based on the abnormal evaluation model to obtain the score of the existence of each traffic abnormal type in each traffic area. Let the second weight of the jth traffic abnormal type be w j , then the score of the jth traffic abnormal type in the ith traffic area is S ij = w j (αP ij + β);

[0104] S305c: Screen the traffic abnormal type corresponding to the highest score in the scores as the traffic abnormal type of the abnormal traffic scene. That is, for the ith traffic area, the traffic abnormal type of the abnormal traffic scene is argmax j S ij ;

[0105] S305d: Determine that the current frame traffic scene has an abnormal traffic scene based on that the score of the traffic abnormal type in the current frame traffic scene of the traffic area is greater than the abnormal score threshold. The abnormal score threshold is 0.01;

[0106] S4: Based on the abnormal traffic scene, perform early warning processing, and give a warning about the traffic anomaly based on the type of the abnormal traffic scene;

[0107] Among them, the types of the abnormal traffic scenes include traffic abnormal types, the time and location where the traffic anomaly exists, and the recommended information for eliminating the traffic anomaly.

[0108] In the present invention, by constructing a highway traffic monitoring video dataset and fine-tuning a pre-trained model based on this to obtain a video classification model, and combining a series of operations such as object detection, modeling, and anomaly evaluation, it is possible to accurately detect targets such as vehicles, roads, and signs in traffic scenes, and accurately judge the types of traffic anomalies. Compared with traditional detection technologies, the accuracy of traffic anomaly detection is greatly improved, effectively solving the problem of low recognition accuracy of existing technologies in complex traffic scenes, and providing a more reliable decision-making basis for traffic management departments.

[0109] In an embodiment of the present invention, the loss function in the model training process adopts a linear weighted sum of a cross-entropy loss function and an adversarial loss function;

[0110] The cross-entropy loss function is: In the formula, C is the number of categories, y i is the true label, y ∈ {0, 1}, y is the annotation in the true traffic scene, and p i is the predicted probability;

[0111] The adversarial loss function is: In the formula, n is the number of traffic scenes with abnormal traffic scenes in the model training process, represents the probability of predicting that there is an abnormal traffic scene in the traffic scene;

[0112] In the detection process of the present invention, a linear weighted sum of a cross-entropy loss function and an adversarial loss function is used as the loss function for model training. This method enables the model to better learn the characteristics of normal and abnormal traffic scenes during training, further improving the adaptability and detection accuracy of the model to complex traffic scenes, thereby more accurately identifying traffic anomalies, further solving the problem of improving detection accuracy in the main beneficial effects, and reducing the occurrence of misjudgment and missed judgment situations.

[0113] In an embodiment of the present invention, the method for determining the traffic anomaly type in the traffic scene is: dividing the traffic scene into grid-like segments to obtain multiple traffic regions, and determining the probability of the corresponding traffic anomaly type existing in each traffic region based on the anomaly characteristics and traffic scene characteristics corresponding to each traffic anomaly type in the traffic anomaly type.

[0114] In an embodiment of the present invention, the method for determining the traffic scene model in step S302 is:

[0115] S302a: Based on the scores of abnormal traffic scenes existing in each traffic region, determine the score of abnormal traffic scenes existing in the traffic scene;

[0116] Among them, the score of the abnormal traffic scene in the traffic scene is: the score of the traffic anomaly type is obtained by traversing and adding the scores of each traffic anomaly type existing in each traffic area.

[0117] Let the total number of traffic areas be X, the probability of a certain traffic anomaly type existing in each traffic area be P, and the score of the abnormal traffic scene in the traffic scene is: In the formula, M is the number of traffic anomaly types;

[0118] S302b: Determine whether the score of the abnormal traffic scene in the traffic scene exceeds the preset score threshold. If so, the traffic scene is an abnormal traffic scene.

[0119] Embodiment 2: As Figure 2 shown, a traffic anomaly detection and alarm system based on a large model includes a data acquisition module, a data input module, a data output module, and a data processing module. The data acquisition module is connected to the data input module, the data input module is connected to the data processing module, and the data processing module is connected to the data output module;

[0120] The data acquisition module is used to obtain highway traffic monitoring camera data and save it in the database;

[0121] The data input module is used to load the traffic monitoring camera data saved by the data acquisition module into the large model, predict and process the traffic monitoring camera data based on the large model, and obtain the detection result;

[0122] Among them, the large model includes a video detection model, a traffic scene model, a traffic anomaly model, and an anomaly evaluation model;

[0123] The video detection model is used to detect the traffic monitoring camera data to obtain the traffic scene;

[0124] The traffic scene model is used to determine the traffic scene;

[0125] The traffic anomaly model is used to analyze whether there is an anomaly in the traffic scene and what kind of anomaly exists;

[0126] The anomaly evaluation model is used to judge whether there is an anomaly in each traffic area of the traffic scene and the score of the traffic area with an anomaly, and determine whether there is an anomaly in the traffic area based on the score.

[0127] The data output module is used to visualize the detection result and push it to the database and traffic personnel;

[0128] Among them, the database is used to store the traffic monitoring camera data collected by the data acquisition module and the detection result obtained by the data output module, and the traffic personnel are traffic police;

[0129] The data processing module is used to screen the detection information and traffic monitoring camera data;

[0130] Among them, the detection information includes the traffic monitoring camera number, traffic scene and detection time;

[0131] As another embodiment of the present invention, the screening of the traffic monitoring camera data is carried out in the following manner: based on the score of the abnormal traffic scene in the traffic scene, it is determined whether the traffic monitoring camera data is an abnormal traffic scene;

[0132] As another embodiment of the present invention, the screening of the detection information and traffic monitoring camera data is specifically carried out in the following manner:

[0133] Based on the detection result of the traffic scene corresponding to the traffic monitoring camera data, it is judged whether there is an abnormal traffic scene;

[0134] Among them, the traffic scene model is pre-constructed and constructed based on prior knowledge;

[0135] If there is an abnormal traffic scene, then the traffic scene, detection result and traffic monitoring camera data are screened out and pushed to the traffic personnel;

[0136] The content of visualizing the detection result is to visualize the abnormal situation of the entire traffic scene, including traffic abnormal features, traffic abnormal types, the duration of traffic abnormality, and location;

[0137] The specific content of pushing to the database and traffic personnel is: pushing the 10 seconds before the abnormal judgment, 10 seconds during the abnormality, and 10 seconds after the abnormal judgment ends to the database and traffic personnel;

[0138] When the traffic personnel analyze the abnormal traffic monitoring camera data in the database, if it does not constitute a traffic abnormality, the traffic personnel can return that the traffic monitoring camera data is normal;

[0139] If it constitutes a traffic abnormality, the traffic personnel can mark the traffic monitoring camera data as abnormal.

[0140] The traffic anomaly detection and alarm system in the present invention is provided with data acquisition, input, processing and output modules, which can not only realize the efficient processing of data and the visualization of detection results, but also push the abnormal situation to the database and traffic personnel in a timely manner, and provide the data of a specific time period before and after the abnormality occurs during the push. This design facilitates the traffic personnel to comprehensively understand the abnormal situation, take effective measures to deal with traffic anomalies in a timely manner, further improves the efficiency of traffic management, deepens the main beneficial effects from the actual application level, and better ensures the safe and smooth operation of the highway.

[0141] The embodiments disclosed in the present invention are preferred embodiments, but not limited thereto. Those of ordinary skill in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. As long as they do not depart from the spirit of the present invention, they are within the protection scope of the present invention.

Claims

1. A highway video detection method based on a large model, characterized in that, It includes the following steps: S1: Obtain highway traffic monitoring camera data to construct a highway traffic monitoring video dataset; S2: Fine-tune a video classification model based on the highway traffic monitoring video dataset and a pre-trained model. The specific steps are as follows: S201: Randomly select N frames of images from the highway traffic monitoring video dataset to obtain N images; S202: Use the pre-trained model to crop the size of each image to H×W, and input the N images into the model to extract features; S203: Input the features of each image into a classifier to obtain the probability of the type to which the image belongs; S204: Use the cross-entropy loss function to perform backpropagation on the classifier to maximize the probability of the image in the video; S205: Repeat steps S201 - S204 until the probability meets a preset threshold, and save the model parameters; S3: Detect vehicles, roads, signs, etc. in the traffic scene, and judge the type of traffic anomaly according to the detection results; S4: Perform early warning processing based on the abnormal traffic scene, and give a warning about the traffic anomaly based on the type of the abnormal traffic scene; Among them, the type of the abnormal traffic scene includes the type of traffic anomaly, the time when the traffic anomaly exists, the location, and the recommended information for eliminating the traffic anomaly.

2. The method for detecting highway videos based on a large model according to claim 1, wherein The specific method for extracting features in step S202 is as follows: S202a: Form a sequence of N images; S202b: Extract the inter-frame information; S202c: Extract the features of each frame of image in the sequence; S202d: After fully connecting the features of the inter-frame information, cascade them with the features of each frame of image to obtain the features of the image.

3. The method for detecting highway videos based on a large model according to claim 1, characterized in that In step S3, detect vehicles, roads, signs, etc. in the traffic scene, and judge the type of traffic anomaly according to the detection results. The specific steps are as follows: S301: Obtain the target features in the traffic scene. The target features include the category, location, and spatial distribution of the target; S302: Based on the target features, model the vehicles, roads, and signs in the traffic scene to obtain a traffic scene model; Among them, the traffic scene model includes traffic scene features and traffic scene probabilities; The traffic scene features are based on a target detection model. Input the images in the traffic scene into the target detection model to obtain the category, location, and spatial distribution of vehicles, roads, and signs in the traffic scene; The traffic scene probabilities are determined based on a classifier. Input each image into the image classifier to obtain the probability of the scene to which the image belongs; S303: Obtain the type of traffic anomaly and the traffic anomaly model based on prior knowledge; Among them, the traffic anomaly model includes the type of traffic anomaly, the abnormal features corresponding to the type of traffic anomaly, and the space of the type of traffic anomaly in the traffic scene; The determination of the abnormal features corresponding to the type of traffic anomaly is obtained by inputting the features of the type of traffic anomaly existing in the traffic scene into the abnormal feature model to obtain the abnormal features; The abnormal feature model includes two fully connected layers and an activation function layer. The features of the traffic abnormal type are input into the first fully connected layer for full connection to obtain the second feature. The second feature is then input into the second fully connected layer for full connection to obtain the third feature. The third feature is input into the activation function layer for activation to obtain the abnormal feature; S304: Based on the traffic scene model, each frame of the highway traffic monitoring video dataset is detected to obtain the corresponding traffic scene; S305: Based on the traffic anomaly model, obtain the abnormal traffic scenes existing in the traffic scene.

4. The method for detecting highway videos based on a large model according to claim 1, wherein The specific method for obtaining the abnormal traffic scenes existing in the traffic scene in step S305 is as follows: S305a: Construct an abnormal evaluation model for evaluating the probability of the existence of traffic abnormal types in each traffic area. The abnormal evaluation model is S = αP + β, where α is the weight, β is the deviation, and P is the probability of the existence of a certain traffic abnormal type in each traffic area; S305b: Obtain the second weight for each traffic anomaly type, evaluate the probability of each traffic anomaly type existing in each traffic area based on the anomaly evaluation model, and obtain the score of each traffic anomaly type existing in each traffic area. Let the second weight of the j-th traffic anomaly type be w j , then the score of the j-th traffic anomaly type in the i-th traffic area is: S ij = w j (αP ij + β); S305c: The traffic anomaly type corresponding to the highest score in the screening scores is the traffic anomaly type of the abnormal traffic scenario. That is, for the i-th traffic area, the traffic anomaly type of the abnormal traffic scenario is argmax j S ij ; S305d: Determine that the current frame traffic scene has an abnormal traffic scene based on the score of the traffic abnormal type in the current frame traffic scene of the traffic area being greater than the abnormal score threshold, where the abnormal score threshold is 0.

01.

5. The method for detecting highway videos based on a large model according to claim 1, characterized in that, The loss function in the model training process adopts the linear weighted sum of the cross-entropy loss function and the adversarial loss function; The cross-entropy loss function is as follows: In the formula, C is the number of categories, y i is the true label, y ∈ {0, 1}, y is the annotation in the true traffic scene, and p i is the predicted probability; The adversarial loss function is as follows: where n is the number of traffic scenarios with abnormal traffic scenarios during the model training process, represents the probability of predicting that there are abnormal traffic scenarios in the traffic scenario.

6. The method for detecting highway videos based on a large model according to claim 1, wherein, The determination method of the traffic abnormal type in the traffic scene is as follows: The traffic scene is divided into grids to obtain multiple traffic areas, and the probability of the existence of the corresponding traffic abnormal type in each traffic area is determined based on the abnormal features and traffic scene features corresponding to each traffic abnormal type in the traffic abnormal type.

7. A highway video detection method based on a large model according to claim 1, characterized in that, The determination method of the traffic scene model in step S302 is as follows: S302a: Based on the score of the existence of abnormal traffic scenes in each traffic area, determine the score of the existence of abnormal traffic scenes in the traffic scene; Among them, the score of the existence of abnormal traffic scenes in the traffic scene is: The score of the traffic abnormal type is obtained by traversing and adding the scores of the existence of each traffic abnormal type in each traffic area; Let the total number of traffic areas be X, and the probability of a certain type of traffic anomaly existing in each traffic area be P. The score of the abnormal traffic scenario in the traffic scenario is: where M is the number of types of traffic anomalies; S302b: Determine whether the score of the existence of abnormal traffic scenes in the traffic scene exceeds the preset score threshold. If so, the traffic scene is an abnormal traffic scene.

8. A traffic anomaly detection and alarm system based on a large model, which uses a highway video detection method based on a large model described in claims 1-7, characterized in that It includes: A data acquisition module for obtaining highway traffic monitoring camera data and saving it to the database; A data input module for loading the traffic monitoring camera data saved by the data acquisition module into the large model, and predicting and processing the traffic monitoring camera data based on the large model to obtain the detection result; A data output module for visualizing the detection result and pushing it to the database and traffic personnel. The database is used to store the traffic monitoring camera data collected by the data acquisition module and the detection result obtained by the data output module. The traffic personnel are traffic police; A data processing module for screening the detection information and traffic monitoring camera data. The detection information includes the traffic monitoring camera number, traffic scene, and detection time; The data acquisition module is connected to the data input module, the data input module is connected to the data processing module, and the data processing module is connected to the data output module Among them, the large model includes a video detection model, a traffic scene model, a traffic anomaly model, and an anomaly evaluation model; The video detection model is used to detect traffic monitoring camera data to obtain a traffic scene; The traffic scene model is used to determine a traffic scene; The traffic anomaly model is used to analyze whether there is an anomaly in the traffic scene and what kind of anomaly exists; The anomaly evaluation model is used to judge whether there is an anomaly in each traffic area of the traffic scene and the score of the anomaly in the traffic area, and determine whether there is an anomaly in the traffic area based on the score.

9. The traffic anomaly detection and alarm system based on a large model according to claim 1, characterized in that, The screening of the traffic monitoring camera data is carried out in the following way: based on the score of the abnormal traffic scene in the traffic scene, it is determined whether the traffic monitoring camera data is an abnormal traffic scene.

10. A traffic anomaly detection and alarm system based on a large model according to claim 1, characterized in that, The screening of the detection information and the traffic monitoring camera data is specifically carried out in the following way: Based on the detection result of the traffic scene corresponding to the traffic monitoring camera data, it is judged whether there is an abnormal traffic scene; If there is an abnormal traffic scene, the traffic scene, the detection result, and the traffic monitoring camera data are screened out and pushed to the traffic personnel; When traffic personnel analyze the abnormal traffic monitoring camera data in the database, if it does not constitute a traffic anomaly, the traffic personnel can return that the traffic monitoring camera data is normal, and if it constitutes a traffic anomaly, the traffic personnel can mark the traffic monitoring camera data as abnormal.

Images